Serco has been ordered to pay £68.5m to the government because it overcharged taxpayers on a contract to carry out the electronic tagging of criminals.
It follows the rejection of a proposed £24m compensation payment by rival outsourcing giant G4S in relation to the same overcharging scandal.
Both firms are under investigation by the Serious Fraud Office (SFO) on allegations they charged the Ministry of Justice (MoJ) for electronic tagging work that was never carried out, such as on dead or still-incarcerated criminals.
"It's good news for taxpayers that Serco have agreed to recompense £68.5m for overcharging," said Francis Maude, minister for the Cabinet Office.
"We are confident that the company is taking steps to address the issues which our review has identified."
Alastair Lyons, non-executive Chairman of Serco, said the company is "very pleased to be making strong progress in further rebuilding the confidence of our UK government customer".
"Serco's cooperation with the intensive series of audits and reviews, alongside the significant steps we are taking as part of our renewal programme, demonstrate our commitment to this," he said.
"The contract issues that were identified should never have happened and we apologise unreservedly for them. We are doing everything in our power to make sure that such issues cannot reoccur anywhere in our business around the world."
Both G4S and Serco have also been prevented by the MoJ from bidding for probation contracts, though they will be able to offer support to successful bidders.
Their rival Capita has taken over both of the electronic tagging contracts in the interim. Capita is also the preferred bidder for a new contract due in 2014 to upgrade and modernise electronic tagging with the use of satellite systems.
Several Serco and G4S executives have quit in the scandal's wake.
Christopher Hyman walked out as chief executive of Serco in October. The firm's UK and Europe chief executive Jeremy Stafford stepped down a month later. G4S lost Richard Morris, UK and Ireland chief executive, in October.
An internal review at security giant G4S has found that it did overcharge British taxpayers on an electronic tagging contract with the Ministry of Justice by billing for work it had not carried out.
As a result of the review, carried out by law firm Linklaters, G4S offered to pay back £24.1m to the MoJ for incorrect billings between 2005 and 2013 on tagging of criminals it did not carry out.
G4S said that the review, which trawled internal documents and emails, had "not identified any evidence of dishonesty or criminal conduct by any employee of G4S". The investigation cost G4S £2m. The MoJ rejected G4S's offer of compensation.