The UK government has branded its contract with benefits testing firm Atos as a "mess".
But Mike Penning, Minister of State of Disabled People, ruled out terminating the French healthcare company's agreement with the Department for Work and Pensions and blamed the previous Labour government on the situation.
"Since this administration came into power we have been warning Atos to improve their situation and improve its service," Penning told the House of Commons.
He added: "We are coming to agreements with them to make sure that has been done – some of which have not come into fruition.
"But it is absolutely wrong that if we terminate [the agreement] as the opposition has said, the financial consequence of that [move] would fall on the taxpayer not on Atos."
The minister also criticised the IT company for disclosing that it is seeking to withdraw early from its government contract - assessing whether benefits claimants are fit to work or not - which is due to run to August 2015.
"Whilst we have seen some improvements with Atos quality, we are still in ongoing commercial discussions," Penning said.
"And, unlike Atos, we respect commercial confidentiality. It is important we get the procurement right, unlike the previous administration, which is why we are in this mess today."
Anyone claiming Employment and Support Allowance (ESA) will have a Work Capability Assessment (WCA) to assess their capability for work.
The controversial tests were introduced in 2008.
The government said between October 2010 and spring 2014 anyone receiving Incapacity Benefit, Severe Disablement Allowance and Income Support paid on the grounds of illness or disability will also have a WCA.
"Despite on-going discussions, we will not walk away from a front-line service," said an Atos Healthcare spokesperson.
"Our total focus remains on delivering the services we are contracted to provide in a professional and compassionate way until a new service begins."