The world has been shocked, but not surprised, at the allegations of torture perpetrated by the CIA. The Senate report has described brutal practices: interrogations lasting for days, prisoners forced to stand on broken legs, sexual assault, and sleep deprivation of over 180 hours. It's emerged that the two contractors responsible for conducting interrogations earned over $81 million doing so.

The CIA are now in full counter attack. Former directors, administration figures from the time and agency officials claim in well placed op-eds and interviews that in fact, the methods were necessary, proportionate and directly helped in several investigations. The former directors described the report as "a missed opportunity to deliver a serious and balanced study of an important public policy question". Even George W Bush made a very rare statement defending the agency.

Bush was right: the CIA does employ the best and the brightest, and those recruits remain excellent intelligence officers. Many of those same officers would have smiled at the irony of the former president's comments.

Bush and his administration misused the agency once proudly run by his father, turning it into a paramilitary police force abroad and a paralyzed bureaucracy in Washington. Bush casually pronounced a political death sentence upon the CIA in 2004 when he said that the agency was "just guessing" about the course of the war in Iraq. No president has ever publicly dismissed the CIA in that way.

Since the founding of the agency, it has consistently been let down by incompetent leadership – both structures and personalities. The organisation has failed to reform nor learnt any of the lessons of the past. Its centrality in the American government ended with the dissolution of the office of director of central intelligence in 2005. Its status today is even further diminished.

Now the CIA must reform if it is to survive. The task will take years. Almost every president, almost every Congress and almost every director of central intelligence since the 1960s have proved incapable of grasping the mechanics of the CIA. Most have left the agency in a worse state than they found it. This week, the agency is back where it began over sixty years ago: in a state of disarray.

Among the occasional successes, the agency's history is littered with failure and ignominy. The Bay of Pigs, the failed coup d'etat in Indonesia, the inability to predict events in the Soviet Union, and the debacle around 9/11 and its aftermath will forever remain stains on its reputation. The US squandered thousands of lives and billions of dollars in Iraq. It is the price Americans pay in blood and treasure when the CIA gets it wrong.

The beginnings of the failure that we have seen dissected in the past couple of weeks can be traced to the late 1990s. According to historian Tim Weiner, in 1998, America's spymasters warned the White House in secret that the US would suffer "a catastrophic, systemic failure" of intelligence unless the nation overhauled the ways in which it gathered information on the rest of the world. That failure came five years later in 2003, when the CIA's false reports on Iraq's weapons of mass destruction gave the president and the Pentagon the moral basis to begin a pre-emptive war.

Do Americans deserve an intelligence service that misreads existential threats of terror, relies on information wrung from torture in secret prisons, and derives unchecked power through secret presidential orders? These are the questions that President Obama has failed to address.

The president still has time to reform the agency. He must give immediate and sustained attention to the heart and soul of the CIA, the spies of its clandestine service, and the analysts of its intelligence directorate. While the agency has had an almost perfect record of resisting fundamental changes suggested by presidential committees and "blue ribbon panels", it will try and satisfy a president's command. The early optimism that Obama might have been the president to affect such reforms dissipated as quickly as his promise to shut down Guantanamo Bay. Like Eisenhower or Kennedy before him, natural suspicion to covert action quickly morphed into adulation.

The President, Secretary of Defence and the Secretary of State need strategic intelligence – the information essential to guiding foreign policy. The CIA has not provided it. If it's to become the organisation its officers demand and the world deserves, the CIA must reform.

Ross Cypher-Burley is a former spokesman for the British ambassador to Israel, and now works for strategic communications firm Portland. You can follow him on Twitter here.