President Obama has been urged to close Guantanamo Bay in an open letter signed by 25 international organisations.
The NGOs, which include Amnesty International, Reprieve and Human Rights Watch, have asked the US president to fulfil the promise he made in 2009 to shut the prison camp.
Lawyers for detainees said they believe 166 men were still held there indefinitely and without charge.
Almost all are on hunger strike, including British man Shaker Aamer, who has been detained in Guantanamo for 11 years.
The hunger strikes were prompted because even though most of the prisoners have been cleared for release, attempts to transfer them from Guantanamo Bay have been abandoned by the Obama administration.
"If ever there were a moment to act upon the promise you made over four years ago to shutter the prison, it is now," the letter said.
"For several weeks, major news outlets, attorneys for the prisoners, and even military officials have been reporting that there is a large hunger strike among the men.
"The situation is the predictable result of continuing to hold prisoners indefinitely without charge for more than 11 years. We urge you to begin working to transfer the remaining detained men to their home countries or other countries for resettlement, or to charge them in a court that comports with fair trial standards."
The authors note that on 20 March, Gen John Kelly of the US Southern Command said prisoners were "devastated" to find out Obama had decided to back away from his promise and that indefinite detention was having a psychological impact on them.
The letter continued: "The United Nations High Commissioner for Human Rights, Navi Pillay, again stated that the continued operation of Guantánamo Bay remains a clear breach of international law and reiterated that it should be closed.
"While we stand ready to support the administration's efforts to close Guantánamo Bay in a manner consistent with its international legal obligations, this problem demands your presidential leadership. We urge you to act now."
Clive Stafford Smith, Aamer's lawyer, said his client was suffering physically because of the strik.
"His hand is shaking permanently," he said, adding that Aamer has lost around 30lb since the strike began.
Cori Crider, legal director of Reprieve and lawyer for several of the men, said: "Some of my clients have actually read [Obama's autobiography] Dreams From my Father, so excited were they by the election of President Obama after his campaign promises.
"But by refusing to go to bat for his policy and keep his pledge to close Guantanamo, Obama dashed the men's hopes almost as soon as he built them. This wasn't intentional - but it was cruel.
"Hunger strikes are about the only option these men have left to protest indefinite detention. It's time for the president to show some spine, and appoint a White House official to get cleared men home."