Cuba's government is holding back on its agreement to release some of the 53 people the US government has said would be freed as part of the warming relations between the two nations.
The prisoner release was a key part of the agreement between Cuba and the US to renew diplomatic relations, reached at the end of last year.
Yet Havana was holding back on releasing all of the detainees, Reuters reported, citing a US congressional aide.
"We've been told that the Cuban government has agreed to release all but several of the political prisoners on the list," the aide said, as quoted by Reuters. "The government in Havana believes that the smaller group has committed acts of violence."
The White House has not named the prisoners that it said would be freed under the deal that was sought to end decades of hostility between Washington and Havana.
US critics of the thaw in relations have criticised the White House's failure to release the full list of names and have urged President Obama to reconsider engaging in formal talks with Havana.
The US government confirmed on Tuesday 6 January that some of the prisoners had been released and that it hoped to see the rest on the list released soon.
US and Cuban officials are set to meet in Havana later this month.