Theresa May has been urged to sack the chancellor Philip Hammond and leave Boris Johnson in place as foreign secretary.
Allies of Johnson say that despite calls for May to assert her authority and demote him, the foreign secretary would simply refuse to go if pushed.
Nadine Dorries is the only MP to publicly call for Hammond to go while Brexit-supporting MPs are discussing concerns about the chancellor's commitment to Brexit.
One Cabinet minister told the Telegraph that: "He (Hammond) has completely failed. He has not given her (May) any domestic announcements that she can sell.
"He is miserable, he talks people down, he is making Brexit hard. He just saps everyone's self-confidence."
Meanwhile, questions remain over whether the prime minister even has the authority to demote Johnson, who has been criticised for his intervention in the Brexit process.
However one of his allies said that sacking Johnson would go down "like a bucket of cold sick" with Brexit voters, while another minister critical of his interventions said his removal would "undermine public confidence in Brexit".
Speculation mounts over whether there would be a cabinet reshuffle towards the end of October after Theresa May managed to see off a coup of around 30 Tory MPs who wanted her out.
Johnson's allies said that removing him would be "insane". One Tory minister pointing out that Andrea Leadsom threatened to resign after being sacked as environment secretary.
The minister said: "Let's say she tries to move Boris to defence secretary. Leadsom said no thank you, and Boris carries significantly more influence. He'd just say no - what is she going to do about it?
"There's a stench of death emanating from Downing Street. If you were a Brexiteer you would be worried. You'd look around the Cabinet table and see Remainers occupying the key positions."
Meanwhile the Times reported that the prime minister will tell European leaders that the UK would make no more concessions on Brexit unless Brussels was ready to compromise on her demands set out in her speech in Florence.
In a statement to the Commons on Monday (9 October), she will say that for Brussels the "ball is in their court" and that both sides "needed to prove the doomsayers wrong" with Brexit talks scheduled for next week's European Council meeting.
A Treasury source told the Guardian: "Britain will leave the European Union, the customs union, and the single market in March 2019. The Treasury is working hard to make sure that can happen."