Mark Carney has told the Treasury Select Committee that splitting up Britain's biggest banks would not necessarily make the country's banking sector more competitive.
"Just breaking up an institution doesn't necessarily create a more intensive competitive structure," said Carney.
The remarks by the governor of the Bank of England contradict Labour leader Ed Miliband's proposals to cap the size of major British banks and force them to sell off branches to smaller rivals.
Andrew Tyrie, head of the committee, also asked Carney if he agreed with the findings of the Vickers Commission into banking that bonus cap set at 100% of base salary was crude.
Carney replied "absolutely", adding fuel to debates carrying on about shareholder (government) limits on bonuses to Royal Bank of Scotland bankers.
At prime ministers questions, Ed Miliband and David Cameron clashed over the pay and bonus culture of the UK's banking sector, with a focus on the partially nationalised lender RBS.
In their exchanges, Cameron pledged to limit the overall basic pay that employees at RBS received but refused to back a cap on bankers' bonuses.
Miliband said RBS should not pay bonuses worth more than 100% of salaries, while there is a "cost-of-living crisis".
Meanwhile the shadow Treasury secretary, Chris Leslie, speaking on BBC Radio 4's Today programme, echoed the arguments of his Miliband.
"It is part of the old economic construct that we have this high-rolling bonus culture. We have got to move to a sustainable, professional, stable form of banking if we are going to earn our way out of this cost-of-living hole we are in and if we are to help the small businesses get the finances they need to build a better economy. And we have got to have more competition in the banking sector," he said.
The European Commission installed a new limit on bankers' bonuses, which means extra pay will be capped at 200% of their salary.
The law, which comes into force from January 2014 and will apply to bonuses paid in 2015.