Lord Fink has confronted the leader of the opposition Ed Miliband over his allegation at Prime Minister's Questions that the peer was engaged in "tax avoidance".

The Labour leader has been accused of making "untrue and defamatory" remarks about Lord Fink, the treasurer of the Conservative party, as Miliband and David Cameron leaders traded blows at PMQs over the HSBC tax avoidance scandal.

Lord Fink was a party treasurer under Cameron and has given £3m to the Conservative Party. He served as the CEO of the Man Group, a hedge fund, from 2000 to 2007. At one time he was reported to be the highest paid executive in the FTSE 500.

In a interview with the BBC, Lord Fink, repeatedly challenged Miliband to repeat claims that had avoided tax using a Swiss bank account outside the House of Commons, where he is protected from libel laws by Parliamentary privilege.

In a letter published earlier today (11 February), Lord Fink wrote: "These are the facts: I was posted by the Man Group to run their office in Switzerland for four years from 1996 until 2000.

"During this time I had need of a local bank account to do simple things like receive my Swiss Franc salary and pay grocery bills.

"As I already banked with HSBC in London, I set up an account with HSBC.

"I subsequently set up an account with Credit Suisse as they had a branch much closer to my home and office.

"I submitted tax returns in both Switzerland and Britain showing my revised tax status, which was accepted by the Inland Revenue.

"I find it extraordinary that you have made claims against me that are without foundation or without contacting me.

"I challenge you to repeat your allegation outside the House of Commons – or to withdraw it publically."

'Serious allegations about Lord Fink'

A Labour spokesman said: "There are serious allegations in The Guardian about Lord Fink including his complex arrangements to minimise tax.

"He still has not justified the reasons why he made these arrangements. He should do so.

"David Cameron must explain whether he is happy to have appointed Lord Fink as Treasurer. Then it will be up to the public to judge."

Lord Fink's estimated net worth is over £200m.

Miliband claimed seven Tory donors revealed in the leaked files have given nearly £5m ($7.6m, €6.7m) to the Conservative Party.

The Labour leader also alleged there was a "revolving door" between Tory headquarters and the Swiss branch of HSBC.

During PMQs, Miliband said: "Let's take Stanley Fink who gave £3m to the Conservative Party. He actually appointed him as Treasurer of the Tory Party and gave him a peerage for good measure.