Ken Livingstone has defended comments he made in the wake of the Naz Shah scandal, where he claimed there was no anti-Semitism in the party before adding that Adolf Hitler was a Zionist.
The former London mayor is currently facing calls for his suspension from a handful of Labour MPs, including John Mann, who confronted Livingstone face to face for his views over comments made by Shah that Israel should be 'relocated' to the US.
"I said in 47 years in the party I have never heard anyone say anything anti-Semitic. In a party of half a million people you are bound to have a handful of mad anti-Semites and a handful of racists," Livingstone told Andrew Neil on the Daily Politics.
"There is not a problem, you're talking about a handful of people in party of half a million," he added.
Despite facing a backlash from within the party, Livingston suggested the issue would blow over and maintained there would be no problem with him remaining on Labour's National Executive. He added that the news had erupted over 24 hours and that people would calm down after they realised what had been said.
"I simply said what I believe to be true, that Naz is not anti-Semitic," he said.
"I am not an apologist for anyone who makes anti-Semitic comments, they will be dismissed from the party.
"I am saying there is a difference between anti-Semitism and someone who criticises Israel."
Shah is now facing an investigation into the comments she made prior to her becoming an MP, and has had the whip withdrawn in the wake of the scandal, which was initially exposed by political blogger Guido Fawkes.
Liberal Democrat leader Nick Clegg, who appeared with Livingstone on the show, said there was a problem with anti-Semitism on the left, explaining: "There is something on parts of the left in British politics at the moment that enter into an incredibly intellectually contorted justification that's starting to link Hitler and modern politics."
Labour has dismissed several members in recent weeks in an attempt to crack down on anti-Semitism.
When asked about his comments on Hitler being a Zionist, Livingstone said: "It's a simple truth, and that was Hitler's policy when he first came to power, to move Germany's Jews to Israel... I denounce that," he said.
"Then why bring it up," Neil asked.
"I'm asked a question, I answer it," Livingstone said
"I'm not raising these points, I was planning to have a nice quiet morning in the garden until I was descended on by all these journalists asking is this true, is that true. I would have been much happier doing the garden," he said.
His interview came as the Campaign Against Antisemitism called for Livingstone's immediate dismissal.
Gideon Falter, chairman of the group, said: "The Labour party must expel Ken Livingstone. Today he has claimed that Hitler was a Zionist and that anti-Semitism is not racism.
"Enough is enough. He should not be suspended, he should be expelled today. He is a hardened politician who has spent his political career accommodating anti-Semitic extremists and making anti-Semitic gaffes.
"Jeremy Corbyn should understand that zero tolerance for racism is all or nothing, and it is time for Ken Livingstone to be banished or for Corbyn to stop pretending to oppose racism."