I know and like Baroness Sayeeda Warsi, but have never concurred with her right-wing views. We were two Muslim women on opposite sides of British politics. That changed on Monday (20 June), when she left Brexit and joined the Remain side in the EU referendum. She was repelled by the xenophobic tendencies of those campaigning to leave the EU, such as the anti-immigration Ukip poster that demonised refugees and asylum seekers from Syria. Brexiters tried to discredit Warsi; racist trolls assailed her in the most grotesque way, thus proving her right.

On that same day, with a trembling voice, Stephen Kinnock MP spoke in parliament where MPs had been recalled to remember the savagely murdered Jo Cox. Rhetoric, he warned, had consequences. He meant for migrants, racially different Britons and refugees who are now seen as the enemy within. Brendon Cox has since said his wife Jo was deeply worried about the hatred that had been whipped up.

On Tuesday I was contacted by Albert Persaud, director of a prestigious psychiatry research centre. Doctors in his team are deeply worried about the breakout of xenophobia. In another email, Sheila Melzak, a consultant child psychotherapist who works with young asylum seekers and refugees, expressed deep worry about her young clients, who were being bullied and targeted.

At a conference on Tuesday, a tearful Polish drama lecturer described to me how hurt and humiliated she had felt over these last few months. On Wednesday, Professor Carmen Molina-Paris, a Spanish immigrant, expressed her dismay in a letter to the Guardian: 'I came to this country in 2001, to carry out research at the University of Warwick. Back then, I believed this country was special. It was tolerant and open to people like me...Now I feel vulnerable and unwelcome'. She doesn't speak Spanish to her kids in public, she is afraid. She blames the 'lacerating language' used by some Brexit champions.

Finally, on Wednesday morning a black British friend, a talented musician, was abused and beaten up in a London park by five men. They told him to 'get back on the boat and fuck off'. He was born here, so too his parents, both retired NHS workers. Though I am used to getting vicious missives, they have got markedly nastier recently.

Brexit leaders have opened up the sewers of hatred...Those who hold anti-immigrant views are not all are racists, but too many are.

Brexit leaders have, predictably, denied all responsibility for the state we find ourselves in. The truth is that their anti-immigration campaigns have opened up the sewers of hatred. The rats are out spreading the plague of racism.

Those who hold anti-immigrant views are not all are racists – but too many are. Their arguments against immigration are veils that cover sickening hostilities and prejudices. We do not, for example, hysterically discuss the numbers of Americans living in the UK. There are more of them here than there are Jamaicans or Lithuanians. Here are the actual figures: 187,000 US-born, 136,000 Jamaican-born and 137,000 Lithuanians. Think about that.

I too want our country back, the country that brought us the Olympics, that Jo Cox represented, open, compassionate, internationalist, principled. If we leave the EU, we will never get that nation back

In the past five years an orthodoxy has been pushed by powerful right-wing politicians, journalists, policy wonks and other establishment figures. We were commanded by people like Nicolas Soames, Frank Field, Andrew Green of Migration Watch, Nigel Farage, Theresa May et al not to link the discourse on immigration with racist attitudes. Worse still, the good guys, those who should have argued ferociously against these injunctions, were too cowardly to do so.

This January, I was on Question Time in Stamford, Lincolnshire, where high Eastern European migration is causing anxieties – some understandable. Poverty and fast social change are hard at the best of times. And these are the worst of times because of biting austerity measures and deep pessimism about the future.

Uncontrolled jingoism and anti-immigrant invective has been flooding the country ever since Nigel Farage launched Ukip... The BBC gave Nigel Farage power and credibility

However, the majority of the people in the audience were fair and generous to those from elsewhere. The majority wanted us to take more refugees from Syria. They were more empathetic and principled than most MPs in the Commons. Labour front benchers and the Lib Dems should have spoken up before now to condemn the uncontrolled jingoism, the anti-immigrant invective that has been flooding the country ever since Nigel Farage launched Ukip. His party has not won a single seat in parliament, yet Farage has been on Question Time more than 20 times. The BBC gave him power and credibility. His malign influence was extended by the national broadcaster and others who should have known better.

I say it again. Brexiters are not all racist, but their rhetoric has make the air foul with the sour breath of prejudice. They say they want their country back from 'Johnny Foreigner', as AA Gill put it in the Sunday Times.

I too want our country back, the country that brought us the Olympics, the country that Jo Cox represented, open, compassionate, internationalist, principled. If we leave the EU, we will never get that nation back. That is the stark choice before us.