I feel so, so very sad for Keith Vaz, his family and children. This weekend, allegations appeared in The Sunday Mirror about secret assignations with young male prostitutes. Since then, the MP has been talked about, attacked, mocked and shamed in the media and by fellow parliamentarians.

No law appears to be broken, but many hearts must be. Interestingly, on the Wright Stuff, (Channel 5) where I am a panellist all week, callers were sympathetic and thought Vaz's private life was his business. However emailers to the programme were unforgiving.

I've known Keith Vaz since 1987 when he became the first post-war Asian MP to be elected since to Parliament. This was when Dianne Abbot, Paul Boateng and Bernie Grant also won seats. For us Britons of colour, this was an emotional moment, an affirmation that we were here to stay and could aim high. British Asians, even those who did not like or esteem Vaz, welcomed his hugely symbolic electoral win.

Here are some barely known facts: in the late Victorian and early Edwardian years, an all white electorate voted in three Asian MPs. Dadabhai Naoroji became the Liberal MP for Finsbury Central in 1892 and three years later, Sir Mancherjee Bhownagree, a Tory, won the seat in Bethnal Green. In 1922 Shapurji Saklatvala, a popular politician and member of the Communist Party, was elected to represent Battersea North. After 1923, our parliament turned all white till the eighties. In 1983, Anglo-Indian Tory, Jonathan Sayeed beat Tony Benn and became the Tory MP for Bristol East, but he did not describe himself in terms of his ethnicity. Vaz, in contrast, embraced his Asianness yet refused to be ghettoised, a complicated act.

Keith Vaz
Labour party MP Keith Vaz leaves his home in north west London Toby Melville/ Reuters

There have been some scandals and questions asked about him on the political grapevines. He undoubtedly has panache and mastery of the dark arts. Over the years, I have both praised and criticised him. He has consistently stood up for various minorities and at times, boldly taken on the powerful executive. But he has encouraged some unwholesome community politics in his Leicester East constituency, has many unsavoury rich Asian acquaintances and did not come out well in the MP's expenses revelations.

Nine years ago, Vaz became chairman of the influential Home Affairs Select Committee and he changed. There was gravitas now, toughness too, real commitment to difficult issues and progressive values. For example, he recently used the Freedom of Information Act to find out where vulnerable Syrian refugees were being housed by the Home Office and earlier invited in a wide range of people – including Russell Brand - to explore our useless drug policies. He won respect. And now this happens.

I don't think he can carry on as chair of the Home Affairs Select Committee. He is too compromised. The position brings with it dignity and respect which, if broken, cannot be rebuilt fast. There will henceforth always be a suspicion that the committee sessions on prostitution and poppers were influenced by his own sexual proclivities. But this is not a simple tale of a scoundrel politician. There are several other aspects to consider.

So terrible are the prejudices against gays and bisexuals in most Asian families and communities, that perhaps Vaz could never have been able to be honest about his sexual preferences.

His right to right to privacy has been breached. The high and low have that fundamental right. In our times, the very idea of privacy or confidentiality are regarded as outdated or forms of censorship. Would you like the world and his dog to know about what you get up to in your own time and space? Should all successful and or famous people have to sign away this precious right? Absolutely not. We do not own them. If Vaz had pronounced on family values and marriage, or voted against gay rights, this story would have moral legs. As far as I know, he has been liberal in his views and voting choices.

Then there are the unspoken cultural attitudes that lie beneath the surface. So terrible are the prejudices against gays and bisexuals in most Asian families and communities, that perhaps Vaz could never have been able to be honest about his sexual preferences. Though in recent years, a small number of Asian men and women have come out, most do not. The cost would be too high, even in modern, middle class families.

Most marry, have kids and meet parental expectations. Many learn to suppress their sexuality or express it secretly. Imagine the fears and fatigue of those trying to live overground and underground lives, trapped by the terrors of secrecy. Vaz represents large numbers of Hindus, Sikhs and Muslims, who must have faced those struggles.

What's happened is a tragedy not some gleeful farce. I hope his Asian constituents, friends and family stand by him. And I hope Vaz does come back from this grubby scandal. This pioneer made history. Do not rush to condemn him.