"Check your privilege" is social media's favourite put-down to anyone arguing their case from a position of inherited advantage – such as being a white male in the workplace. Certainly, white men still dominate commerce, but is it fair to label all white males inherently privileged?

I'd argue that white men can suffer discrimination – even prejudice – in the workplace every bit as acute as those experienced by ethnic minorities and women. The bias comes in three forms, in my view, though none are without their controversies.

Least controversially is the discrimination faced by white males based on class. Men from working class backgrounds are paid less for the same job and advance at a slower pace than their posher rivals. And while this is also true with working class women, class immobility is getting worse for men while improving for women, perhaps because men are less adept at masking their origins.

Those with regional accents face mockery and put downs in a professional environment that undoubtedly translates into curtailed advancements when it comes to promotions and pay. Men more so than women? Yes, because mocking women for their backgrounds can increasingly be labelled sexist bullying. Working class men, meanwhile, are expected to take the abuse in good humour, even when they know it negatively impacts more serious assessments of their capabilities.

I expect many will dismiss my gripes as educational – we comprehensively-educated grunts being less qualified. Yet my professional qualifications and experience matched those of my privately-educated boss in the media job I held for most of the 1990s: it was his classical references and Latin-laced put downs I struggled to match – both critical attributes for a job in financial journalism, he considered.

Junior men are invisible

Yet class clearly cuts across gender, which cannot be said for my next gripe. And that's the very particular prejudice experienced by young men in the workplace. Thanks to the Harvey Weinstein -related scandals, I venture onto thinning ice but – in a patriarchal office run by senior males – there's one demographic that can struggle to get noticed for either the right or wrong reasons: young, junior, males.

Young women getting the wrong sort of attention hides the fact that, more normally, young women win more of the right sort of attention than young men. Certainly, they earn more and are generally more trusted with responsibility from a much younger age than male graduates in (or applying for) equivalent jobs.

Of course, much of this will be on merit. Girls certainly mature quicker than boys and there's no reason why this shouldn't carry on into the post-graduate workplace. Yet there's also no doubt that some of this is not down to mental maturity – or a willingness to be immediately diligent (while many men are still mentally exploring their options). Men in their 20s also look younger than their female contemporaries, which can impact judgements regarding their capabilities.

But it goes beyond both attitude and appearance – hence that thinning ice. Now in my 50s, I hobnob with people at the peak of their professions – usually men and women also in their 50s. And I cannot help noticing that important people of both genders prefer the company of young women to young men.

For instance, I've recently become involved with a think tank. This involves attending lots of events made up of the great and the good. Nearly all the important people in the room are over 50, with a 65:35 split male to female (70:30 on occasion). Yet the remaining 10-15 percent of the room – made up of the younger juniors – is heavily biased the other way: with nearly all the younger attendees female. What's more, they're usually of a certain class and nearly always "well presented" (to borrow the artful euphemism used by recruiters).

Event after event had the same bias, to the point I eventually quizzed one of the few young man I could find.

"They actually invited my colleague Charlotte but she couldn't make it," he claimed.

Accepted, there are industry biases here, with media and finance (my two industries) potentially skewing my view. And there's also the frivolous undercurrent to the invites. Being "table totty" (as I heard it described at a recent debate) must be frustrating, I agree. But so is going home uninvited, knowing your female colleagues can put their case for advancement while you microwave a ready-meal.

Danny DeVito
Danny DeVito is 5ft tall. Reuters

Short man syndrome

Which brings me to the most definitive form of workplace discrimination visited upon white men: or at least upon short men.

Height discrimination is the great unstated prejudice in the workplace, and is almost exclusively a male concern. Studies show that short men are paid less, promoted less and more likely to be unemployed than their taller peers.

Much of the bigotry is unconscious but no less savage for that. Otherwise rational, fair-minded and accomplished people happily denigrate professional men on the basis of stature. Just look at the continual abuse heaped upon House of Commons Speaker John Bercow for his 1.68m height – insults that would be denounced if for any other physical attribute, as the Speaker himself has pointed out.

Chelmsford MP Simon Burns called Bercow a "stupid, sanctimonious dwarf", which was viewed as rude but allowable. Any equivalent racial or gender-based epithet, meanwhile, would have seen Burns heavily censured as a result.

The pseudo-condition "short man syndrome" is banded around and applied to almost all males of below-average height (1.75m in the UK), no matter how they behave. Strangely, I've noted this accusation is most-often made by women, who should surely have a more sympathetic understanding of the likely behavioural changes caused by being judged on your physical appearance rather than your words and deeds.

Winning through

Yet words and deeds eventually win through. I've suffered all three indignities and have learnt to see them in their proper context: shortcuts for judging my credibility. Historically, tall, posh, older men were judged as the most credible cohort of all, which is why so many mediocre examples hold positions of power while others struggle against their own glass ceilings. The rest of us – including white males not in that exclusive club – have to look and act more credibly and work doubly hard to win through. Over the long run, however, such diligence and professionalism works to our advantage.

And we can always be comforted by the thought that – as with the Burns and Bercow exchange above – those resorting to prejudice in order to put-down their rivals are revealing far more about themselves than their targets.

Robert Kelsey is the best-selling author of a series of "anti-self help, self help" books, including What's Stopping You?, Get Things Done and The Outside Edge.