It's time to crack open the Brie and raise a celebratory glass of olive oil. For according to the National Obesity Forum, butter, cheese and greasy fry ups won't make you fat after all but can be part of a healthy weight-loss regime.

In a controversial report, the charity says the official obsession with low-fat food over the last few decades has been "disastrous" for public health. Rather than worrying about cheese, cream and bacon rind, we should be cutting down on carbs and sugar, it says.

Just in case we haven't got the message, spokesman Dr Aseem Malhotra tells us: "Eat fat to get slim, don't fear fat, fat is your friend. It is now try time to bring back the fat." It sounds more like an advertising pitch from the lard industry than a reasoned medical argument, and not surprisingly it's come under fire from the weight of the entire health establishment.

Public Health England, the British Heart Foundation, and a bevy of academics have lined up to accuse the charity – and co-authors Public Health Collaboration – of cherry picking evidence and misquoting science.

While it's always tempting to side with Davids rather than Goliaths in these sorts of public spats, this time the establishment has a point. This obesity report doesn't just contain some flabby thinking, it is potentially dangerous.

I should declare an interest. I'm part of the obesity epidemic sweeping the Western world and I know perfectly well why I'm fat. It's not metabolic, it's not a disease, and it's not because I'm large-boned. It's because I eat too much and don't do enough exercise.

And I'm not alone. In a nation where food is available round the clock, portions are getting bigger, and it's getting easier to do nothing but stare at screens all day, a quarter of British adults are obese.

Clearly the official messages of the last few decades have failed. Being told to eat five a day, cut down on fat and eat less sugar hasn't worked for me or millions of fellow fatties. It's partly this failure of public health messages that that has spurred the National Obesity Forum and Public Health Collaboration into action.

Yet rather than look at why we ignore official dietary advice, their solution appears to be to make the dietary advice more palatable. Their most striking conclusion, and the one that has the health establishment seething, is that eating fat doesn't make you fat.

The report correctly says there is good data from "multiple controlled trials" that higher fat, lower carb diets are better for weight loss and heart health than low-fat diets. Rather than focus on low-fat diets, we should cut down on carbs, sugar and low-fat foods which are often loaded with sugar and salt, and avoid snacking, it says.

Much of this is good, common sense. However, when it comes to the 'fat is fab' message, the report falls short. Plenty of studies don't show the supposed benefits of high-fat diets and the scientific jury is still out.

As Prof Naveed Sattar, Professor of Metabolic Medicine at the University of Glasgow, says: "To make the headline message that we should all eat more fat as the cure to reverse obesity trends and thus type 2 diabetes is NOT warranted based on the totality of evidence."

The report claims saturated fat doesn't cause heart disease and that full-fat dairy products protect the heart. Again, that's not backed up by the totality of evidence. The British Heart Foundation says diets high in saturated fats have been shown to raise cholesterol, which has been shown to be linked to heart disease.

As for the official dietary advice from Public Health England, the report's authors accuse public health bodies of colluding with the food industry. That's a bit rich coming from the National Obesity Forum, whose official partners include GlaxoSmithKline, Safeway, Canderel and Slim Fast.

But perhaps the biggest problem with this report is that it links the rise in obesity to the official dietary advice. This is advice, after all, is followed by just 5% of the population according to some estimates.

This report is polemic, not the basis of public health messages. The ideas are interesting and could form the basis of conferences and journal discussion. But a message that could be interpreted by millions of people that it's fine to eat as much fat as they like is misguided at best, and dangerous at worst.

The obesity epidemic isn't caused by the wrong dietary advice. It's caused because people like me eat too much, do too little and refuse to follow any advice at all.