Thursday, 21 May 2015

Bad science by press release

Ben Goldacre wrote a book called Bad Science a few years ago which explained why obvious quackery was quackery. The targets were deserving, though a bit soft for my tastes (homeopaths, nutritionists etc.), and it sold a lot of copies so it must have said something people needed to hear. He also used to write a column in The Guardian which often worth reading.

On his good days, Goldacre would patiently explain the difference between trustworthy evidence and dubious evidence. Published studies are better than unpublished studies, peer review is better than no peer review, randomised control trials are better than observational studies, literature reviews are better than individual studies, and so on. In short, the pyramid of evidence looks like this...

Goldacre has built his reputation on being a dispassionate, apolitical observer of human folly; a man driven by a thirst for evidence, not ideology. Given the choice between a Cochrane review of randomised control trials (top of the pyramid) and an unpublished conference abstract (third from bottom), it's pretty obvious which one he would prefer, right?

Wrong. In December, the first - and so far only - Cochrane review of e-cigarettes was published. It found 'evidence from two trials that ECs [e-cigarettes] help smokers to stop smoking long-term compared with placebo ECs'.

This week, a conference abstract for another literature review - one that included non-RCTs - was released to the press. It concluded that 'Evidence that electronic cigarettes are effective for smoking cessation long-term is lacking'.

To my knowledge, Goldacre has never mentioned the Cochrane review, nor has he mentioned the systematic review that was published in PLoS One last month which concluded that 'Use of e-cigarettes is associated with smoking cessation'. Nor, indeed, has he mentioned the study in Addiction which found e-cigarettes to be more than twice as effective than nicotine-replacement therapy, or the study published three months ago which found that intensive users of e-cigarettes were six times more likely to quit than non-users. But when presented with an unpublished conference abstract, he wasted no time in spreading the news with this bilious Twitter communiqué...

What's got into him? He has previously referred to e-cigarettes as the 'tobacco industry's latest scheme', seemingly unaware that until 2012 there was no tobacco industry involvement in the sector whatsoever and even today the combined might of global Big Tobacco owns just seven of the thousands of brands on the market. On the subject of vaping, Goldacre has shown himself to be no better than the two-bit hacks who write "I reckon" articles about e-cigarettes based on whatever stray thought enters their head. Gateway! Formaldehyde! Children!

But this is not a post about Goldacre's confirmation bias. There is another mistake he makes which was also made by the Daily Mail, namely that the study shows that 'e-cigs don't help smokers quit'. Even the press release doesn't claim that. It says only that, based on its findings, the evidence is 'lacking'.

Absence of evidence is not evidence of absence, particularly when you look at the results in the abstract. The review found the odds of e-cig users (versus placebo users) staying abstinent after one month were 1.71 (1.08-2.72). After three months they were 1.95 (0.74-5.13) and after six months they were 1.32 (0.59-2.93). The first of these is statistically significant, the others are not, hence the lack of firm evidence for cessation at six months.

The study also mentions that the 'only study to evaluate continuous abstinence' found that e-cigarette users' odds of abstinence were 1.77 (0.54-5.77) after six months compared to the placebo group. This, again, is not statistically significant, but nor is it close enough to 1.0 for a positive effect to be ruled out. The same is true of the other findings, all of which are well above 1.0. It would be quite wrong to say that these statistically non-significant odds ratios proved that e-cigarettes work better than placebos, but it is equally wrong to say that they show that 'e-cigs don't help people quit'. All they show is that larger trials will be needed unless e-cigarettes more than double the chance of quitting. The Cochrane review identified one randomised controlled trial which did indeed find that e-cigarettes more than doubled the chances of quitting. We don't know which studies the conference review included (or why) because, as I say, it hasn't been published.

I wouldn't claim for a moment that the Cochrane review is the last word on e-cigarettes and nor do its authors. Cochrane reviews only look at RCTs and only two RCTs have so far been conducted, making the review weak by Cochrane's high standards. Nevertheless, it does have the merit of being peer-reviewed and published, which is normally the minimum Goldacre requires before taking research seriously. That is, unless he wants to wind up a section of society - most of whom no longer smoke thanks to e-cigarettes - that he deems 'vile', in which case any old press release will do.

Wednesday, 20 May 2015

Pubs and predictions

The IEA have just published my new report Drinking, Fast and Slow - you can download it here. It looks at the gloomy predictions about what the Licensing Act (AKA 24 Hour Drinking) would do to the sovereign people of England in 2005.

The short read is on the IEA blog and Guido has a handy infographic.

The even shorter read is that all of the predictions have been proven wrong. The major reason why they were wrong is that there were informed, consciously or not, by the simplistic temperance idea of 'availability theory'.

Anyway, here's the long read. Enjoy.

As an amusing footnote, it is worth remembering that the Licensing Act was a Labour policy. Paul Flynn, the Labour MP, voted strongly for the policy and then voted against delaying its implementation. But that didn't stop him having an involuntary spasm this morning when he heard that the IEA had concluded that the law wasn't a complete disaster.

It's an affliction with these people, isn't it? For the record, none of my IEA reports are commissioned, they are all my own work (aside from editing), nobody funds specific reports and the first the pub/booze industry heard about this report was yesterday when the press release went out (if not today). As far as I know, the Licensing Act isn't a live issue for the alcohol industry's lobby squad as there is no chance of it being repealed or amended in the near future. The Act is, however, an excellent example of deregulation working and illiberal people being hilariously wrong. Hence my interest.

Speaking of regulation, the first rays of reality are dawning on CAMRA and the deeply misguided 'Save The Pub' campaign. Now that the government has broken the beer tie, Enterprise Inns are going to flog off a thousand pubs and convert another thousand into commercial properties. Some of them will remain pubs, of course, but certainly not all. As Ed Bedington says in the Morning Advertiser...

We are starting to see the unintended consequences of the pubs code... Enterprise's announcement shows that from its current estate of more than 5,200 pubs, at least 2,000 of those are at risk - by my maths, that's about 40%.

... And for all those well meaning politicians watching those unintended consequences coming home to roost? Cheers guys. The first round in post-MRO landscape is on you.

I made my prediction of what would happen if the government fell for CAMRA's schtick last year. So far, it's proving to be more reliable than the predictions made about the Licensing Act.


I've written more about this at The Spectator. Please have a read. 

Monday, 18 May 2015

Alcohol Concern, Dry January and dodgy figures

Alcohol Concern has been dishing out awards to its Dry January champions. Tellingly, all but one of the awards has gone to public sector organisations. The sock puppet charity also put out a press release claiming that...

More than 2 million go dry for biggest Dry January yet

Alas, the figure of 'more than 2 million' was contradicted by the quote in the press release from Alcohol Con's head lobbyist...

Emily Robinson, Director of Campaigns at Alcohol Concern, said: “We’re incredibly proud of all our participants and fundraisers and want to recognise and celebrate those who took the biggest Dry January campaign yet to a new level.”

“The aim of Dry January is to help people to think about their drinking, and to get support in breaking bad habits. With over 50,000 people taking part, it’s great to see how people took this further and supported their friends and communities.

The figure of 50,000 has previously been cited by Alcohol Concern and it is the figure that appears on the Dry January website. It is much the more believable of the two.

When this was pointed out to Alcohol Concern they amended their press release... by removing the reference to 50,000 sign ups! (You can still see the quote in the various local rags that churnalised the press release eg. here and here.)

A tweet made it clear that only 50,000 had signed up for Dry January but two million 'were dry for January'.

Considering that there are more than two million observant Muslims and Methodists—not to mention children—it is not much of a claim to say that more than two million people happened to be teetotal in January. These people took part in Dry January in the same way that I took part in Elton John's boycott of Dolce and Gabbana. They were never likely to do otherwise.

In January, the Daily Mash published an article headlined 'Non-alcoholics enjoying pretend battle with drink'. Life has imitated satire.

Why this urge to inflate the figures by a factor of forty? Two possibilities spring to mind.

Firstly, Public Health England gave Alcohol Concern £500,000 to increase the profile of Dry January. I have written about this disgraceful squandering of taxpayers' money before, but even a bloated quango like Public Health England must conduct some sort of cost-benefit analysis after the event. 50,000 participants sounds pretty feeble (because it is). Two million sounds more impressive.

Secondly, Cancer Research UK does exactly the same thing in the same month under the name 'Dryathlon'. This year it got 54,000 people to sign up. The difference is that the Dryathlon doesn't suck up a penny of taxpayers' cash.

In short, Dry January wastes public money trying to do something that another charity does better, hence the need to fiddle the figures. If Public Health England throws another half a million pounds at Alcohol Con next year, there should be a public enquiry.

Sunday, 17 May 2015

E-cigarette survey

The Centre for Drug Misuse Research is conducting a large, international survey of e-cigarette users. If you have ever used an e-cigarette, you should complete it. It takes about five minutes and you could win a prize.

Complete the survey.

Friday, 15 May 2015

We are the 82 per cent

82 per cent of Britons do not think that obesity is a disease and 1 in 5 people who are technically overweight do not consider themselves fat. I think they might have a point.

Read more at The Spectator.

A tiny victory

Royal Society for Public Health is a ghastly organisation which I wrote about when they decided they wanted doctors to get into town planning. Earlier this week, they put out a press release to accompany the dodgy OECD report on alcohol under the headline 'UK's rising alcohol consumption a stark reminder of the need for tougher action began'. It began...

New research which shows alcohol consumption in the UK is on the rise is a stark reminder of the pressing need for tougher action

Alcohol consumption is not rising, of course. On the contrary, it has been falling faster than at any time for 80 years. Anybody who claims to speak with authority on the subject should know this. To its modest credit, the RSPH subsequently removed this press release and replaced it with this...

We welcome the OECD report which has brought tackling alcohol related harm back into the spotlight.

RSPH have long been advocating for the implementation of a range of measures to combat alcohol-related harm, including minimum unit pricing, calorie labelling and compulsory PSHE education, and urge the new government to take action.

We are however concerned that the figures in the report do not correlate with official government statistics which lean towards a downward trend for alcohol consumption and binge drinking in the UK. It is important the public is presented with an accurate picture of the nation’s health and an evidence base that is robust. The health and social consequences of excessive drinking are too serious to risk confusing the public.

This admission comes a few weeks after the RSPH criticised Aseem Malhotra's guff in the British Journal of Sports Medicine about exercise. Slightly pathetically, the RSPH only complained about Malhotra 'sending mixed messages' rather than being scientifically illiterate, but it was better than nothing.

You don't win any medals for pointing out the bleeding obvious, but as far as I know the RSPH is the only 'public health' organisation to have responded to the OECD report by drawing attention to the fact that alcohol consumption is falling—and, even then, only after putting out a press release saying the exact opposite.

Such is the level of endemic deceit in the public health racket that one organisation accepting one easily verifiable fact counts as a win. These scoundrels will have to be dragged kicking and screaming before they face reality.

Thursday, 14 May 2015

Aseem Malhotra screws up again

Our old friend Aseem Malhotra is starting to become a regular at Retraction Watch. A year after sparking an investigation at the British Medical Journal, Malhotra found himself in hot water with the British Journal of Sports Medicine.

You almost certainly heard about Malhotra's article in the BJSM attacking the 'myth' that exercise helps people lose weight. This risible claim was broadcast around the world. What you probably did not hear is that the article was taken offline within 48 hours and only reappeared last week.

Why was it suspended? Any number of factual inaccuracies could have been responsible (see here for an example), but rumour had it that undisclosed conflicts of interest were the main reason. As I mentioned at the time, Malhotra's co-authors are neck deep in the low-carb/Atkins/banting diet and one of them has a book to sell. This is a fairly mild competing interest, but it is typical of a BMJ publication to be more concerned about an author receiving an indirect financial benefit than about his article being truthful.

The competing interest was not the only reason, however. The article has been subtly altered and the following notice has been added...

Correction notice This article has been amended from the original published on 29th April 2015. The body of the text was slightly edited and a reference removed. Competing interests have been added.

The edited part looked like this in the original...

It now looks like this...

As you can see, the journal has removed the part about the tobacco industry "buying the loyalty of bent scientists". This is important because the paragraph in question accuses the food industry of using "tactics chillingly similar to those of big tobacco."

And, in case anybody was in any doubt that Malhotra et al. are accusing the food industry of hiring 'bent scientists', reference 5 (which has now also been removed) was 'Sugar: spinning a web of influence' by Jonathan Gornall. I wrote about Gornall's article (which was published in the BMJ) a few months ago.

A theme is emerging in his hatchet jobs. First, he takes a policy which is controversial with the public but which has legitimate arguments for and against. He then treats the policy as a no-brainer which could only possibly be opposed by vested interests. He then looks for any kind of funding from business to civil society and the public sector; if he cannot find any he implies that it exists. Finally, he pads out his articles with quotes from activists and presents their failure to persuade government to bring in the controversial policy as being the result of 'webs of influence'.
Last year it was minimum pricing with the Alcohol Health Alliance. This time it's food reformulation with Action on Sugar.

Gornall's articles for the BMJ have all been of a very low quality and the journal embarrassed itself by publishing the sugar article, in particular. Gornall failed to understand that collaborations between the government and the food industry are not a dirty secret. On the contrary, they are the norm, and everybody who works in the field understands this.

The great and the good of 'public health' nutrition lined up in the Rapid Responses to attack the article. Prof Barry Popkin described it as "not only naïve but misguided" and said that Dr Susan Jebb, who bore the brunt of Gornall's poison pen, was "quite incorrectly impugned". The Chief Executive of the Medical Research Council said "please add my name to your 'tangled web'. It would be an honour to stand alongside scientists such as Susan Jebb, Ann Prentice and Ian Macdonald, who are committed to improving public health through research." It's worth reading the Rapid Responses in full.

This should have been enough for the BMJ to end its association with Gornall but I understand that he is currently carrying out more "research" for the journal. Even Gornall, however, did not describe Susan Jebb, Ian MacDonald et al. as "bent scientists". By using that phrase and citing Gornall's article as proof, Malhotra and his chums came very close to doing so. In the eyes of the libel lawyers, they may actually have done so, hence the retraction and correction.

Here we have two people—Malhotra and Gornall—who should not be allowed within a hundred miles of a medical journal publishing utter tripe, with one referencing the other. This Laurel and Hardy act would be quite amusing if it didn't push, and indeed exceed, the boundaries of what can be printed in a serious magazine. As I have shown time and again on this blog, Malhotra writes down the first thing that comes into his head. He is extremely credulous and not terribly bright. This is the second time in twelve months that he has got a journal into trouble, despite the fact that he rarely writes for journals. How many more chances is he going to be given?