Share this post
Don't jump to rash(er) conclusions
Written by Alex Massie
25 March 2021
Earlier this week almost every newspaper, including the more reputable ones, reported a most alarming story. According to researchers at Leeds University, eating 25 grams of processed meat a day – typically illustrated as being roughly the equivalent of one rasher of bacon – was “associated” with a whopping 44% increase in the likelihood of developing dementia. Well, yikes!
According to Huifeng Zhang, who led the study, “worldwide, the prevalence of dementia is increasing and diet as a modifiable factor could play a role. Our research adds to the growing body of evidence linking processed meat consumption to increased risk of a range of non-transmissible diseases”.
Bad news always makes it into the papers and so does surprising news. By definition, the media – which is to say, my trade – is biased in favour of pessimism and novelty. “Sun rose again this morning” is not a news story; the sun’s failure to make an appearance would be considered quite a big deal. And admittedly, in this particular case, quite correctly so.
Newspapers, then, offer a first interpretation of history but an unavoidably partial one. The news is dominated by the unusual, from which it often follows that the news is unavoidably riven with irony: your morning paper is full of important stuff, but it is often the case that things which are in the news are less important than they may initially seem. Unusual happenings make the news and, most of the time, you probably do not need to be concerned, or at least not overly concerned, by unusual things.
I would not go so far as to suggest this means you may safely discount what’s in the news – some of us require at least some of you to continue to purchase newspapers – but I offer this merely as a reminder that, like anything else, the news should be treated with a certain measure of caution and an awareness of its inbuilt prejudices.
So, frankly, should science. If a finding seems unusual, that may be because it really is worth a raised eyebrow. In the case of “eating bacon is linked to developing dementia” such scepticism would seem to be well-placed. This is so, even though the study was, on the face of it, a large one. The researchers looked at the habits of half a million British adults, typically aged 57 or thereabouts, who had signed up to the UK Biobank project. Of these, around 2,900 people developed dementia over the eight years covered by the analysis.
As The Times noted, however, “it may be that people who eat lots of processed meat also behave in many other ways that make them vulnerable to dementia, which the researchers were not able to allow for”. This seems a tolerably significant point, does it not? And indeed so: for people more likely to develop dementia were, in the first place, more likely to be carrying a gene strongly associated with a higher risk of dementia and, secondly, they were more likely to be older, poorer, less well educated, more likely to smoke, less likely to take regular exercise and more likely to have relatives who had previously developed dementia themselves or had strokes.
All of these factors seem significant and, frankly, rather more substantial than your choice of breakfast. Indeed, according to Clive Ballard, executive dean for medicine at the University of Exeter, “we should not assume from this research that one rasher of bacon a day increases your risk of dementia by 44% – it is simply impossible to demonstrate that in a study like this”.
So that is good news! The bad news is that bad science – or at least over-hyped science – is an easy way to fill a newspaper and this remains so even if the full story contains, as it does in this instance, a refutation of its headline promise. This is not fake news, per se, but nor is it brilliant journalism.
Few journalists, frankly, know very much about numbers. I include myself in this category and not just because it is seemly to do so. That explains why even when the numbers presented in news stories are not wrong, they are often deprived of the context which would make them more comprehensible – and, therefore, more accurate.
For instance, suppose “X doubles the risk of Y” in circumstances where Y is a Very Bad Thing. This is alarming and might lead one to think we should crack down on X. But what is the prevalence of Y, in the first place? If Y is a one in a million chance, X making it a two in a million possibility is both a dramatic increase and a minor one. It’s probably not something worth worrying about too much. Absolute risk is, perhaps, more important than relative risk.
That is one of the many pieces of good sense contained in a new – and short – book by Tom Chivers, a journalist, and his cousin David, an economist. How To Read Numbers is one of the most useful works published this year. Reading it will help most of us become a little bit more numerate.
For numbers are slippery creatures at the best of times. They may appear solid and impassive, but they are open to endless manipulation. In my own field of political journalism, much attention is paid to opinion polling and while reputable polling companies take great care to conduct their research honestly and without bias, the interpretation of their work is not subject to the same levels of diligence. Outlying polls – the ones with surprising results – are afforded greater prominence than they merit and too much attention is likewise paid to relatively inconsequential differences between polls.
Thus, “YouGov finds support for independence up by two points!” tempts us to think that history will be running in one direction and “Panelbase reports support for independence slipping by two points,” prompts us to respond in precisely the opposite way. This can be so even if both polls reach the conclusion that 50% of voters support independence and 50% do not.
As an election campaign begins in Scotland, then, a reminder: even the best polling is subject to a margin of error (typically of around three points in either direction) and what counts is the trend, not the detail of individual surveys. That confirms what we instinctively knew anyway: support for independence rose appreciably last year but has, according to most recent surveys, softened just a bit since then. The country remains split more or less down the middle and it is foolish, indeed a waste of energy, to obsess too much over Poll A which reports a 51:49 divide and equally pointless to worry too much about Poll B which finds it is actually 48: 52. The chances are they are both right.
Precision is often a hindrance to understanding, too. Economic forecasts, for instance, are often made to look foolish by their insistence on too much precision. A prediction of GDP growth of 1.8% would be better made if accompanied by an acknowledgement of the range of plausible outcomes. If we have a 95% confidence in growth being between one per cent and three per cent, we have a better understanding of what is really most likely; for the world is an uncertain place and trying to impose too much order upon it distorts our understanding just as much as it superficially aids it.
So, a rule of thumb to bear in mind: the more extravagant or surprising the number, the more you should treat it with a certain measure of suspicion. Boring numbers are just that, but they are likely to be more reliable. Outliers exist, but less frequently than we typically think. If something is in the newspaper, it is novel because the news depends on novelty, but what is novel is not always the same as what is significant. You may, I think, have an extra rasher on your morning roll and do so without too much anxiety. Probably.
About Beyond the Street and Alex Massie
We are all guilty of being too inwardly focused sometimes, especially as we navigate these changed times. It is all too easy to be caught up in the problems close to home, and for overarching trends to pass us by.
To remedy that short-sightedness, Charlotte Street Partners has enlisted the ingenuity and talent of the writer and commentator Alex Massie. As our new correspondent at large, Alex will look at the bigger picture for us each week. We have challenged him to come up with something a bit different: broad, lateral thinking, thematic insights and a more global perspective.
Alex is a freelance journalist and commentator based in Edinburgh. Not only is he Scotland editor of The Spectator, but he also writes a political column for The Times and The Sunday Times. He features regularly on the BBC as a political commentator and has written in the past for The Telegraph, Politico, The Washington Post, the Los Angeles Times, The New York Times, the New Statesman, The Observer, and TIME magazine, among others. He was also the Washington correspondent for The Scotsman and assistant editor of Scotland on Sunday.
We hope you enjoy his work – and do feel free to forward this email on to colleagues and friends you think might be interested. They can subscribe to Beyond the Street and our other regular briefings here.