Bicycle helmets, the Irish media, and “lies, damned lies, and statistics”

COMMENT & ANALYSIS: Since last Thursday the Irish media has suffered from a serious case of bicycle helmets and “lies, damned lies, and statistics“.

It started with a news story — “Almost 70% of cyclists without helmet at time of head trauma” — which The Irish Times deemed worthy of its front page on Thursday. This compelling headline fits right into the phrase “lies, damned lies, and statistics”.

So, it’s first worth saying that The Irish Times has a bit of a messed up relationship with reporting on cycling as transport and related issues such as BusConnects. We have highlighted this more than a few times (see our archive).

Irish Times journalists continue to insist the issues with their coverage of cycling is a myth. While it is true that the newspaper publishes regular pro-cycling articles, but these are heavily weighted towards the features section — the way cycling related news is covered leaves a lot to be desired and the comment section of the newspaper is a mixed bag. Reporting of cycling related news by the Irish Times is too often flawed,

Last week’s cover story on bicycle helmets in The Irish Times was followed by coverage on radio programmes on RTE, Newstalk and other stations, and articles in the Irish Examiner and in The Sunday Times. So, the old “newspaper of record” still packs a punch. This makes reporting such a flawed story even worse.

Proportion of cyclist with/without helmets

If you’re going to plaster a statistic on a headline over an article on a newspaper’s cover page, the stat should have some statistical significance and that should be backed up within the article.

The excuse of ‘we’re just reporting or providing the data’ does not work here as the data is clearly presented to support helmet wearing.

So, if your headline is ‘almost 70% of cyclists presenting with a head injury don’t wear helmets’, then the proportion of cyclists generally without helmets should be statistically significantly lower.

However, according to a 2016 observational study by the Road Safety Authority, the helmet wearing rate is around 38% (public and private bicycle combined). This is also close to the findings of a 2017 national self-reporting survey conduced for the Department of Transport ahead of National Bike Week which found 39% of cyclists saying they wear helmets.

The difference between 70% and around 60% is not statistical significance. It’s well within a margin for error. This is made worse again by the other factors here (see below).

There’s no published research

A number of reports and interest groups statements referred to the research as if it has been published. It has not been published even now. So, The Irish Times is publishing a front page story about bicycle helmets based on snippets of yet-to-be published research. It seems to be all from an outline of the data given verbally at a conference in Dublin.

Research will at best be a case-control study

In their BMJ article, Ben Goldacre, Wellcome, research fellow in epidemiology and author of Bad Science, and David Spiegelhalter, Winton professor for the public understanding of risk argue that case-control studies which “suggest that, for individuals, helmets confer a benefit” are “vulnerable to many methodological shortcomings”.

The two explain: “If the controls are cyclists presenting with other injuries in the emergency department, then analyses are conditional on having an accident and therefore assume that wearing a helmet does not change the overall accident risk. There are also confounding variables that are generally unmeasured and perhaps even unmeasurable. People who choose to wear bicycle helmets will probably be different from those who ride without a helmet: they may be more cautious, for example, and so less likely to have a serious head injury, regardless of their helmets.”

Data set too small and incomplete

According to media reports, of the 26 cyclists treated at the national neurosurgical centre at Beaumont Hospital over 30 months, 18 were not wearing helmets, 2 were and the remaining 6 were unknown.

A data set of 26 is small and when 23% of the data being used is incomplete, it makes you wonder how reliable the data set is. The sample volume seems to be too small to draw conclusions.

Sports and transport

The Irish Times reported that: “Cycling accounted for all four of the sports and exercise-related deaths recorded at the centre over the period. Two of these patients were not wearing helmets, one was and the status of the other mortality was unknown. Two of the dead cyclists are recorded as having fallen off their bicycle, while the other two were knocked off.”

Lumping mountain biking, racing cycling and an old lady cycling to the shop is about as useful as lumping motor racing with commuting by car. But medical researchers seem to have a serious issue lumping cycling and sport and cycling as transport together because their data seems to be so unreliable to start with.

Colouring the data

The Irish Times quotes lead author Dr Phil O’Halloran of Beaumont Hospital as stating: “The figures show no sport is immune to head injury, so common sense needs to prevail.”

This brings up the question: Were the researchers interested in scientific research or were they just looking for data that might back up helmet wearing?

Personally, if I’m going to rely on “common sense”, as O’Halloran asks, I think I’ll go with ~17 million people in the Netherlands when the helmet wearing rate is under 1% and they have very high levels of young and old people cycling, yet, they have the safest cycling at least in the EU.

Bias for higher numbers over more serious?

It’s unclear if this issue is of the newspaper’s or the medics or both, but there seems to be a bias for higher numbers overall being taken over more serious incidents. Deep within the story we read that that rugby “accounted for the highest rate of patient transfer, and five of these six patients required major brain surgery.”

The 2008 Phillips Report on head injures in Ireland also comes to the conclusion of promoting bicycle helmets with no support for such in the data provided. However, if we look at the data in that report, we find that people cycling mostly suffer mild brain injury when compared to pedestrians or motorists. Also compared to other road users, cyclists — including non-helmeted ones — had by far the lowest irreversible injury rate and lowest mortality rate.

Helmets cannot stop brain injuries

This is a scientific fact — bicycle helmets cannot stop brain injuries for two reasons. Firstly, concussion are basically an internal injury caused by movement of the brain within the skull and, secondly, more serious skull fractures are way above the force a flimsy bicycle helmet can take. The air-bag type of bicycle helmets seem to be more of a help with the latter but are an impractical solution given the low level of risk.

Groups like Acquired Brain Injury Ireland and medics researching the area used to nearly imply otherwise, but there seems to be a change of  tactics and in the last week both researches and head injury groups have been trying to preempt arguments around the limits of helmets.

The problem with making this argument with this research is that you must then allow for it in your findings. But if we account for concussions and very high level impacts, the already limited data would be lowered in quality again.

Helmets are an understandable emotional response

The promotion of bicycle helmets is an understandable emotional response especially from people who care for head injury victims. But such makes for bad science, poor journalism, and poorer policy — misdirecting attention away from solutions needed.

I was told on Twitter by one medic that perhaps it’s easier for cycling advocates to take a population-level approach to this issue because they’re not responsible for the care of people whose brain injuries might have been less severe had they been wearing a helmet. This is an understandable response, but bad science.

As Goldacre and Spiegelhalter wrote: “Standing over all this methodological complexity is a layer of politics, culture, and psychology. Supporters of helmets often tell vivid stories about someone they knew, or heard of, who was apparently saved from severe head injury by a helmet. Risks and benefits may be exaggerated or discounted depending on the emotional response to the idea of a helmet.”

It’s not one or the other, we’re told

The story then goes on that there’s nothing stopping the advocacy of both helmets and cycling infrastructure. There’s two major problems m with this.

There’s little evidence that the people focused on promotion of bicycle helmets have also focused in any meaningful way on the promotion of cycling infrastructure. Even where they support cycling infrastructure, they still spend a undue amount of time of promoting gear. Time and focus are not unlimited. Focusing on one thing takes the focus off the other.

Barbara O’Connell, CEO of Acquired Brain Injury Ireland, is reported as reacting to the original story as saying: “People think when they come off their bicycles that they have been knocked off. For many people it could be a pedestrian stepping out in front of them, it could be a rock on the road or a pothole.” But the problem is that there’s no evidence of Acquired Brain Injury Ireland ever asking pedestrians to look out for cyclists or for road authorities to better maintain our roads.

The second problem with such claims is that you cannot have Dutch-style mass cycling and unchallenged heavy helmet promotion by the state. The people who claim helmets are not a barrier are the people who already cycle with helmets, or those who don’t understand what mass cycling on a Dutch-scale is, or those who put at best weak evidence to support helmets over solid evidence of the health benefits of mass cycling.

***

Individuals of course should be free to chose, but too many people see a major distinction between mandatory helmets and getting helmets stuffed in your face all the time, mostly with people so sure that you’re just an idiot. Recently people have started to imply that helmets are like vaccinations when that’s not the case. It’s worth noting that while he has debunked helmet research in one article, Ben Goldacre has spent a lot more time writing about anti-vaxxers.

For the record: I’d much prefer if I’d never have to write about bicycles helmets again, but I don’t have the obsession with them, I am defending against other people’s obsession of them. I’m not the one who put a flawed article on the cover on a national newspaper when said newspaper has failed in any way to report on, for example, councillors voting down cycle route proposals.

I am editor of IrishCycle.com and have reported on and commented on cycling in Ireland for over a decade. My background is in journalism -- I have a BA in Journalism from DCU and HDip in Print Journalism from BCFE. I wrote about cycling for national newspapers, and then started CyclingInDublin.com for overflow stories. Later the website was re-branded to reflect a more national focus.

19 Comments

  1. Great article again on the subject. It’s a shame you still have to write about the matter though. In Northern Ireland, the PSNI are constantly promoting helmet wearing and the comments on their Facebook posts are completely in line with the researchers conclusion about emotional response (even when people were not present in the incident).
    Your articles about bicycle helmets are my go to place to clear my ideas and information on the matter.
    Keep up the good work.

  2. Cian….really solid and useful article again in relation to this issue. It should be required reading for all cyclists and not just cycling advocates!

  3. It’s always curious how the pro lid commentariat are always just for polystyrene lids, and never go for full face bmx helmets, or motorbike helmets, or as the logical conclusion to their ill though out “if it saves one life” notions the FIA8860-2018 Formula 1 helmet.

  4. “Helmets cannot stop brain injuries
    This is a scientific fact”. Where is the research to back this up?

  5. @Jim, brain injury is caused by the brain hitting the inside of the skull. Helmets provide no protection for these scenarios. Head trauma, on the other hand, is where the head suffers a wound with blood flowing, following some type of collision with a hard object or surface. Helmets do provide protection from head trauma, but concussion may still arise.

    From a cyclist point of view, it is frustrating to hear so many advocates for cycle safety talk about helmets and high-viz, and very little about the big issues: segregation and driver behavior.

    I wonder how many car passengers or drivers suffered head trauma injuries, in the same period covered by the IT article. What proportion were wearing safety helmets?

  6. Alternate headline for IT: “More than 30% of cyclists suffer brain injury despite wearing helmet”.


  7. However, according to a 2016 observational study by the Road Safety Authority, the helmet wearing rate is around 62% (public and private bicycle combined). This is also close to the findings of a 2017 national self-reporting survey conduced for the Department of Transport ahead of National Bike Week which found 39% of cyclists saying they wear helmets.

    “the *non* helmet wearing rate is around 62%”? Otherwise the second stat doesn’t back it up (62% wearing rate >> 39% wearing rate).

    Excellent article. One could calculate a relative risk ratio and a 95% confidence interval for the ratio and see if the interval crossed the value of 1.0 (the value which would indicate the wearing of helmets doesn’t make a significant difference). But I’m not sure it’s worth that much effort. The Spiegelhalter/Goldacre editorial makes it pretty clear these hospitalisation studies are not really all that appropriate.

  8. @Dermot yes, that was the non-helmet wearing rate figure — I’ve replaced it with the helmet wearing rate. Thanks.

  9. Thanks, Cian.

    Well, if you assume a background general-population non-wearing rate of about 60%, and try a z-test for one proportion, based on a sample of 26 seriously head-injured cyclists (non-wearing rate of about 70%), you don’t get a statistically significant result.

    At least, when I tried it out here:
    https://www.medcalc.org/calc/test_one_proportion.php
    (Observed Proportion: 70%, Sample size:26, Null hypothesis value: 60%)
    I got p=0.3
    You’d want p<0.05 for it to be statistically significant.

    26 is a pretty small dataset.

    But even if the background non-wearing rate is much lower than 60%, the usual shortcomings (confounding, patients telling the doctor what they think they want to hear) would still mean this set of observations isn't of much value.

  10. I did write to Prof. Dr. O’Halloran (Beaumont) seeking a copy of his paper on this study but he informed me that it has not been published. My guess is that the Irish Times reporter was at the neurology conference at which these findings were presented.
    Cyclist.ie did tag Ben Goldacre on a tweet but no response.
    Thanks Dermot and Cian for good analysis.

  11. @Cian, I’m still waiting for you to back up your claim: “Helmets cannot stop brain injuries
    This is a scientific fact.”

    I think your article is misleading, dangerous and completely lacking balance. The likelihood of a traumatic brain injury (TBI) caused by a skull fracture arising from a head hitting the road can be reduced by a good helmet protecting the head and preventing the skull fracture at the point of impact. Helmets can stop some serious TBIs. If you wanted to state that helmets cannot stop all concussions, you would be on safer ground.

    @Hugh and @colmd Head trauma causes brain injuries too. Helmets can prevent these dangerous skull fractures. That’s a scientific fact.

    https://www.aans.org/Patients/Neurosurgical-Conditions-and-Treatments/Sports-related-Head-Injury

    https://www.youtube.com/watch?v=skzmeqPbSUY

    https://www.researchgate.net/publication/303592836_Helmet_efficacy_against_concussion_and_traumatic_brain_injury_A_review

  12. @Jim that was said within the context of the full paragraph. As in helmets can’t stop all brain injuries.

    As for your researchgate.net link, meta-analysis if weak research is going to be still weak — I would refer you back to what Goldacre and Spiegelhalter said: https://www.bmj.com/content/346/bmj.f3817?ijkey=I5vHBog6FhaaLzX&keytype=ref&fbclid=IwAR1N9mkbTsJU6msWaZdoF-jU6hb5_1QQhf3sgFTReHmP41E-2UmNuKbxrVk

    As for the claim that the article is dangerous — that’s not borne out by what we know about (1) how weak pro-helmet research is, (2) the low real-life risk of hitting ones head and (3) the population level results from the Netherlands.

  13. I wonder what percentage of car drivers presenting with a head injury don’t wear helmets? I’d hazard a guess that it’s 100%.

  14. @cian nowhere in that paragraph or the entire article do you acknowledge that helmets can help prevent many serious brain injuries. That’s why its an unbalanced and dangerous piece of writing / anti-helmet propaganda.

  15. @Jim… are you, like O’Halloran, relying on “common sense”?

    Otherwise, unfortunately you cannot back up what you’re saying other than what could be classed as poor or bad science. As according to Ben Goldacre, Wellcome research fellow in epidemiology, and David Spiegelhalter, Winton professor for the public understanding of risk, the direct benefits of helmets “seem too modest to capture”.

    The duo write: “The enduring popularity of helmets as a proposed major intervention for increased road safety may therefore lie not with their direct benefits—which seem too modest to capture compared with other strategies—but more with the cultural, psychological, and political aspects of popular debate around risk.”

    Note: “direct benefits—which seem too modest to capture”.

    And: “Such findings suggest that, for individuals, helmets confer a benefit. These studies, however, are vulnerable to many methodological shortcomings. If the controls are cyclists presenting with other injuries in the emergency department, then analyses are conditional on having an accident and therefore assume that wearing a helmet does not change the overall accident risk. There are also confounding variables that are generally unmeasured and perhaps even unmeasurable.”

    Note: “These studies, however, are vulnerable to many methodological shortcomings” and “There are also confounding variables that are generally unmeasured and perhaps even unmeasurable.”

    Jim, for all I know you’re an expert in this area but you’re providing little to convince me Goldacre and Spiegelhalter are wrong in what they have outlined in detail. You can do all the muck slinging like calling my work “dangerous” or “propaganda” you like but you’re really not supporting what you are saying.

  16. Re: the BMJ Goldacre piece. That’s opinion, not science. There is no hypothesis tested, only a (very interesting and useful in it’s right) piece surveying some literature. More valid opinion given the authors are involved in epidemiology and risk analysis? Not really IMO. The data on cycling and helmets is necessarily vague given, as everyone correctly identifies, there are so many cofounders. If you can summarise that, then you can’t draw conclusion on either side, can you?

    Now, if you hit your head, you are better to have a well fitted helmet on. That is indeed common sense. Baffles me how some seemingly intelligent people who seemingly only want to ‘win’ an argument will not simply admit that.

    “But there are other things that are more important like infrastructure”, yes would be great to have mass cycling and associated culture. Still better to be wearing a helmet if you hit your head while cycling. “You are more likely to have head injury in shower/car/stairs/wherever else”, yes, but you’re not at the mercy of others in those environments, and those environments do include other specific personal protection. “But if you hit your head hard enough there is no point in wearing a helmet”, yes, but we hope to mitigate injury by use of personal protection, what’s the point if a bullet proof vest when you can be shot in the face too?

  17. @Bruce — When you can’t say if it has a positive or negative effect, you’re in the world of snake oil. People who react like you are reacting the same way US football reacted to the idea helmets didn’t solve its problems and a reliance on helmets to do so had the unintended consequences of injury and death.

    One of the named authors is an epidemiologist, and the other is a professor for the public understanding of risk —- this is key: if there’s a policy perception for helmets while cycling, there should be such for loads of other things.

    Re: “but you’re not at the mercy of others in those environments, and those environments do include other specific personal protection” — you should wear a helmet if you want based on your fears of the dangers of others, but you’re clearly wrong if you think people walking or in cars are not open to danger due to the accruing of others. Car users could greatly benefit from helmets and could wear heavy-duty motor racing helmets not usable when cycling. It’s deemed too much to ask for in a balance of safety and being practical.

    If you want to fling muck too and go on about intelligence, that’s fine. But when you’re taking about common sense, I’ll stick with the bulk of 17 million Dutch people. Relying on common sense is bad science.

    Re bullet proof vests: people who are in high likelihood of been shot at also wear bullet-stopping ballistic helmets and those really in the line of fire (like SWAT entry teams) use ballistic shields. Bringing this back to bicycle helmets… why not face-protecting BMX helmets?

  18. ==
    Re: the BMJ Goldacre piece. That’s opinion, not science. There is no hypothesis tested, only a (very interesting and useful in it’s right) piece surveying some literature. More valid opinion given the authors are involved in epidemiology and risk analysis? Not really IMO.
    ==

    This is a poor take. The majority of the papers concerning the efficacy of helmets is based on epidemiology or statistics. Spiegelhalter and Goldacre were ideally qualified to put what was then a new piece of research in the context of the previous research. It’s a very well written and perceptive overview of the literature written by two experts.

    And there are plenty of papers that back up what they’re saying. What do you think they’re giving their opinion on? It’s not a transcript of a conversation they had in the pub.

1 Trackback / Pingback

  1. Here's How - Here's How 93 – Bike Heads

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: