COMMENT & ANALYSIS: Since last Thursday the Irish media has suffered from a serious case of bicycle helmets and “lies, damned lies, and statistics“.
It started with a news story — “Almost 70% of cyclists without helmet at time of head trauma” — which The Irish Times deemed worthy of its front page on Thursday. This compelling headline fits right into the phrase “lies, damned lies, and statistics”.
So, it’s first worth saying that The Irish Times has a bit of a messed up relationship with reporting on cycling as transport and related issues such as BusConnects. We have highlighted this more than a few times (see our archive).
Irish Times journalists continue to insist the issues with their coverage of cycling is a myth. While it is true that the newspaper publishes regular pro-cycling articles, but these are heavily weighted towards the features section — the way cycling related news is covered leaves a lot to be desired and the comment section of the newspaper is a mixed bag. Reporting of cycling related news by the Irish Times is too often flawed,
Last week’s cover story on bicycle helmets in The Irish Times was followed by coverage on radio programmes on RTE, Newstalk and other stations, and articles in the Irish Examiner and in The Sunday Times. So, the old “newspaper of record” still packs a punch. This makes reporting such a flawed story even worse.
Proportion of cyclist with/without helmets
If you’re going to plaster a statistic on a headline over an article on a newspaper’s cover page, the stat should have some statistical significance and that should be backed up within the article.
The excuse of ‘we’re just reporting or providing the data’ does not work here as the data is clearly presented to support helmet wearing.
So, if your headline is ‘almost 70% of cyclists presenting with a head injury don’t wear helmets’, then the proportion of cyclists generally without helmets should be statistically significantly lower.
However, according to a 2016 observational study by the Road Safety Authority, the helmet wearing rate is around 38% (public and private bicycle combined). This is also close to the findings of a 2017 national self-reporting survey conduced for the Department of Transport ahead of National Bike Week which found 39% of cyclists saying they wear helmets.
The difference between 70% and around 60% is not statistical significance. It’s well within a margin for error. This is made worse again by the other factors here (see below).
There’s no published research
A number of reports and interest groups statements referred to the research as if it has been published. It has not been published even now. So, The Irish Times is publishing a front page story about bicycle helmets based on snippets of yet-to-be published research. It seems to be all from an outline of the data given verbally at a conference in Dublin.
Research will at best be a case-control study
In their BMJ article, Ben Goldacre, Wellcome, research fellow in epidemiology and author of Bad Science, and David Spiegelhalter, Winton professor for the public understanding of risk argue that case-control studies which “suggest that, for individuals, helmets confer a benefit” are “vulnerable to many methodological shortcomings”.
The two explain: “If the controls are cyclists presenting with other injuries in the emergency department, then analyses are conditional on having an accident and therefore assume that wearing a helmet does not change the overall accident risk. There are also confounding variables that are generally unmeasured and perhaps even unmeasurable. People who choose to wear bicycle helmets will probably be different from those who ride without a helmet: they may be more cautious, for example, and so less likely to have a serious head injury, regardless of their helmets.”
Data set too small and incomplete
According to media reports, of the 26 cyclists treated at the national neurosurgical centre at Beaumont Hospital over 30 months, 18 were not wearing helmets, 2 were and the remaining 6 were unknown.
A data set of 26 is small and when 23% of the data being used is incomplete, it makes you wonder how reliable the data set is. The sample volume seems to be too small to draw conclusions.
Sports and transport
The Irish Times reported that: “Cycling accounted for all four of the sports and exercise-related deaths recorded at the centre over the period. Two of these patients were not wearing helmets, one was and the status of the other mortality was unknown. Two of the dead cyclists are recorded as having fallen off their bicycle, while the other two were knocked off.”
Lumping mountain biking, racing cycling and an old lady cycling to the shop is about as useful as lumping motor racing with commuting by car. But medical researchers seem to have a serious issue lumping cycling and sport and cycling as transport together because their data seems to be so unreliable to start with.
Colouring the data
The Irish Times quotes lead author Dr Phil O’Halloran of Beaumont Hospital as stating: “The figures show no sport is immune to head injury, so common sense needs to prevail.”
This brings up the question: Were the researchers interested in scientific research or were they just looking for data that might back up helmet wearing?
Personally, if I’m going to rely on “common sense”, as O’Halloran asks, I think I’ll go with ~17 million people in the Netherlands when the helmet wearing rate is under 1% and they have very high levels of young and old people cycling, yet, they have the safest cycling at least in the EU.
Bias for higher numbers over more serious?
It’s unclear if this issue is of the newspaper’s or the medics or both, but there seems to be a bias for higher numbers overall being taken over more serious incidents. Deep within the story we read that that rugby “accounted for the highest rate of patient transfer, and five of these six patients required major brain surgery.”
The 2008 Phillips Report on head injures in Ireland also comes to the conclusion of promoting bicycle helmets with no support for such in the data provided. However, if we look at the data in that report, we find that people cycling mostly suffer mild brain injury when compared to pedestrians or motorists. Also compared to other road users, cyclists — including non-helmeted ones — had by far the lowest irreversible injury rate and lowest mortality rate.
Helmets cannot stop brain injuries
This is a scientific fact — bicycle helmets cannot stop brain injuries for two reasons. Firstly, concussion are basically an internal injury caused by movement of the brain within the skull and, secondly, more serious skull fractures are way above the force a flimsy bicycle helmet can take. The air-bag type of bicycle helmets seem to be more of a help with the latter but are an impractical solution given the low level of risk.
Groups like Acquired Brain Injury Ireland and medics researching the area used to nearly imply otherwise, but there seems to be a change of tactics and in the last week both researches and head injury groups have been trying to preempt arguments around the limits of helmets.
The problem with making this argument with this research is that you must then allow for it in your findings. But if we account for concussions and very high level impacts, the already limited data would be lowered in quality again.
Helmets are an understandable emotional response
The promotion of bicycle helmets is an understandable emotional response especially from people who care for head injury victims. But such makes for bad science, poor journalism, and poorer policy — misdirecting attention away from solutions needed.
I was told on Twitter by one medic that perhaps it’s easier for cycling advocates to take a population-level approach to this issue because they’re not responsible for the care of people whose brain injuries might have been less severe had they been wearing a helmet. This is an understandable response, but bad science.
As Goldacre and Spiegelhalter wrote: “Standing over all this methodological complexity is a layer of politics, culture, and psychology. Supporters of helmets often tell vivid stories about someone they knew, or heard of, who was apparently saved from severe head injury by a helmet. Risks and benefits may be exaggerated or discounted depending on the emotional response to the idea of a helmet.”
It’s not one or the other, we’re told
The story then goes on that there’s nothing stopping the advocacy of both helmets and cycling infrastructure. There’s two major problems m with this.
There’s little evidence that the people focused on promotion of bicycle helmets have also focused in any meaningful way on the promotion of cycling infrastructure. Even where they support cycling infrastructure, they still spend a undue amount of time of promoting gear. Time and focus are not unlimited. Focusing on one thing takes the focus off the other.
Barbara O’Connell, CEO of Acquired Brain Injury Ireland, is reported as reacting to the original story as saying: “People think when they come off their bicycles that they have been knocked off. For many people it could be a pedestrian stepping out in front of them, it could be a rock on the road or a pothole.” But the problem is that there’s no evidence of Acquired Brain Injury Ireland ever asking pedestrians to look out for cyclists or for road authorities to better maintain our roads.
The second problem with such claims is that you cannot have Dutch-style mass cycling and unchallenged heavy helmet promotion by the state. The people who claim helmets are not a barrier are the people who already cycle with helmets, or those who don’t understand what mass cycling on a Dutch-scale is, or those who put at best weak evidence to support helmets over solid evidence of the health benefits of mass cycling.
Individuals of course should be free to chose, but too many people see a major distinction between mandatory helmets and getting helmets stuffed in your face all the time, mostly with people so sure that you’re just an idiot. Recently people have started to imply that helmets are like vaccinations when that’s not the case. It’s worth noting that while he has debunked helmet research in one article, Ben Goldacre has spent a lot more time writing about anti-vaxxers.
For the record: I’d much prefer if I’d never have to write about bicycles helmets again, but I don’t have the obsession with them, I am defending against other people’s obsession of them. I’m not the one who put a flawed article on the cover on a national newspaper when said newspaper has failed in any way to report on, for example, councillors voting down cycle route proposals.
Great article again on the subject. It’s a shame you still have to write about the matter though. In Northern Ireland, the PSNI are constantly promoting helmet wearing and the comments on their Facebook posts are completely in line with the researchers conclusion about emotional response (even when people were not present in the incident).
Your articles about bicycle helmets are my go to place to clear my ideas and information on the matter.
Keep up the good work.
Cian….really solid and useful article again in relation to this issue. It should be required reading for all cyclists and not just cycling advocates!
It’s always curious how the pro lid commentariat are always just for polystyrene lids, and never go for full face bmx helmets, or motorbike helmets, or as the logical conclusion to their ill though out “if it saves one life” notions the FIA8860-2018 Formula 1 helmet.
“Helmets cannot stop brain injuries
This is a scientific fact”. Where is the research to back this up?
@Jim Here’s an overview https://en.wikipedia.org/wiki/Concussions_in_American_football
@Jim, brain injury is caused by the brain hitting the inside of the skull. Helmets provide no protection for these scenarios. Head trauma, on the other hand, is where the head suffers a wound with blood flowing, following some type of collision with a hard object or surface. Helmets do provide protection from head trauma, but concussion may still arise.
From a cyclist point of view, it is frustrating to hear so many advocates for cycle safety talk about helmets and high-viz, and very little about the big issues: segregation and driver behavior.
I wonder how many car passengers or drivers suffered head trauma injuries, in the same period covered by the IT article. What proportion were wearing safety helmets?
Alternate headline for IT: “More than 30% of cyclists suffer brain injury despite wearing helmet”.
However, according to a 2016 observational study by the Road Safety Authority, the helmet wearing rate is around 62% (public and private bicycle combined). This is also close to the findings of a 2017 national self-reporting survey conduced for the Department of Transport ahead of National Bike Week which found 39% of cyclists saying they wear helmets.
“the *non* helmet wearing rate is around 62%”? Otherwise the second stat doesn’t back it up (62% wearing rate >> 39% wearing rate).
Excellent article. One could calculate a relative risk ratio and a 95% confidence interval for the ratio and see if the interval crossed the value of 1.0 (the value which would indicate the wearing of helmets doesn’t make a significant difference). But I’m not sure it’s worth that much effort. The Spiegelhalter/Goldacre editorial makes it pretty clear these hospitalisation studies are not really all that appropriate.
@Dermot yes, that was the non-helmet wearing rate figure — I’ve replaced it with the helmet wearing rate. Thanks.
Well, if you assume a background general-population non-wearing rate of about 60%, and try a z-test for one proportion, based on a sample of 26 seriously head-injured cyclists (non-wearing rate of about 70%), you don’t get a statistically significant result.
At least, when I tried it out here:
(Observed Proportion: 70%, Sample size:26, Null hypothesis value: 60%)
I got p=0.3
You’d want p<0.05 for it to be statistically significant.
26 is a pretty small dataset.
But even if the background non-wearing rate is much lower than 60%, the usual shortcomings (confounding, patients telling the doctor what they think they want to hear) would still mean this set of observations isn't of much value.
I did write to Prof. Dr. O’Halloran (Beaumont) seeking a copy of his paper on this study but he informed me that it has not been published. My guess is that the Irish Times reporter was at the neurology conference at which these findings were presented.
Cyclist.ie did tag Ben Goldacre on a tweet but no response.
Thanks Dermot and Cian for good analysis.
@Cian, I’m still waiting for you to back up your claim: “Helmets cannot stop brain injuries
This is a scientific fact.”
I think your article is misleading, dangerous and completely lacking balance. The likelihood of a traumatic brain injury (TBI) caused by a skull fracture arising from a head hitting the road can be reduced by a good helmet protecting the head and preventing the skull fracture at the point of impact. Helmets can stop some serious TBIs. If you wanted to state that helmets cannot stop all concussions, you would be on safer ground.
@Hugh and @colmd Head trauma causes brain injuries too. Helmets can prevent these dangerous skull fractures. That’s a scientific fact.
@Jim that was said within the context of the full paragraph. As in helmets can’t stop all brain injuries.
As for your researchgate.net link, meta-analysis if weak research is going to be still weak — I would refer you back to what Goldacre and Spiegelhalter said: https://www.bmj.com/content/346/bmj.f3817?ijkey=I5vHBog6FhaaLzX&keytype=ref&fbclid=IwAR1N9mkbTsJU6msWaZdoF-jU6hb5_1QQhf3sgFTReHmP41E-2UmNuKbxrVk
As for the claim that the article is dangerous — that’s not borne out by what we know about (1) how weak pro-helmet research is, (2) the low real-life risk of hitting ones head and (3) the population level results from the Netherlands.
I wonder what percentage of car drivers presenting with a head injury don’t wear helmets? I’d hazard a guess that it’s 100%.
@cian nowhere in that paragraph or the entire article do you acknowledge that helmets can help prevent many serious brain injuries. That’s why its an unbalanced and dangerous piece of writing / anti-helmet propaganda.
@Jim… are you, like O’Halloran, relying on “common sense”?
Otherwise, unfortunately you cannot back up what you’re saying other than what could be classed as poor or bad science. As according to Ben Goldacre, Wellcome research fellow in epidemiology, and David Spiegelhalter, Winton professor for the public understanding of risk, the direct benefits of helmets “seem too modest to capture”.
The duo write: “The enduring popularity of helmets as a proposed major intervention for increased road safety may therefore lie not with their direct benefits—which seem too modest to capture compared with other strategies—but more with the cultural, psychological, and political aspects of popular debate around risk.”
Note: “direct benefits—which seem too modest to capture”.
And: “Such findings suggest that, for individuals, helmets confer a benefit. These studies, however, are vulnerable to many methodological shortcomings. If the controls are cyclists presenting with other injuries in the emergency department, then analyses are conditional on having an accident and therefore assume that wearing a helmet does not change the overall accident risk. There are also confounding variables that are generally unmeasured and perhaps even unmeasurable.”
Note: “These studies, however, are vulnerable to many methodological shortcomings” and “There are also confounding variables that are generally unmeasured and perhaps even unmeasurable.”
Jim, for all I know you’re an expert in this area but you’re providing little to convince me Goldacre and Spiegelhalter are wrong in what they have outlined in detail. You can do all the muck slinging like calling my work “dangerous” or “propaganda” you like but you’re really not supporting what you are saying.
Re: the BMJ Goldacre piece. That’s opinion, not science. There is no hypothesis tested, only a (very interesting and useful in it’s right) piece surveying some literature. More valid opinion given the authors are involved in epidemiology and risk analysis? Not really IMO. The data on cycling and helmets is necessarily vague given, as everyone correctly identifies, there are so many cofounders. If you can summarise that, then you can’t draw conclusion on either side, can you?
Now, if you hit your head, you are better to have a well fitted helmet on. That is indeed common sense. Baffles me how some seemingly intelligent people who seemingly only want to ‘win’ an argument will not simply admit that.
“But there are other things that are more important like infrastructure”, yes would be great to have mass cycling and associated culture. Still better to be wearing a helmet if you hit your head while cycling. “You are more likely to have head injury in shower/car/stairs/wherever else”, yes, but you’re not at the mercy of others in those environments, and those environments do include other specific personal protection. “But if you hit your head hard enough there is no point in wearing a helmet”, yes, but we hope to mitigate injury by use of personal protection, what’s the point if a bullet proof vest when you can be shot in the face too?
@Bruce — When you can’t say if it has a positive or negative effect, you’re in the world of snake oil. People who react like you are reacting the same way US football reacted to the idea helmets didn’t solve its problems and a reliance on helmets to do so had the unintended consequences of injury and death.
One of the named authors is an epidemiologist, and the other is a professor for the public understanding of risk —- this is key: if there’s a policy perception for helmets while cycling, there should be such for loads of other things.
Re: “but you’re not at the mercy of others in those environments, and those environments do include other specific personal protection” — you should wear a helmet if you want based on your fears of the dangers of others, but you’re clearly wrong if you think people walking or in cars are not open to danger due to the accruing of others. Car users could greatly benefit from helmets and could wear heavy-duty motor racing helmets not usable when cycling. It’s deemed too much to ask for in a balance of safety and being practical.
If you want to fling muck too and go on about intelligence, that’s fine. But when you’re taking about common sense, I’ll stick with the bulk of 17 million Dutch people. Relying on common sense is bad science.
Re bullet proof vests: people who are in high likelihood of been shot at also wear bullet-stopping ballistic helmets and those really in the line of fire (like SWAT entry teams) use ballistic shields. Bringing this back to bicycle helmets… why not face-protecting BMX helmets?
Re: the BMJ Goldacre piece. That’s opinion, not science. There is no hypothesis tested, only a (very interesting and useful in it’s right) piece surveying some literature. More valid opinion given the authors are involved in epidemiology and risk analysis? Not really IMO.
This is a poor take. The majority of the papers concerning the efficacy of helmets is based on epidemiology or statistics. Spiegelhalter and Goldacre were ideally qualified to put what was then a new piece of research in the context of the previous research. It’s a very well written and perceptive overview of the literature written by two experts.
And there are plenty of papers that back up what they’re saying. What do you think they’re giving their opinion on? It’s not a transcript of a conversation they had in the pub.
@DermotRyan “And there are plenty of papers that back up what they’re saying”
Totally agree with your first paragraph. All the papers back up what they say about the one they ‘review’. They are well placed to review these epidemiological studies. Well written, perceptive, true. But, and here’s where my take isn’t poor, the editorial isn’t about helmets, it’s about epidemiology. Goldacre says as much here: https://www.badscience.net/2013/12/bicycle-helmets-and-the-law-a-perfect-teaching-case-for-epidemiology : “the complex contradictory mess of evidence on the impact of bicycle helmet”. So take home message is that the complexities involved with the particular case of wearing a bike helmet renders science on the same as essentially void, and certainly no conclusion can really be drawn.
Therefore, trying to use this editorial to bolster either side of the helmet debate is to miss the point. The closing line “The enduring popularity of helmets as a proposed major intervention for increased road safety may therefore lie not with their direct benefits …” is not about helmets not being useful in that most obvious context, i.e. protecting your head in a fall, but that other issues might be (more) relevant. But we don’t know, because there is no evidence. So we can’t say.
@Bruce — if what you are saying is correct, it makes the pro-helmet side into cargo cult science.
“When you can’t say if it has a positive or negative effect, you’re in the world of snake oil”
Some of us can and do say it has a positive effect. Can you admit, even internally, that if you are to fall off your bike, in that instance, it would be better to have a well-attached bike helmet on your head? Is this a step too far? Snake oil has no properties given the claims it makes, helmets do though surely?
“People who react like you are reacting the same way …”
I say: bike helmet is better than none in case of an accident. You say: helmets in NFL were meant to save people and didn’t. I take the point and agree that the intended safety device might not meet it’s purpose, but I hold to my statement. It is true. The NFL case (and CTE generally) is more of an issue given repeated head trauma.
“but you’re clearly wrong if you think people walking or in cars are not open to danger due to the accruing of others.”
Thankfully I don’t think that, wouldn’t want to be wrong about something (again!). Seatbelts, airbags, crush-panels; footpaths, pedestrian lights, guard rails. A bit of, dare I say it, common sense. Risk avoidance is factored into all areas of our lives. Balance and practicality, absolutely, totally agree. Hi-viz, lights, helmet, pretty practical and basic stuff. Very little barrier to entry. Don’t want to. Also fine. Good culture to build and maintain IMO, glad road cyclists have gotten there on that.
“If you want to fling muck too and go on about intelligence, that’s fine. But when you’re taking about common sense, I’ll stick with the bulk of 17 million Dutch people. Relying on common sense is bad science.”
It has very little to do with science.
“Re bullet proof vests: people who are in high likelihood of been shot at also wear bullet-stopping ballistic helmets”
Had to laugh: so in an environment with increased danger we should wear helmets!
“Bringing this back to bicycle helmets… why not face-protecting BMX helmets?”
They obscure vision, are heavy and uncomfortable. Impractical. Balance in risk aversion strategies? It’s a disingenuous or short-sighted comment, tend to the former given the general tone. BMX helmet is to protect the face. Coming off on a track, at speed is likely in BMX racing. Your face is at increased risk. But races are short, and helmet is only on for a few minutes at a time. No culture of BMX riders outside of racing wearing helmets. If you come off your bike on the road, the impact to your cheek or jaw will be less likely to be life-threatening. Risk of coming off is far reduced vs. BMX. Upper and back of head is far more serious, so still risk and should be avoided if desired.
Two final points:
1) I fully accept and agree that infrastructure is required to make cycling safer. Denmark and Netherlands are beacons of this, and the level of cycling engagement is amazing to see. This will never happen in Ireland in my opinion, based on many mostly unresolvable issues. We have poor road conditions, bad planning of same, cycling is dangerous in Dublin certainly, outside likely. To wear a helmet and cycle is a choice, but I hate to see people try to claim helmets are useless. The old chestnut of saving one life is always true, I hope it is mine or one of my loved ones.
2) Helmet is really optional, and largely based on how you ride a bike. I don’t give my mum a hard time for not wearing one, as she tips down the shop. I would much more seriously advocate hiviz and lights. I would fully support laws on visability of cyclists.
@Bruce — Sorry for the delayed reply, I’m going to reply to this first because it’s more important and it might be colouring the rest of what you have said:
Your stance here is partly why change is painfully slow — spending ages arguing for helmets and then kicking the boot in by saying it “will never happen in Ireland”. And, as I said, it colours your reply…
For clarity: I’m mainly only commenting on cycling as transport. People who are outliers can make their own minds up if they want to wear one or not.
The term ‘snake oil’ is wider than the watery substance it was based on. One definition quickly found seems to fit well with what I mean: “a product, policy, etc. of little real worth or value that is promoted as the solution to a problem”.
As Dermot says above: “You might think that (unquantified) potential small benefit is worth it, and more power to you. Of course, given the nature of the evidence, a small disbenefit is also within the range of possibilities.”
You’re misunderstanding the reference to the NFL issue. It’s not that it’s exactly the same as cycling. It’s that helmets are seen as great with little backing in both cases.
You said: “’You are more likely to have head injury in shower/car/stairs/wherever else’, yes, but you’re not at the mercy of others in those environments, and those environments do include other specific personal protection.” Were you careless with what you said first or what’s happening here?
Even with “Seatbelts, airbags, crush-panels” car drivers are still at the mercy of trucks and head injures are still a larger issue for car users than bicycle users. As for “footpaths, pedestrian lights, guard rails” — people are sadly all too regularly killed on footpaths by motorists and while crossing at pedestrian lights. Guard rails barely slow down cars as that’s not their design.
The article linked to bmj.com above is co-written by David Spiegelhalter, Winton professor for the public understanding of risk argue. Risk is relative and if you say that the “old chestnut of saving one life is always true”
For everyday utility cycling high-vis and helmets are not practical. But this is coloured by your idea of what kind of everyday cycling is possable in Ireland.
Yes… that’s the problem. Common sense has little to do with science.
Helmets for cars and walking then — cycling head injures are more recoverable type than those suffered by car users and pedestrians. Your sense of danger colours your view on if helmets are needed for cycling or other modes of transport.
Helmets generally are impractical for wide-ranging everyday cycling and a real barrier to mass cycling. The people who think otherwise are people who already wear helmets and those who don’t think mass cycling is possable here — you fit both.
I only came back here to mention the Here’s How podcast (already in the Pingback section), but Bruce, you can of course find compelling evidence of real-world effects with epidemiology. The reason you can’t find strong evidence via epidemiology for the efficacy of helmets is because the benefit, if any, is small relative to the resolution of the statistical techniques available. If it were a large effect, you could definitely find it.
You might think that (unquantified) potential small benefit is worth it, and more power to you. Of course, given the nature of the evidence, a small disbenefit is also within the range of possibilities.