The BCA have no evidence that chiropractic can help with ear infections

Based on this quote from Simon Singh the British Chiropractic Association (BCA) decided to sue him for libel:

“the British Chiropractic Association claims that their members can help treat children with colic, sleeping and feeding problems, frequent ear infections, asthma and prolonged crying, even though there is not a jot of evidence. This organisation is the respectable face of the chiropractic profession and yet it happily promotes bogus treatments.”

The BCA claimed that there was a ‘plethora of evidence’ supporting their position and disproving Simon Singh’s, the BCA have now released a statement and listed their evidence and make the claim that

This proves that far from there being “not a jot of evidence” to support the BCA’s position, there is actually a significant amount

A number of blogs have taken it upon themselves to assess the strength of the papers against the claims the BCA make for it.  I will be doing the papers on frequent ear infections of which there are three, suggesting that the BCA’s definition of plethora is even further estranged from that of the dictionary than Judge Eady’s interpretation of ‘bogus’.

The BCA claim that Mills MV et al (2003) ’suggest[s] a potential benefit of osteopathic manipulative treatment [(OMT)] as adjuvant therapy in children with recurrent acute otitis media; it may prevent or decrease surgical intervention or antibiotic overuse’, an almost verbatim quote from the abstract’s conclusions.  However, the BCA have not explored the faults of the paper, many of which the authors draw attention to, such as using fewer patients than their preliminary statements suggest as necessary:

Results of power analysis suggested we would need 50 children in each group, on the basis of a predicted 50% decrease in antibiotic use, episodes of AOM in the group receiving medical treatment alone, and a 75% improvement in the group receiving medical treatment and OMT. With a type I error rate set at .05, we estimated an expected power of 85% if there were a 10% dropout rate and a power of 80% with a 20% dropout rate.

There were 25 patients in the intervention group and 32 in the control group considerably raising the possibility of a type I error, a false positive result, in the statistical interpretation of data.   This is likely to have happened given the range of variable assessed by the authors (Table 2 below).  Also the authors’, and the BCA’s, contention that this may decrease antibiotic overuse seems untenable given the lack of statistical significance regarding antibiotic usage during the study (P=0.13).

*update*

Richard in the comments points out:

That the Mills et al (2003) is smaller than their power test suggested was needed does not affect the type I error (incorrectly rejecting the null hypothesis). Instead it increases the risk of a type II error (incorrectly accepting the null hypothesis). As such, the small size of the study works against them.

The greater problem is that they make 23 tests without, apparently, correcting for multiple comparisons. This will massively inflate the risk of type I errors and it is not in the least surprising that they have a couple of apparently significant results.

The authors also did not use a placebo control:

The issue of whether to include a placebo control for the control group was considered. If we had included a placebo control, any presumed placebo intervention might have had an unintended treatment effect, introducing potential confounding variables and necessitating 3 groups: intervention, placebo intervention, and nonintervention. We chose instead for this study to report any difference in outcomes between 2 groups that were selected to be as equal as possible except for the application of OMT. Placebo effect due to the number of visits was minimized by the design of the study, leaving the influence of touch as potentially having a placebo effect. Larger studies are needed to replicate and elucidate the causal mechanisms of this effect.

The authors are acknowledging here that they cannot distinguish between the placebo effect of touching a patient and the OMT treatment.  Therefore their investigations cannot say whether OMT has an effect above placebo.  The study might suggest a benefit, but only if you throw caution to the wind and ignore chances of false positives and make the a priori assumption that OMT can have an effect on ear infections, in which case you would be bringing considerable bias into your interpretation. After all you would have to ignore the awkward fact that there is no conceivable way in which OMT could effect the immune response to infection or actively contribute to the reduction of infection in any other way.

The BCA claim that Froehle RM (1996) shows that ‘93% of all episodes improved. The study’s data indicates that  … the addition of chiropractic care may decrease the symptoms of ear infection in young children’.  The study has no control group and is little more than a collection of case notes on the duration of ear infection, of which the author notes ‘very little data was found regarding the natural course of ear infections’.  Therefore the study says nothing about the impact of chiropractic care on the duration of ear infection compared to the natural course of healing but it does say infections get better over time.  This is not evidence supporting the use of chiropractic care in ear infection so much as supporting evidence for the old adage that time heals all wounds.

I cannot get access to the Fallon JM (1997) paper which the BCA claim of which ‘[t]he results indicate that there is a strong correlation between the chiropractic adjustment and the resolution of otitis media for the children in this study.’ as it is not listed in a PubMed indexed journal.

*update*

Thanks to everyone in the comments and others for providing the link to the abstract of this paper.  Looking at it it seems not to involve a control group, thus cannot tell us anything about the efficacy of chiropractic vs other treatment or placebo, and is more of an argument for the use of a measuring technique in assessing the impact of chiropractic rather than the BCA’s stated claim.  In fact, the BCA’s (and the author’s) claim about a correlation is disingenuous given that in the absence of a placebo control it is impossible to say if the correlation is due to a placebo effect, the BCA (and the author) would do well to remember that correlation is not proof of causation.

This ‘plethora’ of three trials, two of which are very badly designed and one of which is unavailable is available in abstract form only, does not constitute a ’significant amount of evidence’ disproving Simon Singh’s claims.  In fact it strongly suggests that the BCA, in reaching into their basket of evidence, are picking cherries and clutching straws.  Frankly it is embarrassing, this is the equivalent of a child attempting to convince an adult that his colander on a stick is in fact a super intelligent robot, only without the innocence and charm.

*updates*

Details of other blogs covering these papers will be posted later. These links largely copied and pasted from Layscience, will add to later if I have time.

Jack of Kent – General commentary and legal background.
Prof. Colquhuon – Detailed look at the nine colic papers.
Ministry of Truth – General review focusing on three of the colic papers.
Andy – Comment on the BCA statement.
Evidence Matters – Review of the paediatric asthma papers.
Layscience – a review of the flaws in all the papers available
Phil Plait – An overview of the BCA statement and aftermath.
HolfordWatch – What would constitute good evidence?
Apgaylard – A more detailed look at the bed-wetting papers.
JDC – General comment on the BCA statement.
Think Logic – General comment on the BCA statement.

References

Mills MV, Henley CE, Barnes LL, Carreiro JE, Degenhardt BF. The use of osteopathic manipulative treatment in children with acute recurrent otitis media. Arch Paediatr Adolesc Med.  2003 Sep;157(9): 861-6

Froehle RM. Ear infection: a retrospective study examining improvement from chiropractic care and analysing for influencing factors. J Manipulative Physiol Ther 1996 Mar; 19(3): 169-77

Fallon JM. The role of the chiropractic adjustment in the care and treatment of 332 children with otitis media. J Clin Chiropract Paediatrics 1997 Oct; 2(2): 167-183

[BPSDB]