Editing reality

I really hope that this is the last post I write about the homeoprophylaxis campaign against Leptospirosis in Cuba during 2007 – 2008 (Bracho et al, 2010).  Deep down I know that this uncontrolled, un-randomised poorly-reported trial published in a terrible pseudojournal, dealing with a highly variable disease which is amenable to personal protective measures, a real vaccine and antibiotic treatment is going to get thrown at me again and again.

Homeopathic propagandists will not worry that real medicines were also used in the treatment region, a media campaign raised awareness of the disease and the homeopaths intervened at the peak of a multi-year problem.  Neither will it bother them that the net outcome was a return to the same infection rate as the rest of Cuba or the Intervention Region in 2004, or that the paper was rejected by proper journals.

However, before I move on I think that the accompanying guest editorial by Roniger and Jacobs (2010) deserves some additional scrutiny. It’s entitled, “Prophylaxis against Leptospirosis using a nosode: Can this large cohort study serve as a model for future replications?”

Why bother, when the study it is editorialising on (Bracho et al, 2010) is so deeply flawed?  I have two reasons: first, as the title suggests, it appears that it is seen as being able to support a broader agenda.  Second, the lead author, Helmut Roniger, is a doctor who practises in the UK and works at the Royal London Homeopathic Hospital.  To see an NHS doctor going into print to argue for unethical trials is instructive.

One of the early points raised in the editorial follows a trend that I have seen repeatedly in writings of homeopathic apologists: happily abandoning homeopathic doctrine when it looks like one of the various, and often contradictory, homeopathic factions has found success.

Classical Homeopathy – an embarrassing friend?

Classical Homeopathy prides itself on individualised treatment based on finding the ‘true’ simillimum, respecting the traditions of the Master Hahnemann and his later disciples.  For these reasons, as the authors note:

“Homeoprophylaxis with nosodes has been practised for many years but is not typically used by practitioners of Classical Homeopathy.”

Homeoprophylaxis is not reconcilable with classical homeopathy.  There is no like-curing-like and no individualisation.  However, now that one homeopathic faction has apparently found success, the authors are prepared to snub Classical Homeopathy:

“the use of a nosode eliminates the interference of ideological prescribing variations.”

It looks to me like an argument of convenience.  They think they have found a ‘success’ so classical homeopathy can be dismissed.  However, I wonder if members of the homeopathic community will welcome their particular practise being described as ‘interfering’ with treatment?

The editorial then starts to discuss the merits of the trial.

How much does size matter?

In lauding the size of the trial Roniger and Jacobs (2010) present misconceptions that are astonishing for a doctor and medical academic:

“the size of the population treated and the dramatic decrease in disease incidence compared to previous years make it difficult to dismiss these results as spurious or occurring by chance.”

Size and coincidence, for them, outweigh any idea of basic plausibility or methodological rigour.  Their contention on the decrease in disease levels is just an appeal to the fallacy of Post hoc ergo propter hoc: B follows A, therefore A caused B.  Put more simply: I get up and the sun rises, therefore me getting up causes the sun to rise!

As for the size of the trial, given that there was no control group or randomization, this does not inoculate it against bias.  Particularly as infection risk is not evenly spread over the population.  Living near a contaminated water source, being in a high risk occupation or failing to take personal protective measures will raise the risk for particular people or places.  Given that these risks will vary within a large entity like a state, size alone cannot compensate for poor design.  It’s just a very large poor trial.

Not comparing like with like

One of the most egregious errors of fact contained in this editorial comes next.  Roniger and Jacobs (2010) describe the homeopathic nosode therapy as:

“Similar to a conventional vaccine”.

This is just straightforwardly untrue.  Real vaccines based on inactivated disease-causing organisms train the bodies’ immune system to recognize and kill any live examples that come its way.  For this to happen, it’s absolutely essential that there are inactivated bugs in the vaccine.  How else can the immune system be trained?

In contrast, the homeopathic preparations used in this work did not contain any dead bugs, being the result of diluting a solution carrying dead bacteria to one part in 10⁴⁰⁰ and 10²⁰⁰⁰⁰, respectively.  So, in the single most important respect –carrying an agent that will train the immune system – the nosode therapy was entirely different to a real vaccine.  Not similar at all.

The editorial then summarises the ‘scientific evidence’ for the use of nosodes.

Scientific evidence?

“The scientific evidence for the prevention of disease using nosodes thus far is sparse, as referenced in the article. It consists mainly of observational studies, one controlled clinical human study using Meningococcinum which was poorly reported,² two controlled animal studies on different organisms^3,4 and a plant based model.⁵“

To call this evidence sparse is an exaggeration.  Only one of the studies addressed people, and has nothing to do with Leptospirosis.  Moreover Mroninski, Adriano and Mattos (2001) was ‘published’ in Homœopathic Links.  This is not a scientific journal; neither is it indexed in PubMed.  It’s a vanity publication for homeopaths.  A “poorly reported” trial in a homeopathic pseudojournal is certainly not scientific evidence.  However, I can appreciate why Roniger and Jacobs (2010) have padded out this sparse list – they cannot cite any other people-relevant papers.

Jonas (1999) conducted laboratory tests on mice, as did de Almeida et al (2008).   This is not good evidence for effects on people.  Scratching the surface undermines this evidence further.  Jonas (1999) experimented with nosode treatment of Francisella tularensis.  In a trial with, “142 male C3H/HeN specific, pathogen-free mice”, six ‘levels’ of a nosode were used.  Mortality and time to death were measured after exposure to a, “potentially lethal dose (LD50 or LD75)”.   The author summarized the results:

“In a series of 15 trials (n = 142), the tularemia nosode consistently produced increased mean times to death. All but 2 of 15 trials showed reduced time to death in the nosode group and decreased mortality compared with controls. Protection rates averaged 22% over controls compared to 100% protection by standard vaccination.”

I only have access to the abstract of this paper, so I am unclear as to which nosode ‘level’ had an effect; Though a protection rate of only 22% of the standard vaccine does not sound convincing.

The study by de Almeida et al (2008) is even less convincing.  They investigated Trypanosoma cruzi infection in mice.  Eighty mice were allocated to five groups, “treated with biotherapy (nosode) of T. cruzi 12dH (12x) pre- and post-infection; Phosphorus 12dH post-infection; infected control treated with control solution and uninfected control.”  Lots of things were measured, “Parasitaemia was monitored, leukocyte and serological responses were evaluated at 0, 7, 14 and 42 days after infection. The prepatent and patent periods of parasitaemia, maximum of parasitaemia, day of maximum parasitaemia and mortality rates were compared between groups”.

The paper claims that one of the homeopathic interventions gave the lowest parasitaemias values on particular, arbitrary post-infection days: “9, 13, 15 (p<0.05), 17 (p<0.05), 22, 24 and 28”.  This looks like a case of multiple comparisons and temporal cherry-picking.

If mice are of little relevance, then including a paper on a plant model strikes me as desperate.  Shah-Rossi, Heusser and Baumgartner (2009) experimented on mouse-ear cress, infected with Pseudomonas syringae.  This study is notable for several reasons; first only four of thirty homeopathic preparations yielded statistically significant effects.  Then in a series of repeat experiments only one of these, “induced a significant reduction of the infection rate”.  This sounds less than convincing.  This ‘successful’ preparation was identified as a, “homeopathic complex remedy” known as Biplantol.  The next notable fact was that this only ‘worked’ in its original form, but not in a 30x preparation.  Finally, the authors note that, “The exact composition of Biplantol SOS is not known […] Scientific progress would be expedited if the exact composition would be disclosed to the public.”

Inconsistent results, lots of trials and the only preparation to have an effect had an unknown composition.  This is not really scientific evidence either.

Next the editorial falls into a very common trap.

Coincidentally …

“This extraordinarily large study provides impressive evidence that a nosode in ultramolecular potencies (200c, 10M) prevented a Leptospirosis epidemic following a natural flooding disaster.”

This just revisits the post hoc ergo propter hoc fallacy.  It was certainly large, but it is not impressive.  At best it’s a correlation observed in an un-randomised, un-controlled intervention for a highly variable weather-dependant disease, in a region where conventional measures were also used.  There is absolutely no evidence that the magic water prevented anything.

Of mountains and molehills

That Roniger and Jacobs (2010) are homeopathic propagandists is clearly seen in a masterly piece of obfuscation.

“The number of cases of Leptospirosis dropped from 38 cases/ week per 100,000 population at the beginning of the study to 3 – 4 cases/week per 100,000 population within three weeks.”

They fail to put this into context: this is the same infection rate that the treated region had in 2004 and the same rate that the ‘untreated’ region had during 2008.  And yet this is built up as a disaster:

“Incidence in the untreated rest of the country (8.8million persons) in both years followed historical trends and correlated with rainfall.”

They do not mention that the ‘intervention’ failed to beat these historical trends to any practically relevant degree.  The comment about rainfall is also misleading – In the rest of Cuba incidence was poorly correlated with rainfall.

For an academic medical researcher and a specialist doctor, the authors seem to have no idea of what constitutes a good trial design.

Well designed?

“The non-randomized cohort design may explain why this innovative and well-designed study with its remarkable results was rejected by conventional journals.”

It is hard to see how a trial with no meaningful control groups or randomisation of any kind can be described by serious medical researchers as “well-designed”.  Given its severe methodological flaws it’s easy to see how journals not run by homeopaths for homeopaths decided to pass.

“In retrospect, it has been suggested that randomization could have been done by province to maximize the rigor of the study design.”

This looks like part of the reason that journals with decent standards have declined to publish the paper.  To imply that the suggestion is unhelpful hindsight is to acknowledge that these authors didn’t take rigorous design seriously.  Randomization is an indispensible scientific tool, which should not be lightly cast aside.

“However, the lack of randomization is strongly counterbalanced by the sheer size of the study, the clear time correlations, and the large drop in incidence rates compared to historical trends.”

It is not possible to use size to counterbalance lack of randomisation.  Without randomisation to internal control groups there is no way to account for differences in the behaviour of individuals – important for a disease like this where personal protective measures matter, or the risk associated with locations – vital where the source of the infection is localised to contaminated water sources.

Neither are the time correlations clear.  The majority of the drop in the rate of incidence came two weeks into the exercise.  Accounting for a typical week-long incubation period this means that the bulk of the effect came within a week.  Although the authors do not say what coverage they had achieved by this point, the evidence in the paper suggests that this was around 40%.

Finally, the drop in incidence only returned the situation to that prevailing in the intervention region throughout 2004.

“[…] the clear time correlations, and the large drop in incidence rates compared to historical trends. It is hard to imagine that such an impressive study of a conventional medicine or vaccine would not be published in a major medical journal, especially given the sample size of 2.3 million subjects.”

Given the weaknesses in the trial methodology it would have been a huge surprise if this had been published in a major medical journal.  However, it is clear that something happened to lower infection rates in the intervention region.  If not homeopathy, then what was it?

No other explanation?

Roniger and Jacobs (2010) cannot see past homeopathy as the explanation.  Neither are they able to see that Bracho et al (2010) were not really interested in alternative explanations.

“Other possible explanations for these findings were explored by the authors but cannot explain the results.”

This statement is just not true.  They approached this question from the opposite direction – expounding the reasons it had to be the homeopathy that did it.  Here are three straightforward explanations that they did not consider in any serious way:

1. Improved implementation of personal protective measures due to the focus given to controlling Leptospirosis by health workers visiting nearly everybody four times during the full trial period, and an extensive media campaign.
2. Better identification of individuals at risk through health workers visiting nearly everybody four times during the full trial period, enabling more effective use of proven interventions (vaccine and antibiotic).
3. The good luck of intervening at the peak of a problem that had been building since 2005, benefitting from regression to the mean.

In fact, their “innovative” trial design means that these more plausible explanations cannot be excluded.  It looks like the approach taken by Bracho et al (2010) shows an extreme care to avoid testing their fragile hypothesis: very unscientific.

Further propagandising for the homeopathic cause is seen in the way that conventional interventions are discussed.

A level playing field?

“Conventional prophylactic measures like vaccination and chemoprophylaxis were evenly distributed over all provinces of Cuba and it was estimated that only 3% of the total population was vaccinated or treated with chemoprophylaxis.”

It’s extremely disingenuous of Roniger and Jacobs (2010) to paint the picture in this way.  Bracho et al (2010) concede that conventional measures were used for outbreak control.  This necessitates an uneven distribution of the measures – as outbreaks, by definition, are not evenly spread over the population.

“This contrasts with an astonishing coverage of 92% of the cohort with the homeopathic nosode,”

The conventional measures also have demonstrable efficacy, whereas the homeoprophylaxis does not.  The ‘coverage’ – an undefined term – is remarkable.  Unfortunately its just alcohol in water!

No apologia for homeopathy would be complete without a dig at the pharmaceutical industry.

The Big Farmer

“even a less effective homeoprophylactic nosode could be easily construed as competition since it is cheap, safe and can be quickly produced in large quantities – at least in Cuba.”

This is arguing that homeopathy may not be ideal, but it’s relatively cheap.  The idea of ‘treating’ people with a few drops of magic water seen in this context is dangerous.  It’s a con, a sop to the populace, pretence of helping.  The real issue is how to make effective measures available.

The editorial then moves on to discuss a wider agenda that they think this trial could help to advance.

Homeopaths running wild

“Homeoprophylaxis with nosodes could be a bridging device in a serious epidemic for the period until a fully effective vaccine is available. In less serious diseases, it could be the only prophylaxis offered. The lack of side effects, low cost, and rapid speed of manufacture and deployment would make it acceptable for use in large populations. Coverage of animal vector populations with the nosode may also be feasible.”

These are grand designs built on a foundation of sand. The trial that they are commenting on provides no evidence that homeopathic nosodes work.

“Future studies of homeopathic nosodes would be relatively easy to implement and should also assess in more detail the effect on morbidity and complication rates, as well as economic impact. Studies of homeoprophylaxis using nosodes in animals should also be considered, perhaps as a prelude to clinical testing in humans.”

This just promises more bad trials measuring more things in the hope of striking it lucky.

“However, research could be hindered by perceived competition with conventional vaccines. Therefore other common infections with significant morbidity and/or mortality lacking effective and/or feasible conventional treatment, such as dengue fever, malaria, and antibiotic-resistant bacterial infections should be targeted first. In spite of our lack of theoretical understanding of homeoprophylaxis, ethical approval of future studies will be facilitated by this impressive study by our Cuban colleagues.”

This position beggars belief.  First off, there are highly effective interventions for malaria ranging from vector control and bite prevention through to treatment (WHO, 2010).  For a medical doctor to suggest otherwise shows, at best, staggering ignorance; malaria is also serious and potentially life threatening.  It is highly unethical to propose the use of pretend medicine for a dangerous problem.

Whilst dengue is not, at present, treatable, vector control and bite prevention are vital.  Magic water is just a dangerous distraction.

Antibiotic resistant infections are treatable with specific antibiotics.  MRSA, for instance, is Methicillin-resistant and can be treated with some antibiotics.   Again, a keen focus on prevention is also vital.  For an NHS doctor to even suggest treating a significant problem afflicting hospital care with delusional interventions is worrying.

It’s disturbing that they confess a “lack of theoretical understanding of homeoprophylaxis” without seeing this as any sort of ethical impediment.  Of course there is a “lack of theoretical understanding”; there is no reasonable scientific hypothesis on the table to account for any putative “effects” of homeoprophylaxis.

“There is currently a crisis around the scientific credibility of homeopathy. The British Medical Association has called for a funding stop of Homeopathy on the NHS and all British Homeopathic NHS Hospitals. Additionally it is urging pharmacists to put homeopathic remedies on shelves labeled “placebos”. The Science and Technology Committee of the UK House of Commons recently declared that the question of homeopathy being a placebo is ‘scientifically settled’.⁹ While this was refuted by the British Homeopathic Association,¹⁰ we urgently need an easily replicable model for a line of research into ultramolecular effects. Homeoprophylaxis could be exactly that, demonstrating replicable results in potencies beyond Avogadro’s number. Infectious diseases are still the bane of humanity, particularly in the developing world. Effectively reducing their morbidity and mortality using homeopathic nosodes could have immediate practical and economic impacts.

There is not so much a “crisis around the scientific credibility of homeopathy” as a closer focus on the quality of the supporting evidence.  Judged against any reasonable scientific standards, homeopathy has no scientific credibility.  The balance of the clinical evidence demonstrates that any effect is a placebo effect (Shang et al, 2005).  Further, there is no evidence that homeopathy is effective for any single condition (Linde et al, 1997) – precisely the issue that concerns patients.

Given that there is no real evidence that it works, its not surprising that there is no credible mechanism by which “potencies beyond Avogadro’s number” could have any specific effect.  These are the considerations that informed the clear and well founded judgement of The Science and Technology Select Committee.

Could the British Homeopathic Association have really refuted this position?  Only in the sense of saying that they “refute” it.  Of course, saying something is so, doesn’t make it so.  The BHA’s response shows the usual reliance on counting numbers of RCTs and calling failure to outperform an entirely inert placebo ‘inconclusive’.   Of course, if you are looking for evidence to support the use of any intervention a failure to beat a placebo is a negative outcome.

In part 2^* of the BHA’s response they claim that:

“there have been 87 RCTs that studied the efficacy of a given homeopathic medicine: 37 of them reported positive findings (see also the BHA’s supplementary memorandum, Ev 53–59 of printed report).  Of 50 other RCTs of this nature, 2 were negative and 48 were inconclusive.”

This translates to 100 negative trials out of 137; or put another way 73% of the RCTs failed to demonstrate that homeopathy is anything more than a placebo.  Neither does that bare statistic account for the potential for bias in the “positive” trials, the 1-in-20 lottery of statistical significance or the question of clinical significance.  This is hardly a refutation of the Committee’s conclusion.  It’s more like a confirmation.

The idea that Homeoprophylaxis could be an, “easily replicable model” for, “ultramolecular effects” is not supported by the evidence.  Frankly, it’s a homeopathic pipe dream.  With no evidence of effect or credible mechanism, talking about economic benefits is pure fantasy.

“While homeoprophylaxis studies are in principle easy to replicate, one needs to consider that the credibility of study results also hinges on the reputation of the institution which produced it. Study designers should therefore seek the collaboration of respected research institutions.”

Roniger and Jacobs (2010) show some awareness of homeopathy’s credibility gap.  How do they seek to address this?  Rigorous trials perhaps?  No, they want to hitch their wagon to prestigious institutions.  I would say that such institutions should be careful not to allow homeopathic fantasists to ensnare them in this way.  Also, this provides yet more evidence of the desperation of these propagandists.

The bottom line

So, what do I make of this editorial?  Roniger and Jacobs (2010) make the usual mistake of assuming that correlation is causation, falling for the post hoc ergo propter hoc fallacy.  This is bad enough for medical researchers.  But, it gets worse when they try to argue that sheer size compensates for lack of control or randomisation with a disease that depends on local factors (water sources and weather), personal exposure and behaviour.  This shows a tremendously weak grasp of the principles of trial design, further evidenced when they argue that this study is “well-designed” and profess surprise that no decent journal would touch it.  These gaffes should be a source of acute embarrassment for an academic researcher and a doctor.

The obfuscation of the significance of the infection rates – declaring victory when this merely returned to the same level as the rest of the country and the level experienced in the intervention region in 2004 – and the inability to identify alternative explanations mark out the authors as homeopathic apologists and propagandists, not credible researchers.

Neither is it edifying to see an NHS doctor claim that an ethanol and water mix is in any way “similar to a conventional vaccine”.   Finally, the most worrying part of this for me is to see a doctor advancing an ethically indefensible research agenda:

“… other common infections with significant morbidity and/or mortality lacking effective and/or feasible conventional treatment, such as dengue fever, malaria, and antibiotic-resistant bacterial infections should be targeted first.”

This aim is advanced in spite of the homeopaths knowing that they don’t know how their magic water could work.  Claiming that malaria lacks “effective and/or feasible conventional treatment” is unworthy of a doctor.  Finally, antibiotic-resistant bacterial infections are a significant problem in the NHS; even considering treating them with magic water is unconscionable.  Going into print with this nonsense should be terminal for Roniger’s career.

Disclaimer

I try to make sure that what I write is both accurate and fair.  If you think that I have got anything wrong please let me know.  If you are right I will happily change what I have written.

This is not medical advice.  If you need that see a properly qualified and registered doctor.

Related Posts

Here Is the News – my review of The Faculty of Homeopathy press release.

Much ado about nothing – my review of Bracho et al (2010).

Homeoprophylaxis: An idea whose time has come—and gone – Peter Lipson’s more elegant dismissal of Bracho et al (2010) on Science Based Medicine.

Dilutions of Grandeur – my review of an even worse paper on homeopathic prophylaxis and treatment of dengue in Brazil.

Dengue: magic water and the great social mobilization – How  homeopathy can be used as a cover for improving conventional interventions.

Dengue: betting on homeopathy? – Another homeopathic press release.

Notes

* http://www.britishhomeopathic.org/media_centre/news/st_part2.html

References

Guidelines for the treatment of malaria. World Health Organization, 20, avenue Appia, 1211 Geneva 27, Switzerland: WHO Press; 2010. Available from: http://www.who.int/malaria/publications/atoz/9789241547925/en/index.html.

de Almeida LRR, Campos M, Herrera HMM, Bonamin LVV, da Fonseca AHH. Effects of homeopathy in mice experimentally infected with Trypanosoma cruzi. Homeopathy : the journal of the Faculty of Homeopathy. 2008 April;97(2):65–69. Available from: http://dx.doi.org/10.1016/j.homp.2008.02.009.

Bracho G, Varela E, Fernández R, Ordaz B, Marzoa N, Menéndez J, et al. Large-scale application of highly-diluted bacteria for Leptospirosis epidemic control. Homeopathy. 2010 July;99(3). Available from: http://dx.doi.org/10.1016/j.homp.2010.05.009

Jonas WB. Do homeopathic nosodes protect against infection?  An experimental test. Alternative therapies in health and medicine. 1999 September;5(5):36–40. Available from: http://view.ncbi.nlm.nih.gov/pubmed/10484829.

Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV, et al. Are the clinical effects of homeopathy placebo effects?  A meta-analysis of placebo-controlled trials. Lancet. 1997 September;350(9081):834–843. Available from: http://view.ncbi.nlm.nih.gov/pubmed/9310601.

Mroninski CRL, Adriano EJ, Mattos G. Meningococcinum, Its protective effect against Meningococcal disease. Homoeopathic Links;14(4):230–234. Available from: http://www.medizinverlage.de/html/10192051.html.

Roniger H, Jacobs J. Prophylaxis against Leptospirosis using a nosode: Can this large cohort study serve as a model for future replications?  Homeopathy. 2010 July;99(3):153–155. Available from: http://dx.doi.org/10.1016/j.homp.2010.06.004.

Shah-Rossi D, Heusser P, Baumgartner S. Homeopathic treatment of Arabidopsis thaliana plants infected with Pseudomonas syringae. TheScientificWorldJournal. 2009;9:320–330. Available from: http://dx.doi.org/10.1100/tsw.2009.38.

Shang A, Huwiler-Müntener K, Nartey L, Jüni P, Dörig S, Sterne JA, et al. Are the clinical effects of homoeopathy placebo effects?  Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet. 2005 September;366(9487):726–732. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67177-2

Edits

None yet!

[BPSDB]