Dilutions of Grandeur

I’ve just come across an article in a new peer-reviewed open-access journal which claims that, “The use of homeopathy may be a useful adjunct the control of epidemics” based on a public health campaign against Dengue which included homeopathy. 

So I’ve given it a read through, and I must say it’s pretty poor.  Perhaps that’s why it’s published in the International Journal of High Dilution Research (formerly Cultura Homeopática) rather than a proper journal. 

If that assessment seems harsh, I hope that the following will bear it out. 

Anyway, Dengue is a pretty unpleasant disease.  It is an, “acute, infectious, mosquito-borne fever that is temporarily incapacitating but is rarely fatal.”  It may give rise to dengue hemorrhagic fever (DHF) which, if left untreated, leads to a, “usually fatal condition known as dengue shock syndrome”. [Encyclopædia Britannica (2009)] 

Unfortunately there is no specific therapy, aside from symptomatic relief and the support of blood volume in the case of DHF. [See the CDC Dengue Brochure for more details.] 

The lack of a specific therapy and likelihood that most people will recover anyway are probably among the factors which make this an attractive proposition for an uncontrolled homeopathy trial. 

In this post, I’m just going to focus on the methodology used in the trial and the results that were obtained.  I’m not really interested in the details of the ‘remedies’ or the history of placebo-interventions for the ‘control’ of epidemics. [See here for some comments on why homeopathic interventions of the past are not relevant to the present, let alone the future.] 

The Author and His Aims

The author, Laila Aparecida de Souza Nunes (MD, BC Homeopathy), is the Municipal Coordinator of Collective Health, Municipal Secretary of Health, Macaé, Rio de Janeiro.  According to the paper: 

“From the 1990s on, the Brazilian public health system underwent deep changes in its political, administrative and organizational structure, allowing for innovation in the planning and management of health-care facilities, including administration of non- conventional medicines. In this context, phytotherapy, homeopathy and acupuncture were included within the practices offered to users of the national public health system (Sistema Único de Saúde – SUS) in Macaé county, state of Rio de Janeiro.” [Emphasis mine]

So it’s clear that the author has a particular agenda.  This is not necessarily a problem – as long as it doesn’t cloud his judgement.  However, the presence of any dubious data handling may indicate that he has not been able to look at the subject with sufficient objectivity. 

The stated aims of the study were: 

“[…] to assess the contribution of homeopathy to the control of the outbreak of dengue in Macaé in 2007, as a complementary strategy and significant coadjuvant, seeking to evaluate its effectiveness and to measure the acceptance by the general population.” 

Essentially this is posing the questions: would adding homeopathy to conventional interventions make a difference?  And, would people take the homeopathic pills? 

How The Trial Was Done

This is essentially an uncontrolled, open, clinical trial.  Two “Campaigns” were undertaken, in April and November of 2007. 

Homeopathic remedies were dispensed (with conventional interventions) along with “Educational materials”.  Health workers also received specific training and, “a “Routine for the assistance of patients suspected of dengue” included a protocol for the use of homeopathy.”  This resulted in the dispensing of 156,000 ‘doses’ between April and May 2007 and 60,000 ‘doses’ between November and December 2007.  (Estimated population of Macaé was 180,000 inhabitants). 

How sure anyone can be sure that people took the homeopathic potion is unclear: the author states that, “Adherence by the population was spontaneous or voluntary”; whatever that means. 

In addition remedies were given to 129 symptomatic patients.  In these cases the, “disease was diagnosed at the moment of receiving the homeopathic remedy or later, through clinical evaluation.” They were seen at an outpatient clinic and provided, “a 5 ml. vial of homeopathic remedy, to take 5 drops p.o. 3 times a day for one week.”  Again, how can anyone be sure they complied? 

Given that this study is published in the International Journal of High Dilution Research it’s no surprise that the levels of dilution are paradigmatic: 

“All three remedies were used in homeopathic dilution 30cH [10^-60], due to the safety guaranteed by a dilution surpassing Avogadro’s number, making virtually insignificant the possibility of side-effects” 

Of course, by surpassing the Avogadro limit to this degree there is in all liklihood no ‘active’ ingredient left in the remedies. 

So, to summarise, there were no: controls, blinding or anyway of knowing if the patients had taken their remedies.  And the homeopathic intervention was delivered along with standard practices:

“[…] control of the vector, epidemiological surveillance and training of both health workers and the general population […] supportive and pharmacological management, as well as clinical follow up […]”

As the author states, “confounding factors were not controlled for”.  I’ll say. 

Given the methodological limitations on view, the results would need to be very spectacular indeed for any conclusions to be legitimately drawn in favour of homeopathy. 

And Now The Results  

Starting with the symptomatic patients, the author reports:

“[…] Follow up showed they presented relief of symptoms from the very beginning of treatment and complete remission was earlier (average = 5 days), without complications, by comparison to patients who did not use homeopathy (average = 8.6 days). None of the homeopathically treated patients progressed to DHF and only 2 were hospitalized, for intravenous hydration.” 

This is effectively a small open-label clinical trial.  The author is claiming a 3.6 day improvement in remission compared to an unspecified control group.  Who were the patients who didn’t use homeopathy? 

Given that the campaign initially delivered 156 000 ‘prophylactic’ doses to in population of around 180 000, I would really like to know how these people escaped.  Were they given doses and not comply – that may indicate a tendency to ignore health messages.  Were they missed by the campaign – missing out on mosquito-bite avoidance advice perhaps?  Maybe they had received their ‘prophylactic’ doses, but chose not to take treatment? 

Perhaps it wasn’t offered to them – in which case I would like to know how that choice was made: were the ‘clinicians’ blinded?  Was allocation to homeopathy random, or the more tractable cases cherry-picked?  We don’t know because the author hasn’t reported on this important aspect of his work.

Neither do we know how many were in the ‘control’ group or how similar they were to the 126 ‘treated’ cases. 

Also, this data was collected during follow-up.  I wonder how long after the patients experienced remission this was.  The author doesn’t say.  Given that they knew they were getting a ‘medicine’ how can confirmation bias be accounted for? 

Finally, no supporting statistical data is provided.  There is no indication whether the mean remission times are (statistically) significantly different. 

In the presence of such poor reporting, how can this result be trusted? Next the ‘global’ results: 

“The homeopathic intervention was followed by a dramatic fall in the number of cases when compared with previous years. (Figure 1) From January to March 2008, 35 cases of dengue were reported in Macaé. In all of them, the disease was benign and no deaths were reported.”

Comparing against previous years is problematic.  This is a mosquito-borne disease, so weather will have a strong bearing: were all years equal in terms of the mosquito population?  Also, this trial is part of a campaign which seems to have been sparked by a bumper year for Dengue cases in 2007.  Were there similar campaigns in previous years?  From the data provided it is clear that the number of reported cases varies considerably year to year. 

I have plotted the data from Figure 1 (below).  It’s clear that 2007 was far and away the worst year, of those reported.  It’s also clear that there were very few reported cases in 2004 and 2005: only 31 and 17, respectively.  This, apparently, was without any help from homeopathy. 

Monthly report of cases of dengue 2002-2007 in Macaé. Reconstructed from the data provided under Figure 1 in Nunes (2008)

There also seems to be some temporal cherry-picking going on here.  Why choose to aggregate January to March, when the previous April and May were worse? 

Report of cases of dengue during January to March 2002-2007 in Macaé. Data taken from Nunes (2008)

Anyway, comparing the same period for the other years shows that three of the six previous years had fewer reported Dengue cases during January to March than 2008. (2004: 12; 2005: 5 and 2006: 10).  From this data there doesn’t seem to be anything remarkable about 2008.

“[…] Annual variation in cases reported in Macaé mirrored the figures reported in the state of Rio de Janeiro from 2000 to 2007, when there was a higher peak in Macaé – triggering the homeopathy campaign. From that moment on, a dramatic fall was recorded, not observed in the remaining localities (Figure 2). The same phenomenon could be observed in the north of the state (where Macaé is located) (Figure 3) […]”

 Figure 2 is interesting for a number of reasons.  It has Macaé’s full-year data for 2008^* – so why wasn’t this used instead of the first-quarter comparison made in the paper?  There again, that’s a bit odd because the paper was submitted on 21^st November 2008 and published during December – perhaps these data are not for the full year? Interestingly the data go back to 2000 – why does Figure 1 only go back to 2002? 

It also shows that Macaé tends to have lower numbers of reported cases, on the average, than the other regions.   In fact, fewer cases were reported in 2000 and 2005 than in 2008.  Also the result for 2006 seems pretty close to 2008. 

The optimistic homeopath says that, “From that moment on, a dramatic fall was recorded, not observed in the remaining localities” but he is ignoring similar steep falls year-to-year in other areas: Litoral Sul Fluminense between 2006 and 2007 for instance. 

Figure 2 shows that reported cases vary year-to-year on something like a five-year cycle.  There’s pretty strong evidence that all we have here is an example of regression to the mean. 

Anyway, our optimistic homeopath ploughs on heedless of his own data: 

“[…] To compare rates with other counties, we chose those in the state of Rio de Janeiro that in January 2007 had an incidence of Dengue similar do Macaé’s, i.e. 20 cases/100,000 residents. (Figure 4) In these counties, Dengue rates rose dramatically between January 2007 and January 2008, whereas the rate in Macaé plummeted by 93%. (Figure 5) […]”

Setting aside the issue of random variation, another obvious question to ask here is: were there equivalent non-homeopathic campaigns in these counties?  

Figure 5 is odd.  I read it (and I could be wrong) as recording 29 cases in Macaé for 2008^**.  This is not consistent with the claim of 35 cases in the first three months or the annual totals shown in figures 2 and 3, which look like being around a couple of hundred. 

Finally, the 93% drop in Dengue cases claimed between January 2007 and January 2008 is not unprecedented: 2003 to 2004 shows exactly the same percentage decrease. 

I must say that I am not convinced.  The only piece of statistical analysis (working out a couple of means) lacks virtually all the relevant details needed to make sense of it (SDs, CIs, any details about the control group). 

The commentary on the graphed data ignores trends and results that don’t fit with the authors disclosed position on homeopathy. 

There is no evidence here for anything other than the confluence of statistical variation, cherry-picking, confounding and wishful thinking. 

What Does the Author Conclude? 

Unsurprisingly for a homeopath he takes the view that: 

“The use of homeopathy in outbreak control complemented more conventional public health actions carried out by the Health Secretary of the county of Macaé […]” 

It is important to note that there is absolutely nothing in this paper to support the homeopathic triumphalism sometimes associated with discussions about the control of epidemics.  This is about homeopathy as an adjunct, not an alternative.   

Even so, there isn’t enough in the paper to make that limited case.  If all the reporting deficiencies were corrected, it is clear from the evidence provided that what happened to the incidence of Dengue in Macaé between 2007 and 2008 is entirely unremarkable.  Add in the uncontrolled confounding factors and the evidence weakens further. 

Now Nunes adds his final flourish: 

On this occasion, considering that health is an integrated activity, the creation of a Nucleus of Integrative Therapies was approved, providing professional health assistance in homeopathy, acupuncture, phytotherapy, homeopathic and phytotherapic pharmacy. 

I think that only a CAM advocate (or someone whose livelihood depended on it) could see extraordinarily weak evidence ‘for’ homeopathy as justifying other, entirely unrelated – and in the case of phytotherapy contradictory – complementary therapies. 

The contents of this paper don’t justify the use of homeopathy – let alone anything else. 

In the interests of fairness I’ve e-mailed the author and posed the following questions: 

1. Did the other regions, which you compare Macaé against, benefit from a similar public health campaign against Dengue?^*** (By similar I mean having all the characteristics of the campaign in Macaé expect for the use of homeopathy)
2. You say that, “From January to March 2008, 35 cases of dengue were reported in Macaé” and yet Figure 5 appears to show 29 cases for 2008.  Am I misreading Figure 5?^**
3. Your main claim relates to the first three months of 2008.  What were the rates of Denge for the other eight months?
4. Given that your paper was submitted on 21^st November 2008, how is it that Figures 2 and 3 seem to have the incidence data for 2008? Is this data only for part of the year? If so, how many months?^*
5. Why do you claim that the homeopathy element of your campaign was effective when 2004 and 2005 showed only 31 and 17 reported cases, respectively; and between 2003/2004 the reported cases fell by 93%?
6. When discussing the symptomatic patients treated with homeopathy you say that: “remission was earlier (average = 5 days), without complications, by comparison to patients who did not use homeopathy (average = 8.6 days)”.  What were the standard deviations for these averages?  What were the (95%) confidence limits?  Were the differences between these two averages statistically significant?
7. The paper gives no details for the control group used to assess the symptomatic patients. How large was it? How similar was it to the treatment group? How was this group selected? Was allocation to the homeopathic treatment made randomly? What allocation method was used? How long after complete remission were patients in the treatment and control groups visited, to record their count of the number of days this took?

It will be interesting to see what he has to say.

Overall this has been a disappointing introduction to a new peer-reviewed open-access journal.  I like the idea of evidence being in the open – it facilitates debate.  It’s a shame that the quality of the evidence isn’t better. 

Still as the International Journal of High Dilution Research is just a re-branded homeopathy journal the ‘peers’ are just as likely to be homeopathy fans as this author obviously is. 

References

Nunes LAS. Contribution of homeopathy to the control of an outbreak of dengue in Macaé, Rio de Janeiro. Int J High Dilution Res [online]. 2008 [cited 2009 Mar 08]; 7(25): 186-192. Available from: http://www.feg.unesp.br/~ojs/index.php/ijhdr/article/view/315/374.

Another paper on homeopathic ‘control’ of Dengue appears in the same issue:

Marino R. Homeopathy and Collective Health: The Case of Dengue Epidemics. Int J High Dilution Res [online]. 2008 [cited 2009 Feb 19]; 7(25): 179-185. Available from: http://www.feg.unesp.br/~ojs/index.php/ijhdr/article/view/312/373.

An excellent critique can be found on Michael Grayer’s nontoxic blog.

Some replies by the author and the content of a Letter submitted to the journal are discussed in the post: Dengue: magic water and the great social mobilization

Acknowledgements

al_capone_junior on the Bad Science forum for pointing out the three month cherry-picking. 

Mojo on the Bad Science forum for the reference to Marino (2008) and Michael Grayer’s critique

Edits

8^th March 2009. Two more questions added to the list submitted to the author; querying the use of apparantly full-year data for 2008 when the paper was submitted before the year-end and asking for detials of the control group used for the symptomatic patient comparison.

 

9^th March 2009. Reference to Marino (2008) and Michael Grayer’s critique added

13^th March 2009. Some random spelling mistakes corrected.

2^nd May 2009. Author’s clarifications added as notes.

Notes

The author has kindly clarified a number of issues raised in this post. These notes provide his unedited explanations.

^*Figures 2 and 3, the chart shows the updated data until April 16, 2008.

^**35 cases was the absolute number, until March 30 and 29 was the incidence rate is calculated per 100,000 inhabitants, and was last updated on 04 April 2008.

^***The regions and municipalities perform the vector control, epidemiological surveillance and health education work in a manner similar to Macaé. The difference is the use of Homeopathy and the great social mobilization

In addition to the distribution of homeopathic medicine, intensive work was done for education in health for the population in the control of the vector, the symptoms of the disease, the flow of care, the expected effect of homeopathy as prophylaxis, and dissemination of dial-dengue, which is a telephone in which the user complains that may have local outbreak of mosquito, and is also for people with dengue notify us to take the measure of the vector control and also the “lock” with the homeopathic medicine, as a blocking “vaccine” in the patient’s residence and the neighborhood. So the shares are made, many times before that health services make the reporting of the disease. It also made the mapping of the areas with the highest rate of infestation of the larvae and greater number of cases of disease in these places and make joint efforts to identify and eliminate the outbreaks, guidance on the control of mosquitoes and also with the prophylactic use of homeopathic medicine. With all this, many people become aware of prevention methods in relation to dengue and seek health services as soon as symptoms begin, thus avoiding agravações disease.

[BPSDB]