Dengue: magic water and the great social mobilization
Posted by apgaylard on May 2, 2009
Back in March I wrote about a study on the homeoapthic prophylaxis and treatment of dengue fever in Macaé (Rio de Janeiro, Brazil) It appeared in a re-branded open-access homeopathy journal grandly called the International Journal of High Dilution Research .
It really wasn’t very good and I sent some questions to its author. He has responded a number of times and seems to be a decent sort. He cleared up a number of reporting problems with the paper (I’ve put some explanatory notes in the original piece).
Some interesting things have come out of our dialogue. He confirmed what I had suspected: the 126 symptomatic individuals who were treated for dengue had not been allocated to this part of the trial at random. Neither had those who formed the ‘control’ group.
This is but one failing among many in this part of the study: no blinding, no placebo used in the control group, no measures of variance for the mean results or any tests of statistical significance. Not that the last point matters much; to quote Fisher, “The theory of estimation presupposes a process of random sampling. All our conclusions within that theory rest on this basis; without it our tests of significance would be worthless.”
Clearly, this part of the work is of no value. It should not have passed peer review; but that it did speaks eloquently about the quality of the peers.
The most interesting part of his reply concerned the attempted population-level prophylaxis. Its effectiveness was assessed by comparing the reported incidence of dengue in the area where it was deployed with other areas where it was not. Therefore it is absolutely critical that, as far as possible, the only difference between the dengue control strategy in these areas was that homeopathy was used in Macaé.
The paper explains that homeopathy was used alongside conventional control strategies. So I wondered whether the localities used as comparitors had benfitted from similar conventional support: was homeopathy the only difference?
The author replied that:
“The regions and municipalities perform the vector control, epidemiological surveillance and health education work in a manner similar to Macaé. The difference is the use of Homeopathy and the great social mobilization“
A “great social mobilization” sounded to me like a significant extra effort to control dengue in Macaé by non-homeopathic means. So I asked for clarification and the author kindly elaborated:
“In addition to the distribution of homeopathic medicine, intensive work was done for education in health for the population in the control of the vector, the symptoms of the disease, the flow of care […] and dissemination of dial-dengue, which is a telephone in which the user complains that may have local outbreak of mosquito, and is also for people with dengue notify us to take the measure of the vector control […] It also made the mapping of the areas with the highest rate of infestation of the larvae and greater number of cases of disease in these places and make joint efforts to identify and eliminate the outbreaks, guidance on the control of mosquitoes […] With all this, many people become aware of prevention methods in relation to dengue and seek health services as soon as symptoms begin, thus avoiding agravações [aggravated] disease”
These are a massively more plausible explanation for any real reduction in the number of cases of dengue than the application of drops of water containing (for all practical purposes) none of the intended ingredients. Or, as the author put it in the original paper:
“All three remedies were used in homeopathic dilution 30cH [1 part in 1060], due to the safety guaranteed by a dilution surpassing Avogadro’s number, making virtually insignificant the possibility of side-effects.”
Having clarified a number of issues with the author, I think that the journal is the best place to put the essentials into the public domain. So, here is a letter that I have just submitted for publication. It will be interesting to see if the journal is prepared to publish it.
Dengue: No Evidence Homoeopathy Works, No Reason Why It Should
Sir,
I read with interest the recent paper by Nunes [1] and have the following observations.
A Flawed Trial
It is clear that no conclusions can be drawn from the reported treatment of 126 symptomatic patients. The author claims that, for these patients, “remission was earlier (average = 5 days), without complications, by comparison to patients who did not use homeopathy (average = 8.6 days)”.
It is notable that means are cited without any measure of variance (such as standard deviations or confidence limits) or statistical significance. In the absence of such data claims of actual differences in remission times cannot be sustained.
In addition, it seems that the trial did not blind either patients or clinicians to the treatment, or use an equivalent placebo in the control group. The allocation of patients to treatment or control groups was not randomised. These methodological flaws mean that the difference in remission times is most likely caused by a combination of bias and expectation effects.
Regression to the Mean
From the data provided in the paper (Figure 2) it is clear that dengue infection rates are cyclical, with an approximately five to six year period: peaks in one year are usually followed by dramatic falls the next year. This means that 2008 was likely to show a substantial fall in dengue infections, even in the absence of the homoeopathy campaign.
Fair Comparisons
As the intervention in Macaé did not include a control group, it is important to ensure that comparisons with different areas or the same location for previous years are fair.
For instance, the author does not disclose whether the comparator regions operated a conventional public health campaign against dengue. If they did not, it is likely that any relative improvement Macaé is attributable to the conventional aspects of this campaign, rather than the novel homeopathic element, such as:
“[…] control of the vector, epidemiological surveillance and training of both health workers and the general population […] supportive and pharmacological management, as well as clinical follow up […]“
Neither does the author establish that comparator regions are of similar socio-economic make-up to Macaé.
It also seems peculiar to aggregate reported infections for January to March, when infections during 2007 peaked in April and May. This would appear to run the risk of missing the months with the greatest potential for dengue infection.
An Unremarkable Result
Commenting on the start of the homoeopathy campaign the author states, “From that moment on, a dramatic fall was recorded, not observed in the remaining localities”. And yet the data show that similar “dramatic falls” are not uncommon.
The main claim for control of dengue in the population of Macaé relates to the first three months of 2008. Totalling the data provided in Figure 1 for this period in other years reveals that three of the previous six years had fewer reported dengue cases during these months than the thirty-five claimed for 2008. (2004: 12; 2005: 5 and 2006: 10).
Looking at the data for complete years, Figure 1 reveals that there were only 31 and 17 reported cases of dengue in Macaé during 2004 and 2005, respectively. Also, between 2003 and 2004 dengue incidence fell by 93%. Finally, Figure 2 shows a similar decrease Litoral Sul Fluminense between 2006 and 2007.
From these data the reported dengue infection rates for early 2008 are certainly not without recent precedent.
Unsupported Conclusions
I wholeheartedly agree with the author that, “confounding factors were not controlled for.” However, the methodological flaws identified preclude the conclusion that homoeopathy made any appreciable contribution to public health in this case. This view is strengthened by the paper not addressing itself to more likely alternative explanations, namely: bias, expectation effects, regression to the mean, inappropriate comparisons and natural variation in the incidence of dengue infections.
This view, rather than the author’s conclusions, is compatible with the outcome of the only RCT of homoeopathy for dengue currently listed in PubMed. A remedy containing Aconita, Bryonia, Eupatorium perforatum, Gelsemium, and Rhus toxicodendron was trialed in Honduras by Jacobs et al. [2]. They concluded, “the results […] do not suggest that this combination homeopathic remedy is effective for the symptoms that are characteristic of dengue fever.”
In the absence of any credible supporting evidence for the efficacy of the homeopathic intervention reported, it is surprising that the author appears to see this work as some sort justification for the promotion of, “acupuncture, phytotherapy, […] and phytotherapic pharmacy.” None of these other alternative approaches were tested and phytotherapy directly contradicts many of the central tenets of homeopathy: similia similibus curentur and the ‘law of infinitesimals’, for example.
This paper provides no credible evidence that homoeopathy made any difference to dengue infections in Macaé during the first three months of 2008. The vanishingly small probability that any molecules of the ingredients were actually administered to anyone during this trial remains the best available explanation for this.
References
- Nunes LAS. Contribution of homeopathy to the control of an outbreak of dengue in Macaé, Rio de Janeiro. Int J High Dilution Res [online]. 2008 [cited 2009 Mar 08]; 7(25): 186-192. Available from: http://www.feg.unesp.br/~ojs/index.php/ijhdr/article/view/315/374
- Jacobs, J., Fernandez, E. A., Merizalde, B., Avila-Montes, G. A., and Crothers, D. The use of homeopathic combination remedy for dengue fever symptoms: a pilot RCT in Honduras. Homeopathy. 2007. 96: 22–26.
The author has told me that a double-blind randomized study is being planned at the moment. I look forward to seeing the results.
As for the putative prophylactic intervention in Macaé, maybe regression to the mean or natural variation in the incidence of the disease just made it look like it worked. However, I do think that there is a chance the campaign worked: it’s just that it wasn’t the magic water drops that did it.
If it did work, the author provides us with a much more sensible explanation: the vector control, education, awareness and surveillance elements of the “great social mobilization.” Now that’s a hypothesis worth testing.
References
Nunes LAS. Contribution of homeopathy to the control of an outbreak of dengue in Macaé, Rio de Janeiro. Int J High Dilution Res [online]. 2008 [cited 2009 May 02]; 7(25): 186-192. Available from: http://www.feg.unesp.br/~ojs/index.php/ijhdr/article/view/315/374
Fisher RA. Development of the theory of experimental design. Proceedings of the International Statistical Conferences. 1947 [cited 2009 May 02]; 3:434–39. Available from: http://digital.library.adelaide.edu.au/coll/special/fisher/212.pdf
Jacobs, J., Fernandez, E. A., Merizalde, B., Avila-Montes, G. A., and Crothers, D. The use of homeopathic combination remedy for dengue fever symptoms: a pilot RCT in Honduras. Homeopathy. 2007. 96: 22–26. doi:10.1016/j.homp.2006.10.004. PMID: 17227744.
Acknowledgements
I would like to thank Dr Laila Aparecida de Souza Nunes MD, BC (Homeopathy) – Municipal Coordinator of Collective Health, Municipal Secretary of Health, Macaé, Rio de Janeiro – for taking the time to reply to my questions and engage in an open, constructive discussion.
Edits
3rd May 2009. Typos and the odd spelling mistake corrected.
[BPSDB]
12 Responses to “Dengue: magic water and the great social mobilization”
Sorry, the comment form is closed at this time.
A canna’ change the laws of physics 
zeno27 said
I’m glad we have people like you who understand these things sufficiently to be able to understand and criticise them thoroughly. Keep up the good work!
apgaylard said
zeno27: I appreciate the sentiment, though I have to say that I’m still learning and the medical stuff takes me outside my comfort zone.
Pages tagged "placebo" said
[…] bookmarks tagged placebo Dengue: magic water and the great social mobilizat… saved by 5 others nightlyangel225 bookmarked on 05/04/09 | […]
dvnutrix said
As ever, admirable persistence and a fine write-up. It is encouraging that the author was willing to discuss various aspects of the paper with you and engage in a critical discussion of the findings, claims, confounders etc.
bengoldacre said
absolutely brilliant stuff as ever.
we can only assume that much of the alternative medicine literature is similarly flawed, and has been so for many years. these deeprooted failings are only now coming to light through the magical power of the internet to bring cheeky nerds in contact with duff research.
it’s also fascinating to me that torpedoing bad methodologies is such a great teaching tool for EBM. just goes to show what a massive missed opportunity universities have made with all these rigorless top secret BSc degrees in alt med.
apgaylard said
Thanks both for the kind words. As a happy amateur I’m always reassured when people who have a background in things medical make such comments. This is tempered by a slight disappointment that medical people of the alt.med persuasion don’t make it harder.
I’ve just found out that Int J High Dilution Res is going to publish my letter in the next issue (June). Very quick service; I’m assuming it means they are struggling for content and/or don’t submit letters to much scrutiny.
dvnutrix: I must say that I’ve been pleasantly surprised by the openness of the author; much more helpful and polite than other alternativists I’ve contacted. He’s even sent me additional data. I’ve let him know about the letter I’ve submitted and it will be interesting to see how he responds.
Ben: I am sure you are right about the alt.med literature. To be fair Int J High Dilution Res is not exactly a top of the line alt.med journal. The tendency to see the ‘magic’ as the cure when it’s taken along with conventional (for want of a better word) treatment is something I’ve come across on a personal level. I’d say that it’s likely to be a pervasive mind-set and I’m not surprised to see it writ large here.
pleick said
There’s not much to add to the previous comments: as always, great work.
By the way, while the international journal of high dilution research may not be the most prestigious homeopathy journal, compare the constructive reply of the author to those you received for your letters to Homeopathy! I hope that if the author chooses to reply to your letter, she’ll maintain this constructive tone and won’t try to dodge the actual issue.
Now, a thing that intrigues me: there must be a market for this type of article!
I’ve looked at a few alt.med. publications myself. Much less than you have, and only the type where I’m in no danger of getting completely lost with my physics/engineering background. It seems to be common that authors will go out of their way to use unspectacular data to justify their pet theory. They will try their best to present this data in such a way that it seems to support the theory, ignore alternative or more mundance explanations and write abstracts and conclusions that are much more optimistic than they should be.
I may underestimate this, but my impression is that – at least in principle – understanding how proper trials and statistical analyses should be done isn’t that difficult. And with all the controversy surrounding alt.med., controversy that its proponents should be aware of, it’s not as if the typical problems of poorly designed trials weren’t well known.
In this particular case, it’s not really the trial design that bothers me – but how the results are presented in journal articles.
Which brings us back to the question of where the “market” is for this type of article.
apgaylard said
pleick: Having a similar physics/engineering background I tend to keep to methodological critiques on the basis that if there is anything called ‘science’ (which I’d argue that there is) then there must be common values and methods (my working definition of science = knowledge + values + methods) that someone from our background can appreciate the absence or distortion of.
I do hope that the cooperative tone will continue. I think that your proposal of a common structure for such studies is spot on.
On the market for work of this kind, all I can say is that I picked up the original reference from a homeopathy propaganda blog, so I guess that it’s grist for their mill. Another entry in various lists of “positive” results. As the author is involved in public health, I’d also assume that it will be cited as evidence for policy-makers in that region; which is worrying if true.
Thanks for your comment, it’s appreciated as always.
pleick said
I wholeheartedly agree. I’ll add that working scientists are not perfect and I myself am no exception. We all have our favorite little theories and we try to promote them in our publications. We tend to overestimate the arguments in favor of our theory, and to underestimate the ones that speak against it.
This seems not too different from so-called “alternative” science: their proponents generally have the same tendency, but take it to a more extreme level.
I’d argue that the key difference between “proper” and “alternative” science lies in the values-part of your definition: if a real scientist overestimates his case, another one will eventually point this out, and he’ll probably enjoy doing so. In “alternative science”, the most glaring contradictions and errors get their citations and happily remain in the literature. This never ceases to baffle me.
Nunes’ paper on dengue fever seems rather weak in the knowledge and methods sections, but OK as far as the values go. Which is why I really hope that the discussion will remain constructive. That said, the prospect of a well conducted RCT trial doesn’t bother me too much: the resources involved for the homeopathic medicines are rather small, and if both the verum and the control group get the “great social mobilization”, it will do more good than harm.
apgaylard said
philippe: Thanks for an interesting comment. I think that it boils down to us all having the same human frailties. That’s why, to my mind, the values aspect of science is so important. It’s values like: the social and professional acceptability of strong mutual criticism, constructive engagement with criticism, limiting the influence of the experimenter over the results, putting in the open all the reasons why you might be wrong, putting your pet theories to the test, etc. which help stop real science from drifting into pseudo (or pathological) science.
Of course, the values of all scientists aren’t equally strong. That’s probably why some drift off into pseudoscience or fraud.
I’d generally agree with your analysis of Nunes work: the methods are weak and a key paper (Jacobs et al) from the homeopathic ‘knowledge base’ has been overlooked. I would say though that while Nunes seems to appreciate the value of constructive engagement with criticism; what I would identify as key scientific values: limiting the influence of experimenter (and subject) bias and looking seriously at why you might be wrong, are not given sufficient weight.
Still no one is a scientific paragon and Nunes is the most constructive and polite homeopath that I’ve engaged with so far.
michaelgrayer said
Excellent stuff – and thank you for the comment over at http://www.nontoxic.org.uk. It is very pleasing to see that the author has engaged in a constructive discussion. It certainly makes a change from receiving insults about taking “allopathic drugs” that some would wish to engage in…
You may have motivated me to do the same thing for the paper that I have covered – if I can find the time away from my PhD that is.
One comment I would like to add is that I find the fact that a double-blind randomized study on the efficacy of homeopathy for treatment of dengue is being planned quite disturbing. Although an RCT is the best study design for testing efficacy, there are all sorts of ethical issues concerning the use of a large number of human subjects for the trial of a treatment for which there is no plausible mechanism by which it can work, and the preliminary observational studies are of such poor quality. This is where medical research differs substantially from physics or engineering (please correct me if I’m over-generalising here). Conducting such a trial also involves the funding body buying lots and lots of doses – if I were a Brazilian citizen I would be deeply concerned about the use of public money to line the pockets of the manufacturers of homeopathic remedies regardless of which way the results go.
Ben: I quite agree with the use of alt.med literature to teach the scientific method. I would particularly like to advocate doing so in the department where I’m based – a humanities department. Making a case for overhauling the teaching of stats and the scientific method is meeting great resistence…
apgaylard said
michael Thanks for the comment. I’d encourage submitting a letter – at least it puts the critique in the same domain as the article. It didn’t take me long to knock up and the editor accepted it within a couple of days.
I take your point about trial design. You can add to your concerns that Jacobs et al have already got a negative RCT result. The best thing that can be said is that this trial will not involve withholding a good treatment, as there is no treatment for dengue. However, supportive care is important and in the original non-trial at least the homeopathy was an adjunct to the standard care and I’d expect that they’d follow the same protocol this time. Not any sort of justification, I know, but at least it limits the potential for harm.
Along with the ethical issues around the effect on people of being in a trial, for which there is no prospect of benefit, the obvious draw-back is, as you point out, the waste of resources.
To be fair to the alt.med people, in my little brach of engineering we do have issues around blinding, randomisation, and sample size for subjective testing of the product. Though I’ll not elaborate further as I like to keep by blogging life separate from my professional role.
Thanks for stopping by.