The Myth of The Secret Eight

There seems to be a persistent myth among homeopathic apologists that the final eight homeopathic studies (large and of high quality) analysed in the famous Lancet paper by Shang et al. are somehow secret.

Here is a recent example of this piece of homeomythology:

“… When Shang et al (Lancet 2005) did a metaanalysis of 110 homeopathy trials, homeopathy was significantly better than placebo. When they restricted it to “high quality” trials (n=20), homeopathy was still significantly better than placebo, same as the ‘matched’ biomedical trials. They then restricted it to “large, high quality” trials (n=8), and claimed they were no better than placebo. Does anyone here know which 8 homeopathy RCTs were selected? Failing that, the 20 “high quality” trials, from which one could identify the 8 largest. And how and when did they set the cutpoint for “large”?”

“Chamfort”, Comment is Free, Guardian, Comment No. 941849, November 22 13:11

So, is it possible to find which eight trials of homeopathy were finally selected?  Yes.  In fact one poster did provide an answer to this question.  He noted that “…The 8 homeopathy trials and 6 conventional trials that were considered to be large, high quality trials are identified by the authors in their reply to comments on the original article. See Lancet 366 (2005); p. 2083. The list of included trials can be found as an appendix to the online version of the original article, see Lancet 366 (2005); 726-732.”

Where did this idea come from?  Well, Shang et al. did omit these details from their original paper.  This was picked up by many apologists, including correspondents to the Lancet: Fisher et al., Linde  and Jonas, Walach et al., and Dantas.  Shang et al. then provided the information, as noted above, in the same issue of the Lancet as the critical letters.

But, as we have seen, the complete story has been missed by some homeopathic apologists.  It would seem that they only read the critical correspondence, or quotations from it.  This ‘secrecy’ myth has now passed into homeopathic folklore.

Here’s a very recent comment from a pro-homeopathy blog:

“…Another oft-raised issue is that homeopathy cannot stand up to conventional medical trials, and that homeopathic studies fail meta-analysis. It is suggested that homeopaths “cherry pick” the results of studies to confirm positive results. Iris Bell MD Phd, director of research at the Program in Integrative Medicine at the University of Arizona College of Medicine, stated the following in response to a negatively biased article in the British medical journal “The Lancet” in August of 2005;

“The researchers started with 110 homeopathy studies and 110 conventional medicine studies, but drew their main conclusions from 8 homeopathy and 6 conventional studies. And in an odd decision by the journal and the researchers, those 14 studies were not identified. This makes it impossible for others to challenge or confirm their conclusions. Given the tiny number left, they really were under an obligation to tell the reader which they used.” …”

http://goodscience.wordpress.com, 24^th November 2007.

This quotes an article that Bell had published in the Journal Of Alternative And Complementary Medicine before the additional information was made available by Shang et al.

So, this criticism gets made over and over again; every time the meta-analysis of Shang et al. is discussed.  It makes me wonder why.  Perhaps perceiving themselves as upholders of an unchanging tradition, homeopathic apologists don’t think that the argument moves on?  Maybe a paranoid world view that includes the great allopathic conspiracy makes them think that such omissions are made with malign intent and evil scientists won’t correct them?  Perhaps it’s just sloppy scholarship?

In fact, quite a lot of detail about the studies included and excluded by Shang et al. is freely available online.  On the website of Shang’s home institution, Institut für Sozial und Präventivmedizin at Universität Bern, there are three files that can be downloaded:

“… List of excluded homoeopathy studies (PDF 32KB) provides a list of excluded homoeopathy studies with reasons for exclusions.

Study characteristics of homoeopathy studies (PDF 72KB) provides details on study characteristics and results of 110 placebo-controlled homoeopathy trials.

Study characteristics of allopathy studies (PDF 72KB) provides details on study characteristics and results of 110 placebo-controlled allopathy trials. …”

This information was originally posted on December 17^th, 2005 and subsequently corrected (for an error in the graphs) on December 23^rd, 2005.

It’s easy to use the second file to determine the identity of the ‘secret eight’:  select the trials identified as of high quality, list them by size and pick the eight largest.  The results of doing this are shown in Figure 1.  The ‘secret eight’ are obvious; as is the cut-off criterion.

Figure 1  The Secret Eight Revealed

Table 1 provides details of the intervention used, clinical indication and outcome.  Two of the ‘secret eight’ studies were of individualised homeopathy.  Others used single or multiple preparations and two were of proprietary homeopathic ‘medicines’.

High Quality Study	n	Intervention	Indication	Outcome
Rottey (80)	501	Mucococcinum 200K	Influenza like disease	Occurrence of symptoms
Vickers (94)	400	Arnica 30X	Physical activity	Soreness
Papp (71)	334	Oscillococcinum®	Influenza like disease	Global assessment, patient
Schmidt (84)	208	Thyroidinum 30C	Fasting	body weight
Labreque (55)	162	Thuya 30C, Ant 7C, Nitr acid 7C	Warts	Global assessment, physician
Jacobs (46)	116	Individual treatment	Diarrhoea	Duration of diarrhoea
Weiser (97)	104	Euphorbium compositum®	Sinusitis	Global assessment, other
Walach (96)	98	Individual treatment	Headache	Pain
Jacobs (49)	81	Individual treatment, 30C	Diarrhoea	Duration of diarrhoea
Jacobs (47)	75	Individual treatment	Otitis media	Global assessment, other
Hart (41)	73	Arnica C30	Hysterectomy	Pain
Wiesenauer (102)	72	Galphimia glauca D4	Pollinosis	Global assessment, other
Zell,von, (105)	69	Traumeel®	Sprain Joint	angulation
Böhmer (15)	67	Traumeel®	Sports injury	Global assessment, patient
Vickers (95)	57	Arnica 30C, Rhus t 30C, Sarc a 30C	Physical activity	Soreness
Chapman (21)	50	Individual treatment	Traumatic brain injury	Activities
Jawara (50)	50	Arnica 30C, Rhus t 30C	Physical activity	Soreness
Tveiten (91)	46	Arnica D30	Physical activity	Soreness
Stevinson (86)	42	Arnica C6	Hand surgery	Pain
Lepaisant (58)	36	Folliculinum 9C	Premenstrual syndrome	Global assessment, other
Chapman (22)	10	Individual treatment	Premenstrual syndrome	Global assessment, other

Table 1.  Details of the High Quality Homeopathic Studies Identified by Shang et al. 

The use of single preparations, or set ‘medicines’, in clinical trials is often criticised by homeopaths; however they seem happy enough when the results appear to be  positive.

I have also yet to see evidence of concerted criticism from the homeopathic community of high street retailers like Boots selling ‘off the shelf’ homeopathic preparations for treating particular conditions.

Homeopathic apologists do tend to miss the point of Shang et al.  This analysis demonstrates (again) that removing bias, trials of low quality and small size progressively diminishes the apparent clinical effects of homeopathy to the point where they cannot be distinguished from a placebo.  This is just what would be expected if homeopathy was just a placebo.  Any apparent effects in excess of this are just so much statistical noise.

So, finally, let’s dispel this particular piece of homeomythology:  the identity of the final eight, high quality relatively large, homeopathic trials selected by Shang et al. are not secret.  They have been in the public domain for nearly two years.