A canna’ change the laws of physics

Scotty, The Naked Time, stardate 1704.3, Episode 7

Novogen and the RCN’s placebo pushing algorithm

Posted by apgaylard on December 15, 2008

a-conservative-clinical-approach-for-treatment-of-menopausal-symptomsIn part three of my two-part series (here and here) on The Royal College of Nursing’s “Complementary approaches to menopausal symptoms – RCN guidance for nurses, midwives and health visitors” I pursue a niggling doubt about the evidence base for red clover isoflavones: the RCN seemed keen on them around the same time the NIH was saying, “studies suggest that it is not effective in reducing hot flashes.” 

The RCN’s evidence centres on a 1996 paper published by Nachtigall and co-workers.  I was so intrigued I bought a copy from the British Library. 

I thought I better have a read as it provides the only evidence offered by the RCN for the efficacy of red clover isoflavones.  They also adopt a modified version of Nachtigall et al.’s ‘treatment algorithm’; this recommends the use of red clover and soy isoflavones along with Black Cohosh

Having read the paper, it is clear that the RCN misrepresent its contents when they say:

“A review of placebo-controlled studies of a standardised 40mg red clover isoflavone demonstrated a reduction in the number of hot flushes experienced compared with placebo (Nachtigall, 2006).” [Emphasis mine]

natchigall_quote

Nachtigall LE, Baber RJ, Barentsen R et al. (2006) Complementary and hormonal therapy for vasomotor symptom relief: a conservative clinical approach, Journal of Obstetrics and Gynaecology Canada, 28(4), p. 281.

From this, I was expecting Nachtigall et al. to demonstrate that red clover reduces the frequency of hot flushes relative to placebo; at least to a statistically significant degree: it does not. I have reproduced the relevant passage from the paper (right).

It does cite an (at the time) unpublished meta-analysis by Thompson Coon et al.  However, the RCN guide fails to tell its readers is that the result of this meta-analysis was not statistically significant: the 95% confidence interval for hot flash frequency ranges from a reduction of 2.94 hot flashes per day to an increase of 0.03.  In other words, the confidence interval includes zero (effect). 

This meta-analysis was subsequently published, in January 2007.  The authors concluded, “There is evidence of a marginally significant effect of T. pratense [red clover] isoflavones for treating hot flushes in menopausal women. Whether the size of this effect can be considered clinically relevant is unclear.”  This is hardly a ringing endorsement: It’s not a statistically significant result and neither are they convinced of its clinical significance.  I also think that it is misleading to call a non- (statistically) significant result, “marginally significant”. 

Nachtigall et al. also mentions that only two of the five trials they reviewed reached a statistically significant difference between red clover and placebo: Van de Weijer (2002) and Jeri (2002).  These were the two smallest studies, both with 30 participants. 

The information provided in the paper also shows that the three largest trials did not show a significant difference between red clover and placebo.  This is glossed over in the text. 

For instance Barber et al. (1999) reported on a trial with fifty one subjects.   The authors noted that, “The combined values for all subjects, regardless of treatment group, revealed a strong negative correlation between the level of urinary isoflavone excretion and the incidence of hot flushes.” 

They concluded, “These data do not indicate a therapeutic benefit from dietary supplementation with isoflavones in women experiencing menopausal symptoms.”  They did suggest that, “the apparent placebo effect in many studies of menopausal symptoms may be attributable to dietary sources of isoflavones.”  This may be an excuse, or it may have merit; in either event dietary supplementation is not supported by this study. 

Similarly Knight et al. (1999) reported a smaller 12-week trial (n=37).  In this study, “women were randomized to three treatment groups: placebo, 40 mg or 160 mg, [red clover isoflavones] delivered in tablet form.”  The results showed, “no significant difference in the incidence of flushes between the three groups at trial conclusion”; or any other outcome for that matter. 

Both of these studies are small.    However, unlike the RCN guide, Nachtigall et al. do include (but don’t discuss) the best trial evidence available to date: Tice et al. (2003)

This is far and away the largest of the trials discussed (n=252).  In fact it had more participants than the other four trials combined.  It was also published in the most prestigious journal: JAMA.  Participants were randomly assigned to one of two commercially available red clover based supplements:  Promensil or Rimostil; or, “an identical placebo”.  

They concluded that, “Although the study provides some evidence for a biological effect of Promensil, neither supplement had a clinically important effect on hot flashes or other symptoms of menopause.”

So, after chasing down the references red clover does not appear offer any better alleviation of symptoms than a placebo.  The RCN have, again, stretched the truth in order to make the intervention look better than it is.  Nachtigall et al. also seem to be trying to accentuate the positive. 

On the basis of this flimsy evidence the RCN give a clear endorsement to three placebo-interventions they have talked-up: 

“Nachtigal [sic] et al. (2006) suggest a stepped approach to treating menopausal symptoms based on the severity of symptoms – from integrating lifestyle modifications and lifestyle treatments through to use of HRT (see Algorithm of conservative management of menopausal symptoms on page 20).” 

This ‘algorithm’ basically recommends that women with mild to moderate vasomotor symptoms are told to make sensible lifestyle changes and use three complementary therapies: red clover isoflavones, soy isoflavones and black cohosh

These placebos are also recommended for the, “severely symptomatic” along with non-HRT prescription medications; as an alternative to HRT. 

This is makes a total nonsense of the RCN’s counsel that its members should require recommended interventions have a good evidence base

The other thing that niggles is that the guide claims (wrongly) that other complementary therapies work.  For instance, it states that, “Homeopathic remedies have been demonstrated to reduce hot flushes and to improve quality of life” (They haven’t).  And that acupuncture trials have, “shown benefits in relieving hot flushes, night sweats and general mood, especially when site specific points for menopause have been used” (They haven’t). Yet these placebos don’t get a look in:  It seems that not all placebo therapies are created equal.  I wonder why, when the RCN claim they work? 

Perhaps, at least for the recommended use of red clover isoflavones, the RCN may have unconsciously favoured the people who sponsored their guide, Novogen? 

A note on the guide points out that it was,

Kindly supported by an unrestricted educational grant from Novogen“. 

Novogen make Promensil; they call it, “A clinically supported, natural product, with 40mg of red clover isoflavones to help relieve hot flushes/flashes and night sweats associated with menopause and may help to maintain bone and heart health.” (By this definition, “clinically supported” seems to mean: shown by the best available evidence to be nothing more than a placebo.) 

The RCN guide misrepresents the trail data it cites for red clover and ignores the best trail conducted to date.  Is it an innocent coincidence that the positive evidence they talk-up comes from small studies using Promensil, whilst the high-quality negative trial not even mentioned in the guide (Tice et al., 2003) said Promensil didn’t have, “a clinically important effect on hot flashes or other symptoms of menopause”?  I hope so. 

Come to mention it, Nachtigall et al. seemed keen to put a positive spin on a few small trials and statistically non-significant results.  As their paper acknowledges, “Dr Nachtigall has done research into the effect of isoflavones on clotting for Novogen.  Dr Barentsen has received a research grant from Novogen … Ms Wysocki is a columnist for the Novogen website …” Hopefully, that, again, is just a coincidence. 

Now, favouring their sponsor wouldn’t explain the RCN’s enthusiasm for soy isoflavones and black cohosh; so the enthusiasm for an intervention that Novogen sells may just be innocent coincidence.  Either way, when you are torturing the evidence to make a recommendation in favour of a therapy the company sponsoring your work supplies- whatever the motive – it doesn’t look good. 

In any event, the RCN are making treatment recommendations for interventions that are not supported by the evidence.  This can’t be good for an organisation which claims that it, “represents nurses and nursing, promotes excellence in practice and shapes health policies.”

[BPSDB]

The RCN, CAM and the menopause – Part one – The herbs don’t work?

RCN, CAM and the menopause – Part two – Credulous nonsense

5 Responses to “Novogen and the RCN’s placebo pushing algorithm”

  1. Blogring for placebo…

    Related Blog Entries…

  2. dvnutrix said

    This enthusiasm for red clover isoflavones is very disappointing. The RCN guide needs to be re-written or withdrawn until such time as it can be written in such as way as to be an adequate and fair reflection of the current evidence.

    As ever – meticulous work.

  3. apgaylard said

    dvnutrix: Thank you for your continued interest in my musings. I agree that the RCN guide needs a serious overhaul. I’ve written to the RCN, summarising the points I made in my previous two posts. I’ll more than likely publish that along with their response, if any, in due course.

    Now I’ve been able to check out Nachtigall et al. I’ll be getting onto them about red clover now as well.

  4. karlokeeffe said

    Hi Apgaylard,

    First up I’d just like to say this is a great blog, I’ve been avidly reading through your back catalogue of posts.

    I really appreciate the detail you go into when constructing your arguments. Especially the quoting of papers with references.

    With regard to the Nachtigall LE, Baber RJ, Barentsen R et al. (2006) paper, and specifically the passage you reproduced, reading of that passage does seem to support the quote you gave from the RCN guide. The RCN guide almost seems to be a rewording of the passage.

    Spotting the fact that the confidence interval including zero makes the result no longer statistically significant is easy to miss (In fact I don’t think I would have realised it, had you not pointed it out!).

    It seems like very poor writing in the Nachtigall paper, which deliberately leads readers to believe the result is significant!

    Do you know of any resources that help with analysing research papers such as this. I am interested in reviewing the data on yoga, and it’s claimed benefits, but would like to avoid getting caught out by over enthusiastic conclusions such the one on the Nachtigall paper.

    Once again, thanks for the great blog. Keep it up!

  5. apgaylard said

    karlokeeffe:
    Thank you for the kind words and taking the time to read some of my stuff. For guidnace on analysing papers I’d suggest R. Barker Bausell’s excellent Snake Oil Science. I’ve done a post which summarises the guidance from the book.

    Trisha Greenhalgh’s How to Read a Paper is often recommended, though I’ve not got around to reading it yet.

Sorry, the comment form is closed at this time.

 
%d bloggers like this: