A canna’ change the laws of physics

Scotty, The Naked Time, stardate 1704.3, Episode 7

Holland and Barrett promise to behave

Posted by apgaylard on December 5, 2008

bigstockphoto_hand_outstretched_with_a_pile__1993865The Advertising Standards Authority (ASA) has let me know that they have concluded two of the three complaints I raised about a recent flurry of e-mailed advertising from Holland and Barrett.  I was concerned that the adverts in question directly associated particular herbs and supplements with implicit unfounded health claims: improving mental healh and beating the menopause to be specific.

Holland and Barrett have reportedly assured the ASA, “that these emails will not be used again and that in future campaigns they will not use headings or product categories that imply health benefits for the listed products.”

It is interesting that they had taken these directly from their website, but were not able or prepared to justify them.  Websites don’t fall within the domain of the ASA – this job falls to trading standards; so I’ll be asking them to take a look at the health claims Holland and Barrett make on the website.

 The ASA are still looking at the issue of Gingko Biloba and claims of improved cognitive function.  This is interesting as the ASA have accepted claims about this herb improving the cognitive function of healthy individuals in the past.  They have since explained their position to me:

 “The basis on which we have been accepting advertisers making limited claims about the mental and physical benefits of using Ginkgo Biloba was an investigation undertaken by the ASA in 2001, when a report by an academic in the field of Pharmacognosy was commissioned. The conclusion of this report was that most research appeared to suggest that there was sufficient proof Ginkgo Biloba was likely to improve short-term memory and blood-flow in healthy individuals.

 As this report is now over seven years old – and after having contacted the expert who wrote the original report – we have concluded that there are sufficient grounds for re-investigating this issue, in light of more recent evidence.”

 This seems very reasonable.  I am glad that the ASA are prepared to review their position.  However, it does raise an interesting question: what should the threshold level be for the ASA to accept claims for various interventions?  The typical natural history of CAM interventions is that initial promise is seen in small, methodologically poor, early trials which evaporates under closer scrutiny. 

Given this, the current evidential bar for advertisers seems to be set too low; generating unnecessary work and confusion.  But how high is high enough?  Looking at the natural history of the evidence for an arbitary set of CAM interventions certainly won’t provide an answer, but it may provide a hint or two. 

The correlation between quality and time is in one sense obvious:  early trials are bound to be relatively small.  Until some preliminary work has been done, and promise shown, large trials are not very likely to get funded. 

But, with CAM, it is arguable that something more is at work.  Early work often appears to be done by enthusiasts; and they start from an assumption that the intervention works.   Maybe this leads to, not just small, but methodologically poor initial studies.

So the passage of time becomes a proxy for both trial size and quality.

A trend of this type can be seen with acupuncture:

“Over the last three decades, a huge number of clinical trials have tested whether or not acupuncture is effective for treating a variety of disorders.  Some of these trials have implied that acupuncture is effective.  Unfortunately most of them have been without adequate placebo control groups and of poor quality – the majority of positive trials are therefore unreliable. 

By focussing on the increasing number of high-quality research papers, reliable conclusions from systematic reviews make it clear that acupuncture does not work for a whole range of conditions, except as a placebo.” (Ernst and Singh, 2008

And with homeopathy:

“We conclude that in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results.” (Linde et al., 1999

“… Studies of high methodological quality were more likely to be negative than the lower quality studies …” (Cucherat et al. 2000

“… smaller trials and those of lower quality showed more beneficial treatment effects than larger trials and those of higher quality. ” (Shang et al. 2005

Studies of herbs, like those sold by Holland and Barrett, also show the same natural history.  For instance, in the case of Black Cohosh NCAM observe:

“This herb has received more scientific attention for its possible effects on menopausal symptoms than have other botanicals. Studies of its effectiveness in reducing hot flashes have had mixed results. A study funded by NCCAM and the National Institute on Aging [Ed. Newton et al. 2006] found that black cohosh, whether used alone or with other botanicals, failed to relieve hot flashes and night sweats in postmenopausal women or those approaching menopause. Other research suggests that black cohosh does not act like estrogen, as once was thought.”

A recent study of Ginkgo biloba (DeKosky et al., 2008) commented that:

“Clinical trials of efficacy of G biloba have focused primarily on patients with dementia. In 1998, a meta-analysis of early trials reported a small benefit, but a Cochrane Collaboration Review in 2007 found that the evidence for benefit of G biloba on cognition in individuals with dementia was not convincing. An intention-to-treat analysis from one trial involving individuals already diagnosed with AD showed a small positive effect size over 52 weeks but also experienced a dropout rate approaching 50%.  Several small, short-term randomized trials have had mixed results … A recently reported feasibility trial randomized 118 older adults, mean age 87 years, to 240 mg daily of G biloba vs placebo and did not find a reduction in dementia incidence after an average follow-up of 3.5 years in an intention-to-treat analysis.”

 It finally concluded:

“In this study, G biloba at 120 mg twice a day was not effective in reducing either the overall incidence rate of dementia or AD incidence in elderly individuals with normal cognition or those with MCI.” 

It certainly looks like letting the dust settle would be a good idea; though it appears that this can take a long time.  For acupuncture, Ernst and Singh imply that it has taken three decades for a clear picture to emerge. 

There again, if the effects of an intervention are clear and dramatic then the dust should settle much more quickly.  If it takes a decade or three to see: the odds are we’re not missing out on much. 

Neither is it helpful for advocates to continue producing small, poor quality trials.  Only a progressive increase in trial size and quality can really help move the evidence base forward. 

In the absence of obvious benefits at least waiting for large, well-conducted, multi-centre RCTs to hit the literature would seem to be a sensible idea before allowing advertisers to make specific therapeutic claims. 

Perhaps a better idea would be to wait until a Cochrane review comes to a clearly favourable conclusion. (e.g. one that doesn’t have the word ‘promising’ in it.)

Anyway, if anyone can think of a better way to set the evidential bar for advertisers, I’d be happy to hear about it. 



6 Responses to “Holland and Barrett promise to behave”

  1. […] Sean Kelley wrote an interesting post today onHere’s a quick excerpt […]

  2. dvnutrix said

    As ever, well-persevered on following this up. I’m heartened that the ASA is minded to review its position on Gingko in the light of more recent work and a more substantial corpus of evidence.

    [I]f anyone can think of a better way to set the evidential bar for advertisers, I’d be happy to hear about it.

    Excellent thought-experiment and I shall think about this further.

  3. apgaylard said

    dvnutrix: Thanks for your kind comment. The ASA do seem to be pretty reasonable. I must admit, they have a difficult job deciding what level of evidence suitable to sustain advertising claims. I look forward to any insight that you might bring to this puzzle.

    Also, thanks for pointing out the DeKosky paper.

  4. dvnutrix said

    When it comes to health claims for products I think that there is a strong need for a weighting system. How this weighting system needs to be constructed needs some public consultation but…for the general purpose health claim of these adverts, there should be some joined-up thinking with Trading Standards:
    i) you can’t make a claim for a product if your own product does not demonstrably, and through verifiable testing, have the same active ingredient in approximate quantities as those in the papers from which you draw your support
    ii) depending on the claim, you must be making a claim for a population for which the product has been studied
    iii) depending on the implied size effect, you need an appropriate number of participants over the trials that have been done
    iv) you can’t cherry pick your trials in support of your particular supplement etc.

    It’s a bit pathetic but there does need to be an overlap with Trading Standards. I’m fairly sure that there needs to be a wider consultation but am aware that there may be difficulties because the standards of evidence may end up being more rigorous than those that are needed to have a product supplied by the Prescriptions Authority (magnetic dressings come to mind).

  5. apgaylard said

    dvnutrix: Good stuff! Thanks. I must say that I find the split between trading standards and the ASA arbitrary and unhelpful: for instance, what is the screed on a website if not advertising?

  6. apgaylard said

    dvnutrix:”you can’t make a claim for a product if your own product does not demonstrably, and through verifiable testing, have the same active ingredient in approximate quantities as those in the papers from which you draw your support”

    The ASA have recently said (about Fish Oil): “We also considered that it was not possible to tell from that document, or from the list of studies, whether the product tested in those trials was given to the child participants in the same formulation or dosage as the Brain Boosters supplement.”

    Great minds?

Sorry, the comment form is closed at this time.

%d bloggers like this: