Holland and Barrett promise to behave

The Advertising Standards Authority (ASA) has let me know that they have concluded two of the three complaints I raised about a recent flurry of e-mailed advertising from Holland and Barrett.  I was concerned that the adverts in question directly associated particular herbs and supplements with implicit unfounded health claims: improving mental healh and beating the menopause to be specific.

Holland and Barrett have reportedly assured the ASA, “that these emails will not be used again and that in future campaigns they will not use headings or product categories that imply health benefits for the listed products.”

It is interesting that they had taken these directly from their website, but were not able or prepared to justify them.  Websites don’t fall within the domain of the ASA – this job falls to trading standards; so I’ll be asking them to take a look at the health claims Holland and Barrett make on the website.

 The ASA are still looking at the issue of Gingko Biloba and claims of improved cognitive function.  This is interesting as the ASA have accepted claims about this herb improving the cognitive function of healthy individuals in the past.  They have since explained their position to me:

 “The basis on which we have been accepting advertisers making limited claims about the mental and physical benefits of using Ginkgo Biloba was an investigation undertaken by the ASA in 2001, when a report by an academic in the field of Pharmacognosy was commissioned. The conclusion of this report was that most research appeared to suggest that there was sufficient proof Ginkgo Biloba was likely to improve short-term memory and blood-flow in healthy individuals.

 As this report is now over seven years old – and after having contacted the expert who wrote the original report – we have concluded that there are sufficient grounds for re-investigating this issue, in light of more recent evidence.”

 This seems very reasonable.  I am glad that the ASA are prepared to review their position.  However, it does raise an interesting question: what should the threshold level be for the ASA to accept claims for various interventions?  The typical natural history of CAM interventions is that initial promise is seen in small, methodologically poor, early trials which evaporates under closer scrutiny. 

Given this, the current evidential bar for advertisers seems to be set too low; generating unnecessary work and confusion.  But how high is high enough?  Looking at the natural history of the evidence for an arbitary set of CAM interventions certainly won’t provide an answer, but it may provide a hint or two. 

The correlation between quality and time is in one sense obvious:  early trials are bound to be relatively small.  Until some preliminary work has been done, and promise shown, large trials are not very likely to get funded. 

But, with CAM, it is arguable that something more is at work.  Early work often appears to be done by enthusiasts; and they start from an assumption that the intervention works.   Maybe this leads to, not just small, but methodologically poor initial studies.

So the passage of time becomes a proxy for both trial size and quality.

A trend of this type can be seen with acupuncture:

“Over the last three decades, a huge number of clinical trials have tested whether or not acupuncture is effective for treating a variety of disorders.  Some of these trials have implied that acupuncture is effective.  Unfortunately most of them have been without adequate placebo control groups and of poor quality – the majority of positive trials are therefore unreliable. 

By focussing on the increasing number of high-quality research papers, reliable conclusions from systematic reviews make it clear that acupuncture does not work for a whole range of conditions, except as a placebo.” (Ernst and Singh, 2008) 

And with homeopathy:

“We conclude that in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results.” (Linde et al., 1999) 

“… Studies of high methodological quality were more likely to be negative than the lower quality studies …” (Cucherat et al. 2000) 

“… smaller trials and those of lower quality showed more beneficial treatment effects than larger trials and those of higher quality. ” (Shang et al. 2005) 

Studies of herbs, like those sold by Holland and Barrett, also show the same natural history.  For instance, in the case of Black Cohosh NCAM observe:

“This herb has received more scientific attention for its possible effects on menopausal symptoms than have other botanicals. Studies of its effectiveness in reducing hot flashes have had mixed results. A study funded by NCCAM and the National Institute on Aging [Ed. Newton et al. 2006] found that black cohosh, whether used alone or with other botanicals, failed to relieve hot flashes and night sweats in postmenopausal women or those approaching menopause. Other research suggests that black cohosh does not act like estrogen, as once was thought.”

A recent study of Ginkgo biloba (DeKosky et al., 2008) commented that:

“Clinical trials of efficacy of G biloba have focused primarily on patients with dementia. In 1998, a meta-analysis of early trials reported a small benefit, but a Cochrane Collaboration Review in 2007 found that the evidence for benefit of G biloba on cognition in individuals with dementia was not convincing. An intention-to-treat analysis from one trial involving individuals already diagnosed with AD showed a small positive effect size over 52 weeks but also experienced a dropout rate approaching 50%.  Several small, short-term randomized trials have had mixed results … A recently reported feasibility trial randomized 118 older adults, mean age 87 years, to 240 mg daily of G biloba vs placebo and did not find a reduction in dementia incidence after an average follow-up of 3.5 years in an intention-to-treat analysis.”

 It finally concluded:

“In this study, G biloba at 120 mg twice a day was not effective in reducing either the overall incidence rate of dementia or AD incidence in elderly individuals with normal cognition or those with MCI.” 

It certainly looks like letting the dust settle would be a good idea; though it appears that this can take a long time.  For acupuncture, Ernst and Singh imply that it has taken three decades for a clear picture to emerge. 

There again, if the effects of an intervention are clear and dramatic then the dust should settle much more quickly.  If it takes a decade or three to see: the odds are we’re not missing out on much. 

Neither is it helpful for advocates to continue producing small, poor quality trials.  Only a progressive increase in trial size and quality can really help move the evidence base forward. 

In the absence of obvious benefits at least waiting for large, well-conducted, multi-centre RCTs to hit the literature would seem to be a sensible idea before allowing advertisers to make specific therapeutic claims. 

Perhaps a better idea would be to wait until a Cochrane review comes to a clearly favourable conclusion. (e.g. one that doesn’t have the word ‘promising’ in it.)

Anyway, if anyone can think of a better way to set the evidential bar for advertisers, I’d be happy to hear about it. 

[BPSDB]

Advertisement