A canna’ change the laws of physics

Scotty, The Naked Time, stardate 1704.3, Episode 7

Stuck at a Red Light

Posted by apgaylard on July 1, 2008

As part of my follow-up on the single, small, un-replicated study that underpins the claims made for the LloydsPharmacy Allergy Reliever, Medinose and BioNase products, I was encouraged [Hat tip to dvnutrix] to try and get my concerns about the poorly concieved sham therapy used in this work published in the journal that originally carried the article: the Annals of Allergy, Asthma & Immunology.

Now, as the article is ten years old I wasn’t very optimistic.  However, I hoped that the currency of the commercial claims being loaded onto it might persuade the editors to publish a short piece of correspondence.

Unfortnately, my initial misgivings have been confirmed and it has been rejected.  Now, I do have some sympathy with the editors: the comment relates to an old article, is not on a very glamorous topic and would take up space that could be given to a more recent work.

However, I do think it is important that where problems are identified in the open literature they are given an airing in that literature; otherwise how are we to trust what we read?  In this spirit I’d just like to share with you the correspondence and covering letter I submitted, along with the editor’s response.

First, the covering letter (hyperlinks added for this on-line version)

21 June 2008

The Editor
Annals of Allergy, Asthma and Immunology
2500 North State Street
Suite N416
Jackson, MS 39216
USA

Dear Sir:

No Illumination: An Inappropriate Sham in Phototherapy

Please find attached my submission for an item of correspondence for your journal.  It concerns the inappropriate use of no-illumination as a sham treatment for the placebo group in red light phototherapy.

Although the paper that I am concerned about was published in 1997, I think that its continued use as the primary evidence of efficacy for three (BioNase, Medinose, Lloyds Pharmacy Allergy Reliever) current commercial products necessitates a suitable caution being placed in your journal.

My sole, potential, conflict of interest is that I have received, from Lloyds Pharamcy Ltd, a free “Allergy Reliever”.  This currently retails at around £14 sterling.

I hope that you will be able to look upon this submission favourably.

Best wishes,
apgaylard

Now, here’s my (rejected) submission.

No Illumination: An Inappropriate Sham in Phototherapy

To the Editor,

One of the key features of a placebo, or sham, is that it is indistinguishable from the intervention being tested.  In their 1997 publication on red light phototherapy in perennial allergic rhinitis Neuman and  Finkelstein [1] used, “devices with internally disconnected light emitting diodes probes … for sham illumination in the placebo group.”

Devices of the type used in this work cause the subject’s nose to glow red when in operation.  This is so marked as to be obvious to both the subject and anyone in the vicinity.  Clearly, a disabled device will not have this effect.  As a consequence it will have been as obvious, to the trial subjects, as the noses on their faces whether they were in the treatment or placebo group.

Hence, the blinding in this study was clearly broken by the use of an inappropriate placebo and the results of this study should be seen in this light.  This is particularly important as the manufacturer of the device tested continues to cite this work as their key evidence of efficacy in the relief of the symptoms of allergic rhinitis; as do manufacturers of similar devices.

Other workers have, more recently, employed, “low-intensity visible light” [2] as the sham illumination for the placebo group.  Whilst this study did not examine the suitability of this approach in any detail and used light from a different part of the spectrum, it would seem to be a step forward.  Perhaps the most important lesson here is that more careful development is required for sham interventions in phototherapy.

[1]        Nueman I and Finkelstein Y, Narrow-band red light phototherapy in perennial allergic rhinitis and nasal polyposis. Ann Allergy Asthma Immunol.1997; 78:399-406.

[2]        Koreck AI, Csoma Z, Bodai L, et al. Rhinophototherapy: a new therapeutic tool for the management of allergic rhinitis. J Allergy Clin Immunol. 2005 Mar;115(3):541-7.

Here’s the response from the Editor.

Dear Mr Gaylard,

Thank you for submitting your paper for publication in the Annals of Allergy, Asthma & Immunology.

I regret to inform you that the Editor responsible for the evaluation of your paper has advised me that it cannot be accepted for publication. Our decision has been based on many factors, including the appropriateness for this journal.

Often contributors do not realize that we can publish only a fraction of the manuscripts submitted to us. Thus, we cannot accept many meritorious manuscripts. I hope you understand that our editorial decision does not reflect on the quality of your work. You may wish to submit your manuscript to another publication.

Yours sincerely,

 To which I sent a brief response.

Dear Dr [snip],

Thanks for letting me know. I thought that publication was unlikely. However, doesn’t it worry you that a clearly flawed paper remains in the your journal and there’s nothing to alert your readers?

Best wishes
apgaylard

To which I have had, as yet, no reply.  It’s a shame that this paper will continue to be used as the sole justification for the claims made by the manufacterers and vendors of these products, without there being a suitable caution in the journal that carried the paper.  Now, it’s not the fault of those responsible for the journal that commercial interests over-sell this work, but it’s a shame that the use of an obviously faulty placebo was not (apparantly) picked up by the reviewers and doesn’t trouble the current editor.

This provides a small illustration (and I’m sure that there are many more significant examples) of the importance of not accepting papers in peer-reviewed reputable journals at face value.  In the absence of robust procedures and policies the onus is on readers to engage their critical faculties: caveat emptor.

About these ads

9 Responses to “Stuck at a Red Light”

  1. draust said

    Perhaps you could turn it into a “case study”, AP – survey how often the original paper has been cited as “scientific bona fides” for the widget over the years, and point out how this all refers to one rather feeble study. It does make kind of an exemplar of the thin basis of the overblown claims.

    How does the “sham” light in the other study compare with the “therapeutic” light? Do they say?

    Inadequate (or no) placebos are a common problem in alternative science papers, of course. An example is a trial I saw on “earthed mats” for sleeping on, where there was no “internally disconnected” mat or indeed ANY suitable placebo control. Again the trial is cited endlessly in advertising material.

    It would also make an interesting comparison with the citation of the same paper in the REAL literature.. i.e. I wonder if it is ever cited in proper journals, as opposed to in Web advertising material directed at the deeply gullible.

  2. dvnutrix said

    Problem is – citing a proper scientific study does make it look more respectable. And, it allows doctors like Chris Steele to recommend it on GMTV. People who wouldn’t purchase some items, might be persuaded to buy this one because the study stands, unqualified by comment from others.

    There needs to be a way of gathering this sort of material together. Not quite on the scale of BPR3 but not far off from it. Maybe this could be converted into a BPR3 post? I will think and be sensible when I’ve had a little more sleep and am less hot. (The last two will, I hope, excuse any obvious stupidities in this comment.)

  3. claireob said

    It might be worth writing to the ACAAI, the body which produces the Annals. After all, one of their stated objectives is:

    “#
    To be the world leader in providing clinically relevant continuing education reflecting state-of-the-art science in all areas pertinent to the allergic diseases and selected related conditions.
    #

    There’s a list of ACAAI leadership here:

    http://www.acaai.org/Member/Leadership/regents.htm

  4. podblack said

    Hi, I’ve just alerted members of the Skeptic Zone podcast to this story – hope we can be of some help? :)

  5. apgaylard said

    podblack I would appreciate any help with this. They hay fever season has just retuned to the UK and I expect to see the purveyors of these devices making claims based on this work – again.

  6. [...] the trial supporting the sale of these devices (Lloyds is not alone in marketing them) employs a sham sham comparison (it is based on red light therapy and the sham light was internally disconnected thus making the one [...]

  7. Alan said

    Great post. You raise an important issue, and it would be nice to know what the “many factors” are on which they based their decision. You should submit your article elsewhere, though journal editors do have a nasty tendency to stick together.

    Alan

  8. apgaylard said

    Alan Thanks for the comment. I hadn’t thought of trying elsewhere as I think this sort of observation only makes sense as a letter to the journal that published the article. I could, I suppose, try to write something on placebo design based on this. I fear that may be beyond my abilities.

    An interesting development is that a trail of the Lloyds Pharmacy Allergy Reliever has just been published. I don’t have access to the journal, but I have written to the lead author to ask about their placebo design.

    Emberlin JCC, Lewis RAA. Pollen challenge study of a phototherapy device for reducing the symptoms of hay fever. Current medical research and opinion. 2009 May; Available from: http://dx.doi.org/10.1185/03007990903024699.

  9. [...] van dubbele blindering is dan geenszins sprake. Na het schrijven van voorgaande vond ik overigens dit artikel op een weblog, dat hetzelfde probleem aan de kaak stelt. De auteur ervan heeft het tijdschrift [...]

Sorry, the comment form is closed at this time.

 
Follow

Get every new post delivered to your Inbox.

Join 612 other followers

%d bloggers like this: