A canna’ change the laws of physics

Scotty, The Naked Time, stardate 1704.3, Episode 7

Happy people, not places

Posted by apgaylard on October 24, 2008

In August, the BBC ran a story claiming that research had determined that some places in Britain were ‘happier’ than others.  You can see the nature of the claims in the TV news report, “Britain’s happiest places mapped“.  There was a big problem with this though: the research found no significant differences between places.  The only differences were accounted for by the socio-economic status of the people

If you read through the on-line version of the story it even pointed this out, eventually

A number of bloggers picked up on this facile churnalism (here, here and here).  Ben Goldacre covered it in his Guardian column.  I entered into a dialogue with one of the journalists responsible, as did gimpy.  After judging that this had run its course, I complained to the BBC’s Editorial Complaints Unit (ECU).  My complaint has just been upheld.  So, I would like to share with you the full text of the adjudication.  It makes some very interesting observations. 

22 October 2008

Britain’s happiest places mapped, BBC News website

I have now had an opportunity to look into the points you made in your letter of 8 September and in previous correspondence with Richard Warry. 

You have directed me to note 7 in the notes to editors appended to the Royal Geographical Society press release about the research in question, and to a passage of the RGS technical notes.  It may be helpful if I set them out for reference: 

7. The estimated ranks that were used to produce this press release provide a guide of the order of the differences by area before and after accounting for socio-demographic composition and do not directly imply better or worse places to live in terms of this particular happiness measure. Note that the residuals underlying the ranks for the model that takes into account socio-demographic characteristics are not statistically significant from each other. The only areas that have statistically lower (worse) levels of well-being than the rest of Britain are “Cynon Valley; Rhondda” and “Merthyr Tydfil; Rhymney Valley; Taff-Ely” in Wales. However, these differences are explained by socio-demographic differences in these areas.  (My emphases)

 And:

 We find it that socio demographic variations from district to district do explain the differences, in the sense that all places are essentially similar with respect to happiness when we take into account the differences their socio-demographic composition.

In simple terms, I take this to mean that such differences as were found by the research reflected the different compositions of the population groups studied, and that the research yielded no basis for suggesting that place affected happiness.

Mr Warry, having acknowledged the validity of your points, undertook to “beef up the reference regarding the statistical power of the story“.  He hasn’t kept a copy of the story as it stood before he altered it, but tells me the alteration essentially consisted of the addition of a paragraph which reads:

The variations they uncovered between different places in Britain were not statistically significant – suggesting that other studies would come up with different findings.

As we are agreed that it would be inaccurate to give the impression that the research had established a link between place and happiness the question before me is whether the alteration offsets that impression.  For the following reasons, I don’t believe it does.

The added paragraph follows one which says “But the team from the Universities of Sheffield and Manchester stress that happiness is more a product of personal circumstances than physical location“.   In that immediate context, I don’t see that it would be clear to readers of the item that the absence of statistical significance meant that the research had yielded no grounds for viewing happiness as in any degree “a product of  … physical location“.  The six paragraphs which follow, which refer to “a map of happiness” drawn up by the research and list a number of the places supposedly found to be more or less happy, continue to foster an impression which the research doesn’t in fact justify, and the remainder of the item, though rightly focussing on the importance of socio-demographic considerations in relation to the research, still leaves it to be inferred that physical location is to some extent a determinant of happiness, even if not a major one.

While I agree with Mr Warry’s view that this was essentially a light-hearted story, I nevertheless think that, even in its amended form, it conveys a significantly inaccurate impression, and I am therefore upholding your complaint.  In due course a summary of my finding, together with a note of the action taken as a result of it, will be posted on the complaints page of bbc.co.uk, and I shall ensure that you are sent a copy.  Meanwhile, I hope that you will accept my apologies on behalf of the BBC and my thanks for giving us the opportunity of looking into your concerns.  I hope that it will give you some reassurance if I add that the Editor of the website in question has decided that further changes to the item should be made in the light of your points.

(For reference, here is the original story, which Mr Warry has apparantly lost: Britain’s happiest places mapped; along with the initial revision: Britain’s happiest places mapped (revision 1))

First I would like to thank the BBC’s Head of Editorial Complaints for his very fair evaluation of the story.  And, of course, none of this means that happiness is not at all related to where you live: just that there is currently no research evidence to support this hypothesis – so the BBC shouldn’t go around pretending that there is.

Neither is the BBC solely to blame here – the RGS press release, “Geographers map happiest places in Britain“, effectively invited journalists to fall into this trap: the caveats buried in the notes made a nonsense of the ‘headline’.  They surely knew better and I hope that they were driven by embarrassment to remove it from their website.

However, I don’t think that it is unreasonable to expect that BBC journalists critically engage with their sources, rather than just regurgitating any old PR twaddle.  If they are not going to do this, what are they for?

Finally, I think that it’s a shame that the real story of the research – that it, “highlights the issue of social justice and cohesion” as the author (allegedly) put it, was obscured by the trivial ‘light news’ agenda increasingly pursued by the BBC’s journalists.

[BPSDB]

About these ads

11 Responses to “Happy people, not places”

  1. gimpy said

    Well done!

    However, Mr Warry does not come out of this well at all. It is simply unbelievable that he should ‘lose’ the original copy of the story, what kind of an editor does not keep records of their edits? It also needs pointing out that a video piece, even worse than the original article, is still up on the BBC site – http://news.bbc.co.uk/1/hi/uk/7585449.stm

  2. apgaylard said

    Gimpy: Thanks. The video is my next target…

  3. Mark Pack said

    Excellent news. Well done for pursuing this.

  4. soveda said

    Good result, hope the video complaint gets dealt with as appropriately.

  5. dvnutrix said

    Again, well persevered, and it is good to see that the ECU agreed with you. Churnalism is bad enough but to have been enabled in it by the RGS who seem to have willingly distorted the serious purport of the research (at a time when it looks like ‘academic research’ is semantically equivalent to ‘irrelevant and a waste of tax-payers’ money’) and supplant it with a trivial take is deplorable. It serves no-one’s agenda unless you are an estate agent in Powys.

    Maybe somebody should tell the journalist about News Sniffer? I ran a search stating the source and the URL of the original story:

    source:bbc url:*7584321*

    Which delivered these versions of the story 0-6; you’ve already provided a link to the original release. Is it possible that some of these web content reporters need a training session in, well, how to use the web to track versions and original sources?

  6. apgaylard said

    Mark Pack, soveda and dvnutrix: Thanks for the kind words. It just takes a letter or two and a bit of motivation, rather than any real skill. I must say that The BBC’s Head of Editorial Complaints has handled four of these on my behalf in a very fair and perceptive manner.

    I was surprised that Mr Warry had lost the original piece, as gimpy says this doesn’t come over very well. I have sent in a copy I kept and hope that will help him out.

    I’m very impressed with dvnutrix‘s sniffer skills; they exceed mine by some margin. I tend to rely on printing copies of things I critique to pdf.

    Once more, thank you all for taking an interest.

  7. [...] news now though: the BBC has ruled on a complaint over its story and decided that, after all, a news story should be accurate. Share and [...]

  8. draust said

    Yes, nice work AP. Or, as my students would say: “Kick ass, bro'”

  9. blacktriangle said

    Great work, really poor standards at BBC online, who are being played like suckers by PROs.

  10. gimpy said

    Just a postscript regarding the video I linked to earlier.

    I discussed this with Mr Warry around the time apgaylard submitted his complaint. Mr Warry made clear he was not responsible for the video but he did defend it saying his colleagues wanted a ‘well I never’ moment. A moment that was planned to the extent of sending a camera crew and journalist to Hae-on-Wye to spend half a day gathering vox pops from the population but not planned to the extent that they had the press release checked by somebody with a basic knowledge of statistics. ‘Well I never’ indeed.

  11. [...] Cox points to a recent post by science blogger APGaylard concerning a complaint to the BBC about an alleged case of “churnalism” on the corporation’s news [...]

Sorry, the comment form is closed at this time.

 
Follow

Get every new post delivered to your Inbox.

Join 609 other followers

%d bloggers like this: