Happy people, not places

In August, the BBC ran a story claiming that research had determined that some places in Britain were ‘happier’ than others.  You can see the nature of the claims in the TV news report, “Britain’s happiest places mapped“.  There was a big problem with this though: the research found no significant differences between places.  The only differences were accounted for by the socio-economic status of the people. 

If you read through the on-line version of the story it even pointed this out, eventually. 

A number of bloggers picked up on this facile churnalism (here, here and here).  Ben Goldacre covered it in his Guardian column.  I entered into a dialogue with one of the journalists responsible, as did gimpy.  After judging that this had run its course, I complained to the BBC’s Editorial Complaints Unit (ECU).  My complaint has just been upheld.  So, I would like to share with you the full text of the adjudication.  It makes some very interesting observations. 

22 October 2008

Britain’s happiest places mapped, BBC News website

I have now had an opportunity to look into the points you made in your letter of 8 September and in previous correspondence with Richard Warry. 

You have directed me to note 7 in the notes to editors appended to the Royal Geographical Society press release about the research in question, and to a passage of the RGS technical notes.  It may be helpful if I set them out for reference: 

7. The estimated ranks that were used to produce this press release provide a guide of the order of the differences by area before and after accounting for socio-demographic composition and do not directly imply better or worse places to live in terms of this particular happiness measure. Note that the residuals underlying the ranks for the model that takes into account socio-demographic characteristics are not statistically significant from each other. The only areas that have statistically lower (worse) levels of well-being than the rest of Britain are “Cynon Valley; Rhondda” and “Merthyr Tydfil; Rhymney Valley; Taff-Ely” in Wales. However, these differences are explained by socio-demographic differences in these areas.  (My emphases)

 And:

 We find it that socio demographic variations from district to district do explain the differences, in the sense that all places are essentially similar with respect to happiness when we take into account the differences their socio-demographic composition.

In simple terms, I take this to mean that such differences as were found by the research reflected the different compositions of the population groups studied, and that the research yielded no basis for suggesting that place affected happiness.

Mr Warry, having acknowledged the validity of your points, undertook to “beef up the reference regarding the statistical power of the story“.  He hasn’t kept a copy of the story as it stood before he altered it, but tells me the alteration essentially consisted of the addition of a paragraph which reads:

The variations they uncovered between different places in Britain were not statistically significant – suggesting that other studies would come up with different findings.

As we are agreed that it would be inaccurate to give the impression that the research had established a link between place and happiness the question before me is whether the alteration offsets that impression.  For the following reasons, I don’t believe it does.

The added paragraph follows one which says “But the team from the Universities of Sheffield and Manchester stress that happiness is more a product of personal circumstances than physical location“.   In that immediate context, I don’t see that it would be clear to readers of the item that the absence of statistical significance meant that the research had yielded no grounds for viewing happiness as in any degree “a product of  … physical location“.  The six paragraphs which follow, which refer to “a map of happiness” drawn up by the research and list a number of the places supposedly found to be more or less happy, continue to foster an impression which the research doesn’t in fact justify, and the remainder of the item, though rightly focussing on the importance of socio-demographic considerations in relation to the research, still leaves it to be inferred that physical location is to some extent a determinant of happiness, even if not a major one.

While I agree with Mr Warry’s view that this was essentially a light-hearted story, I nevertheless think that, even in its amended form, it conveys a significantly inaccurate impression, and I am therefore upholding your complaint.  In due course a summary of my finding, together with a note of the action taken as a result of it, will be posted on the complaints page of bbc.co.uk, and I shall ensure that you are sent a copy.  Meanwhile, I hope that you will accept my apologies on behalf of the BBC and my thanks for giving us the opportunity of looking into your concerns.  I hope that it will give you some reassurance if I add that the Editor of the website in question has decided that further changes to the item should be made in the light of your points.

(For reference, here is the original story, which Mr Warry has apparantly lost: Britain’s happiest places mapped; along with the initial revision: Britain’s happiest places mapped (revision 1))

First I would like to thank the BBC’s Head of Editorial Complaints for his very fair evaluation of the story.  And, of course, none of this means that happiness is not at all related to where you live: just that there is currently no research evidence to support this hypothesis – so the BBC shouldn’t go around pretending that there is.

Neither is the BBC solely to blame here – the RGS press release, “Geographers map happiest places in Britain“, effectively invited journalists to fall into this trap: the caveats buried in the notes made a nonsense of the ‘headline’.  They surely knew better and I hope that they were driven by embarrassment to remove it from their website.

However, I don’t think that it is unreasonable to expect that BBC journalists critically engage with their sources, rather than just regurgitating any old PR twaddle.  If they are not going to do this, what are they for?

Finally, I think that it’s a shame that the real story of the research – that it, “highlights the issue of social justice and cohesion” as the author (allegedly) put it, was obscured by the trivial ‘light news’ agenda increasingly pursued by the BBC’s journalists.

[BPSDB]

Advertisement