Aric Sigman and a problem with newspaper comment

Welcome to my first post of 2010, in which I identify and illustrate a problem with newspaper comment on matters of science.

Dr Aric Sigman is a popular author and presenter with a Phd on “The roles of attention in hypnotic and feedback control of heart rate” and an interest in blaming certain of societies ills on new technology and new media.  He has been criticised previously for his habit of cherry picking and extrapolating wildly in various publications exploring this concept.  In his latest comment piece on The Guardian’s Comment is Free site Dr Sigman constructs his thesis in fewer than 500 words arguing that:

What children see through product placement increases their sense of entitlement to what they see, which in turn increases their “pester power”, which in turn makes their parents’ job of mediating their children’s demands more difficult. Ultimately, this retards the development of our children’s capacity for deferred gratification, now referred to as “impulse control”.

Cited for support in this series of assertions are newspaper articles, from one and two days previous, reporting on criticism from a variety of bodies, including those representing healthcare, of a recent government consultation exploring a changing of the rules regarding product placement on television.

Following this opening argument Dr Sigman explores his interpretation of the implications of delayed development of “impulse control”.

However, impulse control is more than a behavioural pleasantry; it is a self-regulation skill that affects a wide range of childhood outcomes that go far beyond demanding a chocolate bar here and now. For example, the development of a child’s impulse control is uniquely related to early academic success and is now considered more important in early academic progress than measures of intelligence. And a recent study published in the British Journal of Psychiatry reported that acquiescing in the face of pester power is linked to later antisocial aggressive behaviour and convictions for criminal violence. The researcher concluded that giving in to a child’s demands may prevent them from learning to wait for something they want: “Not being able to defer gratification may push them into more impulsive behaviour, which is strongly associated with delinquency.”

This passage is notable for containing two references, the last in the article, that are in respectable academic journals and allow us to use them as a proxy to measure the breadth and depth of Dr Sigman’s scholarship.  Do they say what he says they do?

The first reference is in the journal Child Development* and is cited as showing “the development of a child’s impulse control is uniquely related to early academic success and is now considered more important in early academic progress than measures of intelligence”.  This paper studied “one hundred and forty-one 3- to 5-year-old children from low-income homes”, a small and narrowly defined cohort that should prevent extrapolation to general conclusions encompassing all populations of children, a point not lost on the authors as they conclude “An important direction for future work will be the replication with diverse populations of effects seen in these studies and assessment of the generalizability of training effects to academic ability.”.  Furthermore neither of the terms ‘impulse control’ or ‘deferred gratification’, used by Dr Sigman, are defined or mentioned by the authors, they use the terms “effortful control, false belief understanding, and the inhibitory control and attention-shifting aspects of executive function” in their investigations.  This work does not support the statement by Dr Sigman.

The second reference is in the British Journal of Psychiatry** and is cited as showing that “acquiescing in the face of pester power is linked to later antisocial aggressive behaviour and convictions for criminal violence”.  This paper reports on the link between childhood consumption of chocolate and violent behaviour in adulthood, there was “a significant relationship between eating confectionery at age 10 years and violence at age 34 years”.  The authors speculate in the discussion that:

Candidate mechanisms linking confectionery consumption to adult violence must account for enduring changes into adulthood. One plausible mechanism is that persistently using confectionery to control childhood behaviour might prevent children from learning to defer gratification, in turn biasing decision processes towards more impulsive behaviour, biases that are strongly associated with delinquency. Furthermore, childhood confectionery consumption may nurture a taste that is maintained into adult- hood, exposing adults to the effects of additives often found in sweetened food, the consumption of which may also contribute towards adult aggression

The speculative nature of these mechanisms is clear from the context and they are not considered as proven by the authors, instead warranting “further attention”.  This work does not support the statement by Dr Sigman.

However, unlike the first reference, a possible source of Dr Sigman’s misunderstanding can be easily found.  The press release from the Royal College of Psychiatrists contains a quote from the studies lead author, Dr Simon Moore:

“Our favoured explanation is that giving children sweets and chocolate regularly may stop them learning how to wait to obtain something they want. Not being able to defer gratification may push them towards more impulsive behaviour, which is strongly associated with delinquency.”

However this bold statement is qualified in the succeeding paragraph:

The researchers concluded: “This association between confectionary consumption and violence needs further attention. Targeting resources at improving children’s diet may improve health and reduce aggression.”

Like the paper itself, the press release makes clear from context the speculative nature of Dr Moore’s comment.  However, this was not sufficient to prevent articles, such as this in the Daily Mail, that take Dr Moore’s speculation as an assertion of fact.  This seems to have perturbed Dr Moore, and in an online response to criticism of his paper on the British Journal of Psychiatry’s website he writes (my bold):

The far ranging media interest stimulated by this paper ranged from deriding the work as a waste of money to one that could bring peace and harmony to a North American city, and possibly further afield. [...] as good Popperians behavioural scientists test null hypotheses: science progresses through the falsification of beliefs, and accordingly we reported the acceptability of the null probabilistically in the regression statistics. But the point remains that at least in the media such results are too often reported as facts when they are statements of belief looking to be challenged. Stating “what works”, inferring causality, and demonstrating “proof” are further examples of naivety that contributes to the disparity between academic and lay reporting.

Like the media, it is entirely possible that Dr Sigman has based his understanding of the paper on an incomplete reading of the press release alone.

In my opinion this basic failure of scholarship renders the arguments and conclusions in Dr Sigman’s article void of academic merit or extension by discussion.  So why was it published?

Matt Seaton, the editor of Comment is Free, in response to my enquiry as to if Dr Sigman’s article was checked for factual errors and if his references were read, has provided some insight into how these errors were missed by The Guardian.

1. Do we check references? Yes, and obviously we would query or correct a reference that appeared to be mistaken. But unless there is a glaring error, we do take a certain amount on trust from authors (unless special circumstances would lead us to do otherwise, which is not the case here).

2. Do we read in full the text linked to in a web reference? No, that would naturally be an impossible burden. In this case, Aric Sigman warned me that he could only provide links to abstracts, as the learned journals in question are subscription-only; to which my response was that this was still useful, in order to ‘show his working’ and avoid the impression of making unwarranted assertions.

It is unreasonable to expect newspaper employees to have the time and expertise to examine references such as those used by Dr Sigman and I fully accept the need to trust an author.  That Mr Seaton expected Dr Sigman to “‘show his working’ and avoid the impression of making unwarranted assertions” is a reasonable burden to place on the author of a comment piece, Dr Sigman failing to do this adequately is not The Guardian’s fault.  But the decision to publish was not Dr Sigman’s.

Some insight into the decision to publish may be gleaned from the comments in response to the article where it seems The Guardian’s online readers are more interested in having their prejudices confirmed or confronted rather than challenging the factual basis for Dr Sigman’s opinions.  This is clearly not a forum for disseminating knowledge but a platform where the uninformed can combat assertion with prejudice.  It is perhaps in The Guardian’s interests to attract the advertising revenues associated with a high volume of readers and they are simply given the readers what they want.  Not fact, but opinion.

Unfortunately it is environments such as these that provide resources for that peculiar species of commentator that prefers to advance their arguments by popular opinion than by peer review. It is worth noting that in the 1980s Dr Sigman produced a handful of papers on hypnotism and since then does not appear to have published anything that appears in an academic database.  Instead he has penned popular books and recorded less popular audio cassettes as well as performing research for commercial organisations on subjects outwith his, now dated, academic speciality.  It appears at no point has he sought professional appraisal of his opinions by submitting them to the rigours of peer review, therefore we cannot know what merit they may have when considered against evidence in the field.

There is a famous speech by Richard Feynman in which in rails against what he calls ‘cargo cult science’ and argues that proper science requires high standards of honesty, absent elsewhere, to avoid the human tendency to be fooled.

The first principle is that you must not fool yourself–and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that.

Dr Sigman, in his inadequate use of references and tendency to assert without proof, has perhaps demonstrated that he has fooled himself, and others, and is little more than a cargo cult academic.

If only this criticism was a cautionary tale of the dangers of misrepresenting scientific evidence in the popular press.  However, contrary to the conventions of the form, Dr Sigman has met no ill fate, only reward.  His books sell, his speaking services are advertised at £3-5k a time and he indulges his passion for travel.  These are not the perks of the average academic bound by research codes, ethical approval, peer review and the need to constrain opinion within the boundaries of available evidence, these are the perks of cargo cult academics.  Slipshod research sells.

The solution to this is perhaps more criticism and challenge from those in academia targeted at those who prefer their theories evade review while assuming the cloak of academic respectability through the shallow use of references and titles.

In the coming year I would like to see academics becoming more proactive in their challenge of misrepresentation of their research by those in the media and I would like to see the likes of Dr Sigman under closer scrutiny regarding the evidence base for their claims.  Here’s to a new year.

Dr Sigman was contacted for the purpose of this blogpost and did not respond.  I hope he will do so post publication and I will update accordingly.  I am grateful to Dr Simon Moore and Matt Seaton for their help and observations.

* Relating Effortful Control, Executive Function, and False Belief Understanding to Emerging Math and Literacy Ability in Kindergarten
Clancy Blair & Rachel Peters Razza
Child Development (78) 2: 647-663

Confectionery consumption in childhood and adult violence
Moore, Simon C., Carter, Lisa M., van Goozen, Stephanie
The British Journal of Psychiatry 2009 195: 366-367