The Centre for Policy Studies has been rather aggressively trailing a report on bias at the BBC. The report, written by Oliver Latham, has now been released earlier than expected.
The report is written in the spirit of, and partly following the method of, Tim Groseclose’s Left Turn: How Liberal Media Bias Distorts the American Mind. Latham also identifies a left-wing media bias, this time in the BBC.
My main problems are not with the report, but with the way in which the CPS has used it. There are two ways to think about bias: absolute bias, understood as a “deviation from an unattainable but theoretically conceivable condition of unbias”; and relative bias, understood as a deviation from the type of content produced by other outlets.
The report claims that the BBC demonstrates a relative bias towards the type of content produced by the Guardian, because it tends to cite the same think tanks. However, the CPS wants very badly to claim that the BBC is biased in an absolute sense. You can claim that — but you would have to start from the premise that the Telegraph is less biased (in an absolute sense) than the Guardian. If you can directly assess levels of absolute bias in this way, you don’t need to muck around looking at citations to different think tanks.
There is, however, a potential problem with the empirics of the report. The original Groseclose/Milyo work exploited the way different members of Congress cited think tanks. The more left-wing members of Congress cited a think tank, the more left-wing the think tank, and vice versa. These members of Congress act as anchors, or bridge points.
Latham hasn’t done this, but has instead used two media outlets as anchors. If you use MPs’ citations to think tanks as anchor points, you get a very different ordering of think tanks.
Using an old cache of parlparse files, I did a quick pattern search for references in parliamentary debates to each of the forty think tanks cited in the report over the period May 2010 to Dec. 2011 (I would have included 2012, but I haven’t set up version control for my parlparse directory, and it’s a pain downloading the whole thing. Email me if you want the Perl scripts I used).
I then divided the number of (unique) Labour citations by the number of (unique) Labour citations plus the number of (unique) Conservative citations. (A citation is not unique if it occurs in the same debate or on the same day). Extrapolating from Latham’s assumptions, the bigger this number, the more left-wing the think-tank.
Unfortunately — and this is only a quick check — there’s no correlation between this measure and the left-right rank of think tanks according to page 5 of the report. The Spearman rank correlation on the 24 think tanks for which the above summary measure is well-defined is only 0.13, which is not significant (p=0.5). (Data here).
I don’t have the full data, so I can’t test whether the BBC does cite think-tanks which are disproportionately cited by Labour MPs more than it cites think-tanks which are disproportionately cited by Conservative MPs. But that would be the test in the true Groseclose-Milyo spirit.