Terence Kealey: Science Notebook
The man, the films, those blondes. Free DVD collection starting this Sunday
Which is more likely to be reliable, a news story in The Times or a peer-reviewed paper in The Lancet? As the recent volte-face over the dangers of vitamin supplements shows, the newspaper story may be the more credible.
Peer-reviewed science is forever doing these volte-face (can you remember if coffee is good or bad for you this week?) but The Times is not suddenly going to announce that, er, sorry, it has discovered that China is in fact a parliamentary democracy.
Peer review is the modern sacred cow. “Has it been peer-reviewed?” people ask, breathlessly, when presented with a finding. But peer review is not definitive.
In a recent letter to the journal Science, Robert Zucker, of the University of California, explained the rules of peer review. Basically, they are: “Give the guy a break.” Professor Zucker wrote: “Do not reject a paper with a brilliant new idea simply because the evidence is not as comprehensive as could be imagined.”
You might suppose no scientist would publish a brilliant new idea (“vitamin supplements kill you”) without comprehensively validating it, yet the vitamin paper did not report what the patients died of.
Actually, no one knows. This is very uncomprehensive evidence indeed, because when we do eventually discover the causes of death, we may learn they owe nothing to vitamins, and science will do yet another volte-face.
Professor Zucker also wrote: “Authors need not exclude every possible explanation for their results.” The vitamin paper was not an original study, but simply an overview of 67 previously published reports in which someone died while taking supplements. But the study ignored 405 other reports in which nobody died, so one possible explanation for the results is that the authors omitted inconvenient data. The authors dismissed the omission with hypothetical statistics, yet the paper passed peer review.
Moreover, the patients were not taking natural vitamins but artificial ones, and the reports all differed in terms of dose, combination of vitamins and duration (weeks or years). The study raises so many questions that we still do not know what supplements might actually do. Yet all this is compatible with peer review.
But we can expect no more from science. Nature is too complicated for any paper to be exhaustive, so scientists have settled on a workaday, not comprehensive, standard of peer review. In defence of that workaday standard, Professor Zucker advised journals not “to reflexively demand that more be done” by authors. The public may treat a Lancet paper as ex cathedra but scientists view it simply as work in progress, which is why The Lancet is less definitive than The Times.
As for vitamin supplements, most doctors instinctively dismissed them as worthless for healthy people decades ago.
Terence Kealey is Vice-Chancellor of the University of Buckingham
Read the training tips and advice that helped our London Triathletes
Times Online's new TV show helps you make the right decisions for your pet
Read our exclusive 100 Years of Fleming and Bond interactive timeline, packed with original Times articles and reviews
The latest travel news plus the best hotels and gadgets for business travellers
Shortcuts to help you find sections and articles


Why good girls pay good money for bad-girl baubles

Search The Times Births, Marriages & Deaths
2007
£47,700
2007
£41,899
2008
£41,445
Great car insurance deals online
£25,510 – 32,000
Transport for London
London
£50k
NHS
Nationwide
£
£90,000 + PRP
Essex County Council
Essex
100K
Confidential
London
5% below developer pre-launch price!
Luxury Appts, beautiful gardens w/ Thames views
Great Investment, River Views
By Funway – Thailand
from £589pp
Christmas Cruises
From only £995pp
APTs East Coast now from only
£2425pp.
Great travel insurance deals online
Contact our advertising team for advertising and sponsorship in Times Online, The Times and The Sunday Times. Globrix Property Search - find property for sale and rent in the UK. Visit our classified services and find jobs, used cars, property or holidays. Use our dating service, read our births, marriages and deaths announcements, or place your advertisement.
Copyright 2008 Times Newspapers Ltd.
This service is provided on Times Newspapers' standard Terms and Conditions. Please read our Privacy Policy.To inquire about a licence to reproduce material from Times Online, The Times or The Sunday Times, click here.This website is published by a member of the News International Group. News International Limited, 1 Virginia St, London E98 1XY, is the holding company for the News International group and is registered in England No 81701. VAT number GB 243 8054 69.
I wish people would make the effort to learn how a systematic review/ meta analysis actually works, before embarrassing themselves by spouting in public.
Graeme, Manchester, UK
This latest slur against vitamins is just another part of the attempt by drug companies to remove competition. Vitamins prevent illhealth and these companies don't want healthy people, they want chronically sick customers dependant on their drugs. They need to be stopped, or we'll all be the victims
Mr Davies, Stockport, Cheshire
"No profits in vitamins."
Kevin, Lincoln
Good Lord. Haven't we been over this?
1) Huge profit in vitamins. Multi-billion dollar industry largely owned by "big pharma".
2) Cochrane Library refuses all corporate funding, so by far your best chance at unbiased science.
Think things through, Kevin.
Tom Chivers, London,
What volte face? Real nutrionists and dieticians (as opposed to pill salesman and "nutritional therapists") have known for ages that very few people are deficient in vitamins, and that some can be harmful in high doses, Naturally the pill industry does not like the excellent Cochrane review.
David Colqhoun, London, UK
"But the study ignored 405 other reports..." and so it should have done. Not sure why? Go read the original at http://mrw.interscience.wiley.com/cochrane/clsysrev/articles/CD007176/pdf_fs.html
"...yet the paper passed peer review." I feel sure this article would not have done
Dr Dave, Exeter,
No profits in vitamins, and if there's even a small chance they keep people healthy and away from big pharms expensive drugs then this kind of manipualtion of the scientific process is inevitable. Each vitamin must be examined in isolation to determine (contradictive) effect. This study is flawed.
kevin, Lincoln, UK
I'm sure if one is looking for absolutism, then science isn't the way to go. Science is all about probabilities and the preponderance of evidence. In science new evidence changes old analyses and conclusions.
However, if absolutism is what one wants then the way to go is religion and journalism.
pv, Treviso, Italy
Another problem with peer review is that orthodoxy can make it difficult for genuine new ideas to surface and flourish.
After all, most peer reviewers are senior members of the scientific community, and many will have built their careers on the prevailing scientific orthodoxy.
Chris Palmer, Southampton, England
So let me see if i understand, Scientists put forward a paper attempting to move an idea forward. This is reviewed by peers who compete for grants for research into the same field?WOW this could be really messy, maybe what you need is something really scary like a big asteroid or global warmi...,OH
steve, Sunshine Coast, Australia
many scientific papers are not perfect.
wilson, guangdong, china
In essence, Scientists are people with their foibles just like the rest of us.I trust science but not those pitching for news attention or funding.
Finding the right notes to play the pipers tune is a fact of both science and journalism.Scepticism is better for you than vitamins ever were.
robert everitt, wolverhampton,
This article complains that researchers looked at "artificial", not "natural" vitamins. Since the study is assessing the safety of vitamin supplements, this makes perfect sense. If you want to find out if vitamin pills are safe, looking at people taking those pills seems a good place to start.
P Wilson, London,
A few points for those who do not understand meta analysis:
Rejected studies were rejected for scientifically sound reasons such as inadequate methodological or data quality, and especially for the fact that as this was a study about deaths, only those studies which had deaths could be looked at!
Adam Gibb, Manchester, England
I think that a peer-reviewed article would make it clear what evidence the authors looked at, what evidence they didn't, and precisely how they reached their conclusions. One can then judge how solid and how applicable those conclusions are.
Newspaper articles almost never do this, nor should they
Hal, New York, United States
If all this article is stating is that there are times when science is not infallible, and it so happens that an article with no scientific basis guesses the correct answer, then, statistically, this is of course an inevitability.
Dominic Pearson, Dubai, UAE
The really sad thing about Fleet Street these days is that the speculative article n today's Times (presented without a single conditional verb, of course) about Madeleine McCann's movements on the night of her disappearance probably would pass peer review amongst ordinary journalists...
Ian Kemmish, Biggleswade, UK
Surely this article notes merely that findings in peer-reviewed science journals may need to be subjected to further research. This article makes no reference to the credibility of newspapers contrary to the comparative opening paragraph. That is therefore not a credible assertion without evidence.
John Scott, London,
This is all very well, but I think more research is needed.
Christopher Chantrill, Seattle, USA
Hm. I'm sure many scientific papers are flawed. But the same can be said of some of the selective reports that appear in newspapers. The Press (and broadcast media) are not renowned for letting facts and balance stand in the way of a good story.
James, Hong Kong, China