The Journal of Young Investigators
science career center
SCC Home PageScience Careers
News
Other SCC Links
|
Recent Articles |
||||||
More Career Articles
|
||||||
Brendan Maher
Our jobs are getting tougher. Or that’s what some media watchers would have you believe after a recent bout of scientific fraud incidents have taken prominent spots in science news coverage. The headliner, of course is Hwang Woo Suk, the South Korean researcher that bewitched and bedeviled editors, scientists, and the public by publishing in one of the top research journals, Science, a series of completely fabricated papers in which he claimed to clone human embryos and derive stem cell lines from them. Regardless of the hubbub, the skepticism that every reporter should take to every story shouldn’t change just because of a few high-profile incidents.
Several instances of misconduct and fabrication have dotted the science pages, but few with the sheer audacity of Hwang (although a clinical research paper appearing in The Lancent, in which nearly a third of subjects had the same birthday, might come close). The dust-up that has since occurred had some pundits pointing fingers at media outlets for hyping the research, some media outlets pointing fingers at journals for being so easily snowed, and many simply wondering if the job of science reporting has just gotten a lot harder. A recent story in the business section of the New York Times portrays the science journalism community as one that has just re-realized its feet.
“Now, news organizations say they are starting to look at the science journals a bit more skeptically,” the article says. The Times author even uses a cover story from the publication where I work to paint the top-tier journals as a beleaguered system, “weakened” under the strain of an increasing submission flood.
Brendan Maher is a Professional Reviewer at JYI and a senior editor at The Scientist, an international life science news magazine. Image courtesy Jason Varney.
Alison McCook, who penned our story on peer review, took exception to the Times characterization. Yes, peer review at top journals is under some strain to sift through an increasing number of submissions, she tells me. Their filter in choosing the best publishable works has indeed let slip a few false positives (like Hwang), and certainly a few false negatives (worthy studies that are rejected), but she didn’t find the screening process to be any less strong than it has been in the past. Journals have worked diligently to make sure that editors aren’t overburdened and that every paper is properly scrutinized. But to think that they’re perfect is fallacy.
Science journalists have always and will always need to be vigilant. Rather than simply relying on the veracity of a journal’s peer review system, every study should be carefully scrutinized, both by the reporter alone, and through reliable sources that can speak critically of the work. Reporters shouldn’t forget that no single study is “the final word” on a subject. All experiments must be repeated and replicated for verification–quite possibly the least glamorous, but most important part of science. Hwang et al. were forced to retract their study, but an overwhelming majority of retracted studies have nothing to do with misconduct. Scientists skeptically reviewing the results of others, and even of themselves, often find that results just don’t stand up to the test of replication. And of course such setbacks rarely make the headlines.
That’s not to say that science journalism is doing a bad job of things. Sure there are exceptions, but most new discoveries are colored in the news with qualifiers like “these results suggest,” or “the authors interpret this to mean.” It’s a worthy endeavor to remind readers that discovery means little without verification. More importantly, most reporters work hard to find outside sources not affiliated with the study that can offer some critical assessment of the work: limitations, alternative interpretations, possible controls they would have liked to see, etc. Although the Times article mentions it as a new practice at some organizations, every reporter should be consistently asking about financial ties to the results of published studies. Moreover, they need to be vigilant that criticisms are more than simply a scientist with an axe to grind. Keeping track of politics within the scope of an already complicated scientific story is challenging, but it shouldn’t be something new for science journalists.
The fact remains, of course, that we were all snowed by Hwang. Part of this is because the deception went deep. Hwang published a series of papers (some of them legitimate) that suggested that his group had the technical prowess to pull off what no one else had done. And part of the problem was that so many wanted to believe that he had achieved what he said. I never once spoke to a source that was highly skeptical or critical of the work. Such overwhelming enthusiasm from normally skeptical experts tends to make a believer out of me. I’m sure there will be a twice-shy edge to all future reporting on human cloning results, published or otherwise. But there’s no need to change the level of vigilance, because we should always be this way.
Several instances of misconduct and fabrication have dotted the science pages, but few with the sheer audacity of Hwang (although a clinical research paper appearing in The Lancent, in which nearly a third of subjects had the same birthday, might come close). The dust-up that has since occurred had some pundits pointing fingers at media outlets for hyping the research, some media outlets pointing fingers at journals for being so easily snowed, and many simply wondering if the job of science reporting has just gotten a lot harder. A recent story in the business section of the New York Times portrays the science journalism community as one that has just re-realized its feet.
“Now, news organizations say they are starting to look at the science journals a bit more skeptically,” the article says. The Times author even uses a cover story from the publication where I work to paint the top-tier journals as a beleaguered system, “weakened” under the strain of an increasing submission flood.
Brendan Maher is a Professional Reviewer at JYI and a senior editor at The Scientist, an international life science news magazine. Image courtesy Jason Varney.
Alison McCook, who penned our story on peer review, took exception to the Times characterization. Yes, peer review at top journals is under some strain to sift through an increasing number of submissions, she tells me. Their filter in choosing the best publishable works has indeed let slip a few false positives (like Hwang), and certainly a few false negatives (worthy studies that are rejected), but she didn’t find the screening process to be any less strong than it has been in the past. Journals have worked diligently to make sure that editors aren’t overburdened and that every paper is properly scrutinized. But to think that they’re perfect is fallacy.
Science journalists have always and will always need to be vigilant. Rather than simply relying on the veracity of a journal’s peer review system, every study should be carefully scrutinized, both by the reporter alone, and through reliable sources that can speak critically of the work. Reporters shouldn’t forget that no single study is “the final word” on a subject. All experiments must be repeated and replicated for verification–quite possibly the least glamorous, but most important part of science. Hwang et al. were forced to retract their study, but an overwhelming majority of retracted studies have nothing to do with misconduct. Scientists skeptically reviewing the results of others, and even of themselves, often find that results just don’t stand up to the test of replication. And of course such setbacks rarely make the headlines.
That’s not to say that science journalism is doing a bad job of things. Sure there are exceptions, but most new discoveries are colored in the news with qualifiers like “these results suggest,” or “the authors interpret this to mean.” It’s a worthy endeavor to remind readers that discovery means little without verification. More importantly, most reporters work hard to find outside sources not affiliated with the study that can offer some critical assessment of the work: limitations, alternative interpretations, possible controls they would have liked to see, etc. Although the Times article mentions it as a new practice at some organizations, every reporter should be consistently asking about financial ties to the results of published studies. Moreover, they need to be vigilant that criticisms are more than simply a scientist with an axe to grind. Keeping track of politics within the scope of an already complicated scientific story is challenging, but it shouldn’t be something new for science journalists.
The fact remains, of course, that we were all snowed by Hwang. Part of this is because the deception went deep. Hwang published a series of papers (some of them legitimate) that suggested that his group had the technical prowess to pull off what no one else had done. And part of the problem was that so many wanted to believe that he had achieved what he said. I never once spoke to a source that was highly skeptical or critical of the work. Such overwhelming enthusiasm from normally skeptical experts tends to make a believer out of me. I’m sure there will be a twice-shy edge to all future reporting on human cloning results, published or otherwise. But there’s no need to change the level of vigilance, because we should always be this way.