#arseniclife, peer review, and the scientific process

I set out this morning to write down some thoughts about the arsenic-containing bacteria study (aka #arseniclife). Three hours later, I realized I was completely in over my head. What follows are lots of links and some musings of my own.

I wasn’t even attempting to delve into the science. (Being a yeast cell biologist by training, this is a bit outside my area of expertise.) Rather, I was interested in the controversy surrounding this paper and what we might be able to learn from it about the scientific process.

Turns out there is quite a lot to mull over.

The study was published online in Science in December and stirred up a great deal of controversy, which wasn’t helped by a provocative press release from NASA that included the phrase “evidence for extraterrestrial life”. Science published online today eight Technical Comments critiquing the arsenic bacteria study, as well as a formal response from the authors, Felisa Wolfe-Simon and colleagues.

[Note 06/03/2011: The Technical Comments and authors’ response are no longer available on Science Express, because they are published in the today’s issue of Science, along with the final version of the arsenic bacteria paper. The links I had included here for the paper take you to the version that was published online. Here’s a link to the final full text of the article, which appears to be freely available, at least for now.]

Some great articles published yesterday or today on the paper and its controversy (listed in the order that I read them, feel free to send me more):

Debate over arsenic-based life enters a new chapter
Brian Vastag, Washington Post

Science Publishes Multiple Critiques of Arsenic Bacterium Paper
Elizabeth Pennisi, ScienceInsider

Science Publishes “Arsenic is Life” Critiques. Game On.
David Dobbs, Neuron Culture, Wired Science Blogs

Arsenic, RNA, and the unpleasant aftertaste of hype
Erika Check Hayden, The Last Word On Nothing

The Discovery of Arsenic-Based Twitter: How #arseniclife changed science.
Carl Zimmer, Slate

Critics weigh in on arsenic life
Erika Check Hayden, Nature News

For a more comprehensive list of articles about #arseniclife, refer to Bora’s epic link-dump[Added 06/03/2011]

I also have to embed here xkcd.com’s excellent take on the matter, published back in December:

Regarding the controversy, I started writing this blog post, because I wanted to say this:

This is what is supposed to happen.

Faced with a potentially paradigm-shifting discovery, the science community is reacting exactly the way they should be: by being skeptical, by debating over the methods used, and by demanding further evidence. In other words, they are making sure the claims in the study are well-supported. This is how the community vets (or should vet) every scientific discovery, and the response to the arsenic bacteria study is dramatic, because the discovery itself is dramatic.

If you want to shift a paradigm, the burden of proof is on you.

I don’t envy the authors right now. Academic discourse can be downright harsh and can get very personal. Jonathan Eisen admonished people for this back in December.

But wait…

That’s not why the scientific community is so upset, is it?

Scientists like new exciting ideas, but they don’t like feeling manipulated. NASA’s provocative press release was the first strike. (Erika Check Hayden’s blog post touches on the ramifications of this.) The quick buildup of critique after the paper was published online led many to wonder how this passed peer review – that was the second strike. The authors’ declining to engage in open dialogue (despite all the discussion in the blogosphere and Twitter) and their insistence on issuing their response through “official” channels was the third strike.

Maybe there are more strikes, but that was conveniently three.

I have to admit I’m sort of cheering for Felisa Wolfe-Simon and her colleagues. The science spectator in me is really hoping their findings withstand all this scrutiny, and in the end, we’ll discover that GFAJ-1 is happily surviving on arsenic after all. The scientist in me though is mindful that we need more work (a bit of an understatement maybe!). As noted by Bruce Alberts in his editor’s note, “… we recognize that some issues remain unresolved. However, the discussion published online today is only a step in a much longer process.”

#arseniclife issues I am musing over 

I have to wonder (as I’m sure others are) what happened to this paper in peer review. Admittedly, scientists are always snarkily wondering how such-and-such paper passed peer review (nearly every journal club I attended ended on that note). But given the massive amount of critique since publication, why weren’t any of these concerns caught/addressed during peer review? Or were they? Given that Science has now published eight Technical Comments on the paper, it would be helpful to understand what was or wasn’t scrutinized in the original peer review.

Of course, there is also chatter about how the peer review process is broken, and so is this a demonstration that post-publication peer review is the way to go? (Carl Zimmer touches on this in his Slate article.)

Why would Science let such a high-profile article languish online (nearly 6 months!) before finally moving it into a print issue? (According to the news article in ScienceInsider, the paper will appear in next week’s issue of the journal.) Publication online means the paper is accepted (according to About Science Express). It is not a problem with backlog – a quick glance at the Science Express line up confirms that most of the other online-published papers that haven’t yet appeared in a print issue (they get removed from Science Express once print-published) are from within the last month. The only other paper languishing online is a paper about SNPs correlated with exceptional longevity (which was subject to its own controversy – here and here).

The editor’s note points out that the authors are making their arsenic-eating bacterial strain available to the scientific community. This is good, and hopefully, we will soon see independent results on GFAJ-1 from other groups (perhaps even those who have publicly critiqued the study?). (However, Erika Check Hayden’s article for Nature News points out why scientists might be reluctant to take on follow-up studies.)

3 thoughts on “#arseniclife, peer review, and the scientific process

    1. I am an idealist – still hoping something interesting will come out of this, even if it isn’t arsenic-eating bacteria! Regarding the delay in print publication, I would guess the editors wanted to publish the paper alongside all the Technical Comments – so that the entire “dialogue” is presented in a single issue of the journal?


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s