Let's Talk: Can Comments Advance Science?
A little over a week ago The National Center for Biotechnology Information launched PubMed Commons, a platform that allows registered users to comment on published abstracts available through PubMed, a database of more than 23 million citations for biomedical research. Yet only a month earlier, Popular Science decided to shut off reader comments entirely, citing psychological research to back up the claim that "comments can be bad for science."
These two different choices are both reactions to the very same challenge: figuring out how to capitalize on the promise of social media and the digital dissemination of scientific information while minimizing the perils.
On the one hand, comments are a great way to engage with one's readership, to stimulate community dialogue, and — in the case of post-publication peer review — to further vet scientific claims and ensure the integrity of the scientific literature. On the other hand, comments can introduce false information, polarize readers and foster general incivility.
How this balance is negotiated depends on several factors, including whether and how comments are regulated. For instance, comments within the scientific community are rarely anonymous. Even when scientists participate in "blind review," the names of reviewers are known to some of those involved in the review process. In contrast, many blogs and online news forums permit anonymous or pseudonymous posts.
Does anonymity matter when it comes to the quality or tone of digital comments? A paper published earlier this year analyzed a subset of comments from newspapers that did and did not allow anonymous posts and found that anonymity seemed to foster greater incivility. Indeed, when The Huffington Post banned anonymous commenter accounts this August, Managing Editor Jimmy Soni explained the decision in an interview by drawing a similar connection:
"The idea, Soni explained, is that people are willing to say lots of nasty things from behind a veil of anonymity that they wouldn't be willing to put their name to. 'We feel like when people put their name on a comment it does invest it with a different meaning,' he said. 'There will be that gut-check moment before they comment.' "
Another difference between science and science journalism concerns the identity of the commenters themselves. Within science, comments are typically restricted to the peers of "peer review." These are other scientists or scholars who have the relevant expertise to evaluate the publication in question. However, determining who has this expertise isn't always easy.
For most scientific journals, decisions about who performs peer reviews are made on a paper-by-paper basis by other experts in the field. This customized approach is hard to scale up and helps explain the strategy adopted by PubMed Commons: While anonymous comments won't be permitted, any author of a publication indexed by PubMed will be allowed to comment on any abstract. Note, though, that this will grant some cardiologists permission to comment on articles about squid evolution, while squid experts who aren't in the system will be excluded, as will any interested, nonexpert readers.
Some public forums outside of science have developed their own strategies for regulating unrestricted comments. For example, on many websites, comments are themselves subjected to a minimal form of peer-review: Readers can vote comments up or down. These and other mechanisms — as writer Maria Konnikova points out in a recent post at the NewYorker.com — can result in self-regulating communities that appropriately discount anonymous posts, identify trolls for what they are and otherwise establish local norms.
But without safeguards of some sort, it's easy for scientific misinformation to spread, and once released, it's notoriously sticky. And even if a positive and constructive commenting community can be self-reinforcing, it can be hard to establish and to successfully shield from those operating under different norms.
So we're left with a puzzle, but not quite the one with which we began. There are lots of reasons why scientific publishing and science journalism may be diverging, including their unique aims (e.g., improving scientific practice versus promoting public understanding), and different policies regarding anonymous and unrestricted posts. But the puzzle of how to engineer large-scale, open scientific communication in a way that maximizes benefits while minimizing costs is a common one, and it's one that may not have an easy solution.
With all of this in mind, I invite the 13.7 community — both regulars and new readers — to weigh in on the role of public comments in science communication. That's right: I want your comments on commenting.
Is there a role for anonymous and unrestricted commenting on scientific content? What do you see as the dangers, and how might they be averted? Here at 13.7, conversations are often — but not always — quite sophisticated and civil. What accounts for these successes and what explains occasional lapses?
I look forward to reading your contributions! I might even comment on them myself.
You can keep up with more of what Tania Lombrozo is thinking on Twitter: @TaniaLombrozo