GraphPad’s advice regarding when to plot SD versus SEM:
If you want to create persuasive propaganda: If your goal is to emphasize small and unimportant differences in your data, show your error bars as SEM, and hope that your readers think they are SD. If our goal is to cover-up large differences, show the error bars as the standard deviations for the groups, and hope that your readers think they are a standard errors.
I’ve been thinking more about Matters (the scientific journal that publishes “single observations”) in the weeks since my last post and am less ambivalent. Reading Julia Belluz’s article in Vox about “small” science only further cemented the conception of Matters as an appropriate response to an overemphasis on storymaking. Lawrence Rajendran, the founder of Matters, was quoted in the article regarding Alexander Fleming’s discovery of penicillin and his publication chances today:
Rajendran notes that Alexander Fleming’s simple observation that penicillin mold seemed to kill off bacteria in his petri dish could never be published today, even though it led to the discovery of lifesaving antibiotics. That’s because today’s journals want lots of data and positive results that fit into an overarching narrative (what Rajendran calls “storytelling”) before they’ll publish a given study. “You would have to solve the structure of penicillin or find the mechanism of action,” he added.
The point stands. Most modern peer reviewers are, however, likely to recognize the importance of a discovery such as penicillin. It might also be worth noting that his initial discovery of the compound in 1929, documented in an article in the now-defunct British Journal of Experimental Pathology, went unnoticed by the New York Times. Furthermore, according to Google Scholar, the article was cited only sparingly in the 10 years following its publication.
A “single observation” with a long lifespan and a story of its own, perhaps.
Another new journal that, like Matters, launched earlier this year: Research Ideas and Outcomes or RIO.
From their front page:
The Research Ideas and Outcomes (RIO) journal publishes all outputs of the research cycle, including: project proposals, data, methods, workflows, software, project reports and research articles together on a single collaborative platform, with the most transparent, open and public peer-review process. Our scope encompasses all areas of academic research, including science, technology, humanities and the social sciences.
Stephen Heard on post-publication peer review:
This seems to me a huge irony about proposals to replace pre-publication with post-publication peer review. At first glance, such proposals seem like the ultimate democratization: everyone’s manuscript on an equal footing. My manuscript and yours, a Nobel prize-winner’s and the rankest amateur’s, all available for readers whose comments will bubble the very best to the top. But this democratization will, I worry, turn out to be self-disrupting. Its very existence (coupled with the enormity of our literature) seems to force us to use prioritization signals that restore the very privilege we thought we were stamping out.
There’s a new open access journal in town: Matters. Their tagline: “Stories can wait. Science can’t.” Science magazine has an article delving into the motivation for its creation. According to that article, “[Matters] aims to create a freely accessible venue for single findings, even confirmatory data and contradictory data.”
I’m ambivalent. A lot of data falls through the cracks or are otherwise not part of a coherent, compelling, or “impactful” story. Matters provides an important means to quickly publish and archive this data.
Yet I cannot keep up with the coherent stories published in my field as it is. How will I be able to keep up with the “single observations” that will, doubtless, flow from Matters?
And the broader question of researcher incentive remains. The simultaneous, and somewhat dissonant, pressure to publish either eye-catching stories in high-impact journals or prolifically in middle-tier journals—the “minimum publishable unit” comes to mind—remain.
Cultivate this virtue
the profound truth that
the manifest and hidden are one.
— Morihei Ueshiba, The Secrets of Budō