Interesting Replicable “Badge” for journal articles

I am writing to point out an interesting experiment that’s started up in the ACM Transactions on Mathematical Software (TOMS) computational journal.  There’s an editorial by Mike Heroux that explains the process.

In brief, TOMS has created a “Replicated Computational Results (RCR)” review process, which is really a designation saying that the computational results published in an article are replicable.  As you might expect, this adds an extra burden on the authors to submit their work in a way that enables this RCR review, an extra burden on the editor to work with this part of the submission, and a completely new burden on the RCR reviewer.  If this process is successfully completed, the paper gets an RCR designation on its cover page, the journal editors hopefully get higher quality work, and the reviewer gets satisfaction in a job well done and a credited contribution to the field.

This seems a really interesting experiment; I hope it succeeds.

p.s.  I’m also personally happy that the first paper that received this designation is an output of an NSF-funded SI2 project: Field G. Van Zee and Robert A. van de Geijn. 2015. BLIS: A framework for rapidly instantiating BLAS functionality. ACM Trans. Math. Softw. 41, 3 (May 2015), with James Willenbring writing the first RCR review.


Since the Reproducible Scientific Computing blog created and curated by Doug Thain has become inactive, this is a repost from there.


Some work by the author was supported by the National Science Foundation (NSF) while working at the Foundation; any opinion, finding, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the NSF.


Published by:


Assistant Director for Scientific Software and Applications at NCSA, Research Associate Professor in CS, ECE, and the iSchool at the University of Illinois Urbana-Champaign; works on systems and tools (aka cyberinfrastructure) and policy related to computational and data-enabled research, primarily in science and engineering

Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s