Reproducibility & Repeatability in Tropical Biology: a call to repeat classic studies

One of the hallmarks of science is the concept of Reproducibility – the idea that by following the steps laid out in the Methods section of a paper, an independent researcher can replicate the results of a prior scientific study. Replicated results reinforces their validity, and hence adds support to the conclusions inferred from them.

Recent years have seen an upsurge in attempts to replicate the results of studies in fields from chemistry to medicine to psychology, which can lead to innovative conclusions when the results of attempts to replicate differ from those of the original study. However, this issue – Reproducibility – doesn’t seem to have permeated field-oriented disciplines such as ecology, perhaps because of the inherent difficulty (or even impossibility) of replicating the conditions under which an experiment was conducted (e.g., rain, temperature, abundance of species present during the experiment).  There has been some discussion of a related concept – Repeatability – and the benefits to be gained from repeating studies, without necessarily expecting to replicate the original results. For instance, in a 2006 Bioscience paper, Cassey and Blackburn argue

In a scientific discipline such as ecology, the search for general rules and laws is greatly hampered by a high degree of historically based, context-specific contingency. Progress toward such principles will thus be best served by the ability to repeat potentially important discoveries across different ecological systems.

I fully agree, but also think there is little incentive for the replication of previously published work – would you allow one of your graduate students to count as a the is chapter a study that was addressing the same questions with the same design as a previously published one, even in a different system and location?  If you were a graduate student would you propose to do so? Probably not.  And yet doing so might provide some really novel insights or provide support for ideas central to the field. Experiments simultaneously replicated by scientists in different parts of the world, such as those investigating the effects of nutrient addition or how soil fauna influence decomposition, clearly demonstrate there is value in repeatability.  But reproducibility – that seems far lower down on the list of priorities despite its fundamental importance to science.

I think Tropical Biology could learn from psychology, in which there is an active effort to reproduce the results of previous studies. This effort is being supported by the journal Perspectives in Psychological Science, which publishes Registered Replication Reports: “multi-lab, high-quality replications of important experiments in psychological science along with comments by the authors of the original studies”. They even provide funding to conduct these large-scale replications!  (A quick shout-out to my UF colleagues Richard Klein and Kate Ratliff, which are helping to lead this project). I propose the ATBC community should consider doing the same.

 

The first step is identifying classic studies in tropical biology.  I can think of a few from my own area of research:

What would you add to the list of classical studies in tropical biology that merit repeating?

The second step is to more complicated – getting a group of scientists in different places to replicate a classic study.  If you are interested in pursuing this possibility let me know. As Editor I would be very excited to see Biotropica publish something like PPS’ ‘Replication Reports’.

EB

PS – This post was inspired by the great NPR story by Shankar Vedantam on reproducibility in psychology and a conversation with our Executive Director Dr. Robin Chazdon, who heard the same story and had the same brainstorm. She suggests those interested could get together at  ATBC 2014 to discuss further…more to follow.

Tags: , , , ,