Reproducibility & Repeatability in Tropical Biology: a call to repeat classic studies
One of the hallmarks of science is the concept of Reproducibility – the idea that by following the steps laid out in the Methods section of a paper, an independent researcher can replicate the results of a prior scientific study. Replicated results reinforces their validity, and hence adds support to the conclusions inferred from them.
Recent years have seen an upsurge in attempts to replicate the results of studies in fields from chemistry to medicine to psychology, which can lead to innovative conclusions when the results of attempts to replicate differ from those of the original study. However, this issue – Reproducibility – doesn’t seem to have permeated field-oriented disciplines such as ecology, perhaps because of the inherent difficulty (or even impossibility) of replicating the conditions under which an experiment was conducted (e.g., rain, temperature, abundance of species present during the experiment). There has been some discussion of a related concept – Repeatability – and the benefits to be gained from repeating studies, without necessarily expecting to replicate the original results. For instance, in a 2006 Bioscience paper, Cassey and Blackburn argue
In a scientific discipline such as ecology, the search for general rules and laws is greatly hampered by a high degree of historically based, context-specific contingency. Progress toward such principles will thus be best served by the ability to repeat potentially important discoveries across different ecological systems.
I fully agree, but also think there is little incentive for the replication of previously published work – would you allow one of your graduate students to count as a the is chapter a study that was addressing the same questions with the same design as a previously published one, even in a different system and location? If you were a graduate student would you propose to do so? Probably not. And yet doing so might provide some really novel insights or provide support for ideas central to the field. Experiments simultaneously replicated by scientists in different parts of the world, such as those investigating the effects of nutrient addition or how soil fauna influence decomposition, clearly demonstrate there is value in repeatability. But reproducibility – that seems far lower down on the list of priorities despite its fundamental importance to science.
I think Tropical Biology could learn from psychology, in which there is an active effort to reproduce the results of previous studies. This effort is being supported by the journal Perspectives in Psychological Science, which publishes Registered Replication Reports: “multi-lab, high-quality replications of important experiments in psychological science along with comments by the authors of the original studies”. They even provide funding to conduct these large-scale replications! (A quick shout-out to my UF colleagues Richard Klein and Kate Ratliff, which are helping to lead this project). I propose the ATBC community should consider doing the same.
The first step is identifying classic studies in tropical biology. I can think of a few from my own area of research:
- Janzen, D. H. 1966. Coevolution of mutualism between ants and acacias in Central America. Evolution: 20(3) 249-275.
- Marquis, R. J. 1984. Leaf herbivores decrease fitness of a tropical plant. Science 226:537–539.
- Schemske, D. W. and C. C. Horvitz. 1984. Variation among floral visitors in pollination ability: A precondition for mutualism specialization. Science 225:519-521.
-
Gentry, A. H., and L. H. Emmons. 1987. Geographical variation in fertility, phenology and composition of the understory of neotropical forests. Biotropica 19: 216–217.
-
Denslow, J. S. 1980. Gap Partitioning among Tropical Rainforest Trees. Biotropica 12(2): 47-55.
What would you add to the list of classical studies in tropical biology that merit repeating?
The second step is to more complicated – getting a group of scientists in different places to replicate a classic study. If you are interested in pursuing this possibility let me know. As Editor I would be very excited to see Biotropica publish something like PPS’ ‘Replication Reports’.
EB
PS – This post was inspired by the great NPR story by Shankar Vedantam on reproducibility in psychology and a conversation with our Executive Director Dr. Robin Chazdon, who heard the same story and had the same brainstorm. She suggests those interested could get together at ATBC 2014 to discuss further…more to follow.
you go, Emilio!!!!
Great minds think alike, eh Robin? Let’s follow up in Cairns.
[…] The Association for Tropical Biology and Conservation is advocating for the reproduction of classic…. Right out of the gate, the Executive Director of ATBC (Robin Chazdon) and the Editor-in-Chief of Biotropica (Emilio Bruna) are cooking up a plan. I think this rocks. […]
[…] Via Small Pond Science, news that the Association of Tropical Biology wants people to try to reproduce classic tropical ecology experi…. Apparently their EiC, Emilio Bruna, was inspired by the reproducibility efforts in social […]
Great article.
‘there is little incentive for the replication of previously published work – would you allow one of your graduate students to count as a the is chapter a study that was addressing the same questions with the same design as a previously published one, even in a different system and location? If you were a graduate student would you propose to do so? Probably not.’
I proposed, I was told no. It always seems strange to me that we strive for repeatability in ecology but rarely actual repeat the study.
thanks, and thanks for commenting. It is strange, no, especially since the potentially different results can be so informative. A good example is:
A.H. Powell; G.V.N. Powell. 1987. Population dynamics of male euglossine bees in Amazonian forest fragments. Biotropica Vol: 19:176-179. DOI: 10.2307/2388742
Which was followed up on by:
Peter Becker, Jesus S. Moure and Francisco J. A. Peralta. 1991. More About Euglossine Bees in Amazonian Forest Fragments
Biotropica Vol. 23, No. 4, Part B. pp. 586-591
Interestingly, the first showed the effect everyone expected – fragmentation has detrimental effects, while the second showed bee abundance was higher in 10- and 100-ha fragments than in continuous forest. The first has been cited 297 times; the second 175 times (google scholar). Repetition taught us something, but why isn’t it cited as much>
Good example. I’m sure there are similar cases in the marine setting but none spring to mind immediately, ill have to take a look around. Your post has definitely got me more interested in the topic.
Perhaps the second isn’t cited as much as its less likely to support other peoples studies on different species and fragmentation ;)…that shouldn’t be a reason not to cite in theory but over a large enough sample it could show some researchers are consciously or subconsciously making the distinction?
I have no doubt the second isn’t cited because it is a negative result, and everyone wants to cite papers showing the effects of fragmentation are uniformly detrimental.