A Peer-reviewer’s Dilemma
Recently, I was asked to review a Randomized Controlled Trial concerning an antidepressant. Among my comments, I requested the authors provide individual data allowing for a re-analysis. Kindly, the authors answered me that my comment was weird and they’ve never received such a request. I did not insist, mainly because there were other serious issues to address in the manuscript. Among those issues, the study was reported as positive, based on a posteriori subgroup analysis, while it was basically a “negative” trial. This issue was fixed and the accepted manuscript was more accurate in reporting the results. It is now explicitly stated that the treatment was not found to be superior to placebo.
Nevertheless, concerning my proposal of data sharing, it was a flop. The data is still inaccessible somewhere on the hard drive of a pharmaceutical company.
I was thus more than enthusiastic when I heard about the Peer Reviewers’ Openness (PRO) Initiative. The signees of the initiative agree to, “not offer comprehensive review for, nor recommend the publication of, any manuscript that does not meet [some] minimum requirements” as of January 1, 2017.
It is surely a strength to be involved in something larger, as a collective initiative that can add credibility and seriousness when as a reviewer one requests data from authors. It may prevent such reactions as I received above. The very constructive nature of the initiative, proposing practical solutions and indeed guidelines to reviewers and scientists is undoubtedly a step in the right direction within the context of other efforts to promote transparency.
Nonetheless, I have yet to sign the pledge because I’m still working through a few possible counterproductive effects of the initiative through different unanswered questions:
- Should we refuse to review the report of aggregated data (an in fact to support an accurate publication of these data) where authors refuse to share their individual level data?
- What will happen if reviewers who think that data sharing matter refuse to perform these comprehensive reviews?
- As reviewers should we really prioritize data sharing above other issues inherent in study design and reporting?
Currently, as part of my technical appraisal, I still continue to emphasize data sharing as a major issue to the editors who ask me to review a manuscript and to the authors who are submitting it. But already we are seeing journal editors adopt data sharing practices as proposed by the International Committee of Medical Journal Editors (ICMJE). This dilemma will hopefully become a moot point…. at least in the field of clinical research.