Metrics International Forum is a weeky online webinar on meta-research topics. It occurs every Thursday 9:00 am PT time.
It started in 2020 as a subscription based forum, and since February 2022 it is open for the public. We do however ask all participants to:
- maintain professional discourse
- post questions in chat - or raise their "virtual" hand - and wait to ask the question live when propmeted by the moderators
- keep zoom on mute when not speaking
Any offensive, discriminatory or unprofessional behaviour may lead to sanctions, incuding being kicked out of the meeting.
This is a supportive and inclusive community, and we are grateful to all our participants for keeping it this way.
The link to the meeting is:
https://stanford.zoom.us/j/92048898481?pwd=TmUwd05BeEFEN0tvMTFGcGZENmlQZz09
Password (if prompted): 007007
If you woud like to present to one of our following meetings, please fill out this form and we will come back to you:
Upcoming speakers:
May 12 Benjamin Djulbegovic. How do guidelines panels decide?
May 19 Danielle Fanelli
May 26 Noah Haber
June 2 Peter Sjograde
Previous Speakers:
2022:
Emmanuel Zavalis (May 5). A meta-epidemiological assessment of transparency indicators of infectious disease models. - Presentation
Daniel Strech (Apr 28). Ready for translation? How inestigator brochures present supporting evidence and how they could improve. - Presentation
Esther Pearl (Apr 22) The Experimental Design Assistant: a tool to improve in vivo research. - Presentation
Ivan Stelmakh (Apr 14). Two Studies on Peer Review: Citation Bias and arXiv Bias - Presentation
Jackie Thompson and Robbie Clark (Apr 7). Incentivising Registered Reports through funder-journal partnerships: a qualitative feasibility study- Presentation
Alexandra Sarafoglou & Suzanne Hoogeveen (Mar 31). The Many-Analysts Religion Project: Key Findings and Meta-Scientific Insights - Presentation
Carole Lunny (Mar 24). Development of a new risk of bias tool for network meta-analysis. - Presentation.
Tim Errington (Mar 17). Barriers to replicating preclinical cancer biology research - challenges or opportunities? - Presentation
Marcus Munafo (Mar 10). From Grassroots to Global: Building a Reproducibility Network - Presentation
Shilaan Alzahawi (Feb 24). Lay Perceptions of Scientific Findings: Swayed by the Crowd? - Presentation
Tom Hardwicke (Feb 17). How should journals handle scientific criticism? - Presentation
(Before Feb 17 Forum was closed - we are seeking permission to upload presentations):
10-Feb Maximilian Siebert
03-Feb Arjun Manraj
27-Jan Tom Stanley Harnessing Excess Statistical Significance for Meta-Research and Meta-Analysis
20-Jan Leonhard Held
13-Jan Matthew Page
06-Jan Benjamin Djulbegovic. Avoidable and unavoidable research waste in (biomedical) research
2021:
09-Dec Jason Chin. Using metaresearch to improve the legal system
18-Nov Nihar Shah. Improving Peer Review via Principled and Practical Approaches
11-Nov Mario Malički. How do manuscripts change and what does that tell us of peer review?
28-Oct Spencer Harpe Meta-research in pharmacy: What are we doing?
21-Oct Jennifer Miller. Equitable access to the benefits of clinical research
14-Oct Omer Benjakob, Rona Aviram, Jonathan Sobel. Wikipedia & COVID-19: Science on the frontlines of coronavirus disinformation
07-Oct Kristijan Armeni and Loek Brinkman. Open science: meta-research shows the way, open science communities walk the walk
30-Sep Philipp Koellinger. Designing incentives for scientists to improve the replicability of research
16-Sep Tracey Weissberger. Training early career researchers to use meta-research to improve science: A participant-guided, "learn by doing" approach
09-Sep Karima Chaabna. The State of Population Health Research Performance in the Middle East and North Africa: a Meta-Research Study
02-Sep Simine Vazire. Evaluating research on its own merits
26-Aug Leo Tiokhin. Structuring Incentives to Improve the Efficiency and Reliability of Science: Modeling the Effect of Incentives for Priority of Discovery
02-Sep Simine Vazire Evaluating research on its own merits
26-Aug Leo Tiokhin Structuring Incentives to Improve the Efficiency and Reliability of Science: Modeling the Effect of Incentives for Priority of Discovery
19-Aug David Pina. The Marie Curie mobility programme as a case study for meta-research
12-Aug Manoj Lalu. Lifting the lid on randomized control trials published in predatory journals
29-Jul Nate Breznau. Observing Many Researchers Using the Same Data and Hypothesis Reveals a Hidden Universe of Uncertainty
22-Jul Daniel Dunleavy.Progressive and degenerative journals: On the growth and appraisal of knowledge in scholarly publishing
15-Jul Maya Mathur. Sensitivity analysis for p-hacking in meta-analyses
24-Jun Zoltan Dienes. Registered Reports 2.0: Introducing the Peer Community in Registered Reports
17-Jun Antica Culina. Unused potential of ecological research, and the role of reproducibility
10-Jun Holly Witteman. Strategies for funding agencies to address inequities in funding allocation
03-Jun Mohammad Hosseini. An ethical analysis of equal co-authorship practices
27-May Balazs Aczel. Developing tools and practices to promote open and efficient science
20-May Isabelle Boutron. Interventional research on research: development and evaluation of interventions to improve research value
13-May Jon Krosnick. The collapse of scientific standards in the world of high visibility survey research
06-May Mark M Musen. When FAIR is foul and foul is FAIR, how can we ever measure data FAIRness?
29-Apr Jeroen Baas. When peer reviewers go rogue - Estimated prevalence of citation manipulation by reviewers based on the citation patterns of 69,000 reviewers
22-Apr Yian Yin. Coevolution of policy and science during the pandemic
15-Apr Riccardo De Bin. Modelling publication bias and p-hacking
08-Apr Brett Thombs and Brooke Levis. Use of individual participant data meta-analysis to address problematic research on mental health assessment
25-Mar Charles Bennett. Harms to physicians, patients, and pharmaceutical manufacturers following clinician reporting of titanic Adverse Drug Reactions
18-Mar Anne-Laure Boulesteix. A replication crisis in methodological statistical research?
11-Mar Florian Naudet Busting. 4 zombie trials in a post-apocalyptic world
04-Mar Jamie Kirkham. Preprint servers are rapidly disseminating crucial pandemic science – but do they safeguard against unfounded medical claims?
25-Feb Sabine Hoffmann The multiplicity of analysis strategies jeopardizes replicability: lessons learned across disciplines
18-Feb Prabhat Jha Serological and mortality studies of SARS-CoV-2 infection in Canada and India
11-Feb Noah Haber Problems with Evidence Assessment in COVID-19 Health Policy Impact Evaluation (PEACHPIE): A systematic strength of methods review
04-Feb Ben W Mol Are the data that ‘inform’ us true?
28-Jan Anna Dreber Predicting replication outcomes
2020: (Forum was closed - we are seeking permission to upload presentations):
Stefan Schandelmaier (Dec 17). A new library of methods guidance and other plans to improve knowledge translation from methods developers and meta-researchers to primary researchers
Terry Klassen (Dec 10). RCTs in Child Health: Frequentist approaches failing, is the future Bayesian analyses?
Katrin Auspurg and Alexander Tekles (Dec 3) .Do male researchers disregard the work of female researchers? The role of gender in citation decisions
Despina Koletsi (Nov 19). GRADE…ing Quality of the Evidence. Where do we stand?
Andreas Schneck (Nov 12). Are most published research findings false? Trends in statistical power, publication bias and p-hacking as well as the false discovery rate in psychology (1975–2017)
Janneke van 't Hooft (Nov 5). The road towards more Useful randomised controlled trials (RCTs). Preterm birth RCTs as an example.
Robert J MacCoun (Oct 29). p-Hacking: A Strategic Analysis
Atle Fretheim (Oct 22). Randomized trial of school closures in Norway (spoiler alert: it didn’t happen)
Hank Greely (Oct 15). CRISPR Babies: Assessing Human Germline Genome Editing
Deborah Zarin (Oct 8). Lack of Harmonization of Outcome Measures in COVID trials, and other Musings
Peter Gøtzsche (Oct 1). Mental health survival kit and withdrawal from psychiatric drugs
Peter Grabitz, Maia Salholz-Hillel, Nicholas Devito (Sept 10). Rapid Results Dissemination of Registered COVID-19 Clinical Trials
Jelte Wicherts (Sept 3). The continuing secrecy surrounding psychological research data
Mario Malički (Aug 27). From amazing work to I beg to differ: analysis of bioRxiv preprints that received one public comment till September 2019
Nikolaos Pandis (Aug 20). Who is running the show? The role of the industry in the practice of dentistry and a brief overview of evidence quality in dental research
David R Grimes (Aug 13). Lies, Damned lies, and statistics - the influence of bad science on public understanding
Kevin Boyack (July 30). A detailed open access model of the PubMed literature
Sally Cripps (July 23). A Tale of > 2 Models
Mario Malički (July 2). How should we classify changes between manuscript versions?
Vinay Prasad (June 18). The lay of the land of cancer research
Valentin Danchev (June 11). Designing a Clinical-Data Marketplace to Accelerate Sharing and Reuse of Covid-19 Clinical Data and Beyond
Chirag Patel (June 4). Real-world meta-science to identify environmental disparities and non-genetic correlates of phenotypesAbstract:
Denes Szucs (May 28). Sample size evolution in neuroimaging research
Perrine Janiaud (May 21). 100 DAYS LATER - The worldwide clinical trial research agenda in the beginning of the COVID-19 pandemic: results from COVID-evidence
Ben W Mol (May 14). Cost-effectiveness of liberal versus strict lock-down for COVID-19: are we using the correct metrics?
Cathrine Axfors (May 7). Population-level COVID-19 mortality risk for non-elderly individuals
Stephan Bruns (Apr 30). Estimating the extent of inflated significance in economic
Noah Haber and Sarah Wieten (Apr 23). DAG With Omitted Objects Displayed (DAGWOOD): A method for confronting causal assumptions
Florian Naudet (Apr 16). Will the ICMJE clinical data sharing policy achieve his intended objectives?
Benjamin Djulbegovic (Apr 9). Probability of discovering new treatments
Ioana Cristea (Apr 2). Reliability and accessibility of findings and tools reported in highly-cited research
Joshua Wallach (March 26). Opportunities for sharing and evaluating research: medRxiv and YODA project
Lars Hemkens (March 19). The COVID-evidence project: brainstorming on a new initiative