Reproducibility advanced by new NTP systematic review handbook
The National Toxicology Program (NTP) released a handbook Jan. 12 detailing procedures for systematic review in literature-based health assessments. The NTP Office of Health Assessment and Translation (OHAT), which developed this methods guide for evaluating potential human health hazards, hopes use of the handbook will enhance reproducibility of environmental health sciences research, which is a growing concern across all fields of scientific research and is the subject of recent National Institutes of Health initiatives.
As part of the systematic review process, the “Handbook for Conducting a Literature-Based Health Assessment Using OHAT Approach for Systematic Review and Evidence Integration” features a tool for assessing the risk of bias in a study’s findings. The tool applies an approach to evaluating the risk of bias in human and animal studies that is similar to the methods used to critically assess clinical trials of new drugs.
“Establishing transparent and objective standards for systematic reviews in literature-based evaluations builds confidence in the subsequent public health decisions,” said Kristina Thayer, Ph.D., director of OHAT. “And as the methodology becomes adopted more widely, the reproducibility of environmental health studies will be enhanced.”
From clinical trials to broader application
“Interest has been growing in the fields of toxicology and pharmacology in extending systematic review methods beyond the traditional area of human clinical trials to consider other evidence streams,” said NTP leaders and staff in a July 2014 editorial in the journal Environmental Health Perspectives. Authors include NIEHS and NTP Director Linda Birnbaum, Ph.D., NTP Associate Director John Bucher, Ph.D., Thayer, and others.
“This handbook provides the methods for completing a literature-based evaluation for environmental health questions, and it addresses whether or not exposure to chemical X is associated with health effect Y,” said Andrew Rooney, Ph.D., OHAT deputy director. “This document is intended to outline NTP methods, but it can also serve as a methods SOP [standard operating procedure] for external researchers,” he said, noting that this is the first handbook to cover systematic review methods for environmental health questions.
Risk of bias tool
Reproducibility of scientific studies increases when consistent means are used to assess internal validity, or risk of bias. Bias can lead to either underestimation or overestimation of an observed effect. According to the editorial quoted above, there is a general consensus on methods used to assess outcomes of human clinical trials. However, environmental health sciences research uses studies of widely varying design — from epidemiology to mechanistic studies. “There is not currently a similar consensus on how to assess that the findings and conclusions of observational human, experimental animal, and in vitro studies are a true reflection of the outcome,” the editorial’s authors said.
To meet that need, OHAT developed a risk of bias tool, included in the handbook, for evaluating environmental health studies in humans or animals. Rooney pointed out that, as with the systematic review methodology, the tool outlines NTP methods and may also be used by external researchers to assess study quality.