Study: Peer Review Predicts Success
Scientists who evaluate National Institutes of Health grant applications often identify the projects that will have the biggest scientific impact, according to an analysis. “[As] it turns out,” he added, “the NIH is doing a pretty good job.”
Overall, applicants with the highest-scoring grants published the most papers, garnered the most citations, and earned the most patents, researchers have found.
The National Institutes of Health (NIH) peer-review scoring system, which is used to select grant proposals for funding, is an accurate predictor of how impactful proposed research will ultimately become…Overall, applicants with the highest-scoring grants published the most papers, garnered the most citations, and earned the most patents, researchers have found.
“This is the most important science policy paper in a long time…”most of the pontifications that you hear—most of the anger, editorials, suggestions for reform—have been remarkably data-free. So this paper, as far as I am concerned, is really a breath of fresh air.”
Overall, they found “the better the score that the [peer-review] committee had assigned, the more likely the grant was to result in a high number of publications, or in publications that are highly cited, or even . . . in research that ultimately gets patented,” Agha told The Scientist. “The results are suggestive that the committees are successfully discriminating even amongst very strong applications.”
… This showed “that the intrinsic merit of a scientific idea is more valuable than the actual person,”….
The results “illustrate the ability of the NIH reviewers to identify which projects are going to be the most promising,” said Brian Jacob, a professor of education policy and economics at the University of Michigan. “It would have been a little worrying,” he added, “if they weren’t getting it right.”
“… Instead of having arguments about opinions . . . we’re debating actual data.”