Adam Mastroianni’s has a superb publish on strong-link vs weak-link problems in science. He writes:
Weak-link issues are issues the place the general high quality is dependent upon how good the worst stuff is. You repair weak-link issues by making the weakest hyperlinks stronger, or by eliminating them completely.
Meals security is a weak hyperlink downside, financial institution or pc safety is a weak-link downside, many manufacturing processes are weak-link, additionally referred to as O-ring problems.
[But] some issues are strong-link issues: total high quality is dependent upon how good the greatest stuff is, and the unhealthy stuff barely issues….Enterprise capital is a strong-link downside: it’s high-quality to spend money on a bunch of startups that go bust so long as one in all them goes to a billion.
….Right here’s the loopy factor: most individuals deal with science prefer it’s a weak-link downside.
Peer reviewing publications and grant proposals, for instance, is an enormous weak-link intervention. We spend ~15,000 collective years of effort yearly attempting to stop unhealthy analysis from being revealed. We pressure scientists to spend huge chunks of time filling out grant functions—most of which will probably be unsuccessful—as a result of we need to make sure that we aren’t losing our cash.
These insurance policies, like all types of gatekeeping, are probably terrific options for weak-link issues as a result of they will stamp out the worst analysis. However they’re horrible options for strong-link issues as a result of they will stamp out the greatest analysis, too. Reviewers are much less prone to greenlight papers and grants in the event that they’re novel, risky, or interdisciplinary. While you’re attempting to unravel a strong-link downside, that is like swallowing a giant lump of kryptonite.
At Maximum Progress, Max Tabarrok has some nice diagrams illustrating the issue:
When you’ve got a weak-link view of science, you’d assume peer overview works one thing like this. The connection between high quality and eventual impression is linear, or even perhaps bowed out a bit. Shifting sources from low enter high quality initiatives to common ones is not less than as essential to eventual impression as shifting sources from common initiatives to top quality ones.
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fecba4f75-22cd-458e-9216-cd57ec21456f_870x561.jpeg)
In a strong-link mannequin of science, filtering the underside half of the standard distribution is much less essential to remaining impression [because the impact of research is highly non-linear].
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6dfd74f9-9ef9-404c-a69d-033a52bcadfa_870x561.jpeg)
Although peer overview has the identical good filter on the standard distribution, it doesn’t translate into massive adjustments within the impression distribution. A lot of sources are nonetheless being given to initiatives with very low impression. Though the common enter high quality will increase by the identical quantity as within the weak hyperlink mannequin, the common remaining impression barely adjustments. Since peer overview has vital prices, the marginally greater common impression would possibly fail to make up for the losses in whole output in comparison with no peer overview.
It is a simplified mannequin however most of the simplifying assumptions are favorable for peer overview. For instance, peer overview right here is modeled as a filter on the underside finish of the standard distribution…But when peer overview additionally cuts out some initiatives on the highest finish, its enhance of the common impression of scientific analysis can be muted and even reversed.
The publish Strong and Weak Link Problems and the Value of Peer Review appeared first on Marginal REVOLUTION.