Being curated into an academic review journal is, on average, detrimental for scientific publications and leads to a dramatic overall decrease in future citations, according to a new study suggesting that academic review articles engage in a sort of "creative destruction."

Academic review articles are articles that summarize previously published academic studies in order to paint, in broad strokes, a holistic picture and explain the current state of understanding in a particular field of knowledge. Academic journals that curate and summarize these previously published works are known as review journals. 

The study, published March 18 in the American Sociological Review, is the first of its kind to quantify the effects of being included in an academic review article.

With data from Clarivate's Web of Science citation database, the researchers were able to examine a dataset that yielded approximately 5.9 million articles published across 1,155 journals spanning disciplines in the biological, physical and social sciences for a period of 27 years — from 1990 to 2016.

The researchers of this study initially approached the topic of review journals with a "black-box" theoretical framework, said Peter McMahan, a co-author of the paper and assistant professor of sociology at McGill University. 

"What a review is potentially going to do is turn this research into a nice black box that people can consume; something where they can say, 'I don't know about all those details in there, I just want resolved solutions — a conclusion to that particular body of research,'" McMahan told The Academic Times.

However, a deeper look at review journals revealed that the work they were performing was actually more robust than originally thought. 

"We ended up having to step out of that a little bit," McMahan said. "It turns out that these reviews are encapsulating knowledge into something digestible for people outside the field. But they don't just do this by simplifying that down to a single finding or a single punchline."

Instead, these review journals create a simplified version of the large scale arguments, divisions and dichotomies that are going on within an area: "Something legible and coherent that you can understand the large strokes of without diving into the nitty-gritty details that the small scale domain experts are very concerned with," according to McMahan.

The researchers determined, then, that review journals perform imperative work in academia that pushes scholarly discourse forward — but it may come at a cost.

In academia, "There's this common idea that you'll hear people talk about: Reviews poaching citations from people — whether they poach attention or promote attention," McMahan said. "We set out to determine whether being included in a review article negatively impacted a research study's number of citations." 

Using a hierarchical negative-binomial model, the researchers found that articles cited in a review, on average, saw a significant reduction in the number of future citations, with the median article receiving 38.3% fewer citations for each review the article is included in. This finding suggests that review articles divert attention away from the original study and replace it with the review article instead. 

"We genuinely did not know whether review articles poach attention or bring attention," McMahan said, but he noted that, "The magnitude of this was surprising. This was a very significant, very visible drop-off once we actually came up with the model and estimated it."

The researchers also noticed another surprising finding: "In general, almost all articles included in a review dropped in their citations, but we noticed that there were a few articles all the time, with every one of these reviews, that would get this massive boost instead," McMahan said. 

The researchers found that a small subset of cited articles that bridged disconnected research clusters received increased citational attention after being included in a review article, and the articles receiving the largest boost in citations "are those that draw connections between research clusters that were previously seen as residing in distinct scientific domains," according to the researchers. 

Bridge articles perform a unifying process that allows the scientific community to use these articles as the linchpins — the exemplars — to talk about a particular set of research as a coherent whole, according to McMahan. 

To illustrate, the researchers pointed to a 1997 review article, titled "Integrated Assessment Models of Global Climate Change."

"Before the review came out, there were all these hardcore computational climate modeling studies and then there were people doing policy level climate stuff," McMahan said. "But there were a handful of articles — just one or two that bridged these — showing that these communities were talking about the same thing, which turned into this larger integrated assessment model framework."

The study found that a handful of bridge articles with the largest boost saw an increase of nearly seven times the future citations than would otherwise be expected.

McMahan cautioned, though, that this research studied only academic attention using citations as a proxy; it does not determine the quality of that attention, whether positive or negative. 

Since the publishing of the study, McMahan told The Academic Times that people have interpreted the study to be showing the vices of review journals, but that is not how the researchers themselves see it. 

"The big takeaway I took from conducting this research wasn't that reviews are damaging in some way or somehow felt harmful towards academic discourse," McMahan said. 

"This kind of curation, this kind of synthesis, is doing productive work," he said. "Even though it might be doing so by drawing attention away from relevant research — hence our title 'creative destruction'— it does that in exchange for creating a coherent picture that is a synthesis of something that is bigger than the sum of its part. "

"Reviews are doing important work by finding these exemplars and showing how they relate to each other," McMahan said, "It's actually making these domains of knowledge — these subfields — more useful and something we can actually do something with as academics and scientists."

The study "Creative Destruction: The Structural Consequences of Scientific Curation," published March 18 in the American Sociological Review, was co-authored by Peter McMahan, McGill University; and Daniel McFarland, Stanford University.