Novelty scores can help journals to predict whether a manuscript will be impactful.Credit: PRUDENCIOALVAREZ/iStock via Getty
A publication platform called DeSci Publish aims to predict the impact of manuscripts by giving them a ‘novelty score’. The developers say that this score could assist journal editors in deciding which studies to publish, and could be an asset for researchers who are looking to be published in a competitive journal.
The novelty score is calculated using an algorithm that compares the combinations of keywords and cited journals in a scientific manuscript with those in previous publications and projects the types of paper that will be published in the future. From this, it identifies novelty as deviations from these predictions. A high novelty score indicates that a paper connects ideas in unexpected ways, which are likely to be innovative and impactful. The algorithm was first published in 2023 by researchers at the University of Chicago, Illinois1, who are not involved with the platform.
“Novelty is defined as a combination of concepts or topics that is unusual [and] that people have not worked on [before],” says Philipp Koellinger, a social geneticist at the Free University of Amsterdam and the president of the DeSci Foundation, a non-profit organization in Geneva, Switzerland, that advocates for openly available and reproducible science.
Since its launch by the DeSci Foundation in September, DeSci Publish has given novelty scores to more than 50 million manuscripts that are catalogued by OpenAlex, an index of scientific documents run by Canadian non-profit firm OurResearch. Koellinger says that the team plans to continue releasing novelty scores of newly published literature.
The hope is that, by releasing novelty scores for millions of papers, the platform will spur academics to explore the scientific literature more broadly by looking into topics that they would not normally have considered, says Koellinger. He also thinks that such scores could give authors more data with which to demonstrate the potential impact of their work. The scores “are actually very highly correlated with the future citations of an article”, says Koellinger.
Haining Wang, an information scientist at Indiana University Bloomington, who has previously reported on measuring novelty in research2, says that the algorithm used by the DeSci team is “on the right track”.
Hard to measure
Previous studies on predicting and measuring novelty have looked at unique word combinations and citation patterns4, but there is no clear agreement on what counts as innovative research. This makes it difficult to come up with a novelty score that everyone can agree on.
Original research is also often risky and hard to track because it gets cited at different rates and can influence many fields, both directly and indirectly.
2024 Research Leaders