A highway to publication

In its 2017 report on Didier Raoult’s unit, the French ‘Haut Conseil de l’évaluation de la recherche et de l’enseignement supérieur’ (High Council for the Evaluation of Research and Higher Education), an independent authority that inspects French research units, noted that the ‘creation of this journal, which serves to publish papers rejected by other journals, is a somewhat desperate initiative’. A careful inspection of the NMNI publication output (see online supplemental table 2) showed that the journal, created in 2013, had published 728 papers up to 25 June 2020. Of these, 231 (32%) were published by at least one author on the current editorial board, 226 (31%) by one editor from Marseille, and 235 (32%) by Didier Raoult, who is not part of the editorial board.

Computing the proportion of contributions published in a journal by any single author can provide a very rough index to spot problematic journals. We explored scientific journals specialised in infectious diseases selected from the National Library of Medicine Catalogue using NMNI MeSH terms (see online supplemental methods). Among these 789 journals, 239 published at least 50 papers between 2015 and 2019.

Figure 1A shows this indicator for the most prolific author for each journal in relation to the volume of the journal’s published output. NMNI is a strikingly clear outlier, with both a large proportion of published papers by the same author (37%) and a large publication volume over the last 5 years (N=598 articles). A sensitivity analysis was computed with ‘journal articles’ only (using the NCBI publication type) in order to exclude contributions such as editorials, news items or comments. The results of this sensitivity analysis were consistent with those for all articles. However, the case of Nature Review Microbiology raises the risk of misclassification bias as this author only contributes to ‘News & Comment’ or ‘In Brief’ section articles and should not have been flagged in the sensitivity analyses. We, therefore, think that a large proportion of papers published by one author could be used as a red flag—to identify journals that are suspected of dubious editorial practice—but deserves a subsequent qualitative investigation of the journal.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Description of the contributions of prolific authors across 239 infectious disease journals that published at least 50 papers between 2015 and 2019. (A) Percentage of contributions of the most prolific authors and numbers of published outputs for all journals (2015–2019). Grey lines correspond to the numbers of articles signed by the most prolific authors. The five journals presented in figure 1B are identified by colours. (B) Distribution of the contributions of each author, across the five journals ranking, respectively, at the minimum, first quartile, median, third quartile and maximum (2015–2019). All queries for PubMed extraction were performed using the easyPubMed library in R. The code to reproduce this analysis is available here: https://osf.io/dqvea/.

We, therefore, explored the 12 journals with an Index value >10.2% corresponding to the 95th percentile threshold. The key features of these journals are shown in table 1. Figure 1B shows the distribution of the index for each author, among the five journals ranking respectively at the minimum, first quartile, median, third quartile and maximum, over the 5 years, by year. Details for the whole sample of journals are presented in online supplemental figure 1. NMNI appears consistently as an outlier over the past 5 years.

Self-promotion journals: a new type of illegitimate publishing entity?

Of course, to avoid publication bias it is expected that all researchers transparently submit all outputs of their research. However, publications are presumed successful if the research is sound enough and a rigorous, unbiased peer-review actually took place. Elsevier’s general policies explicitly state that ‘the editor must not be involved in decisions about papers which she/he has written him/herself, or which have been written by […] colleagues.’ In application of this policy, more than 40% of published papers should not have been handled by Michel Drancourt, the editor-in-chief. This is a very large proportion for the editor, supposed to be responsible for the whole journal content. It is also expected that an editor of a journal should publish editorials delineating the agenda of the journal. However, a high proportion of this type of article raises questions about the plurality of viewpoints and the independence of the journal. A similar case was described in 2008 with Elsevier’s theoretical physics journal Chaos, Solitons and Fractals, whose Editor in Chief M.S. El Nashie published 332 papers in the journal as an author.11

In contrast with the El Nashie case, NMNI appears to prioritise the productivity of a larger network of editors/authors. We suggest that there is (1) a consistently large proportion of papers published by a group of authors, (2) particularly in the presence of relationships between the editors and these authors and (3) publication of low-quality research all key characteristics of a new type of illegitimate publishing entity, that is, ‘self-promotion journals’. The first two criteria have the advantage of being simple and easy to obtain for a given journal, but they are likely to detect only the more problematic journals deserving further investigation. Coincidentally, Dorothy Bishop in a blog post reported a similar analysis for the addiction subfield of psychology12 and found that 3 of 99 journals had more than 8% by the most prolific author. These two preliminary studies—in the field of infectious diseases and psychology—are being extended in a comprehensive survey of biomedical journals to refine the description of such dubious editorial practice.13 This survey showed that prolific authors were often associated with shorter lags between submission and publication which reinforces the idea of ‘self-promotion journals’.

‘Self-promotion journals’ could be deployed to game productivity-based metrics, with a ripple effect on decisions about promotion, tenure and grant funding. COVID-19 has clearly shown the detrimental effects of such practices: authorisations issued in March 2020 for chloroquine and hydroxychloroquine for emergency use, were suspended by the Food and Drug Administration 3 months later.14 Didier Raoult implicitly acknowledged using his publication capacity as clout in his own research ecosystem, even threatening to go on strike over the signing of his own publications.15 Indeed, in France, hospitals are rewarded according to the volume of publications. Various initiatives, including the Declaration on Research Assessment (DORA),16 warn against the use of incentives based on scientific productivity, which can easily be gamed and could be related to a kind of natural selection of bad science.17 In the case of NMNI, it is not possible to ascertain the integrity or quality of the peer review process because the journal does not have an open peer-review policy.

Possible consequences of dubious editorial practice on evidence-based medicine

A cornerstone of evidence-based medicine is the use of the best available evidence that have to be obtained from trustworthy findings. By carefully managing the peer-review process, editors are responsible for the quality of manuscripts that are accepted and consequently disseminated to the medical community.18 The International Committee of Medical Journal Editors explicitly states that ‘in return for the altruism and trust that make clinical research possible, the research enterprise has an obligation to conduct research ethically and to report it honestly’.19 In our opinion, such relationships between authors and editorial board members could (1) facilitate selective publication of clinical results driven by cronyism rather than the peer-review process and (2) facilitate publication of studies with high risk of bias, studies underpowered or misreporting/selective reporting research. Once published, these low-quality positive studies disrupt evidence-based medicine at several levels (table 2).

Rewarding integrity instead of productivity

Authorship is an important component of scientific integrity, it entails responsibilities20 and any doubts on actual authorship call the trustworthiness of the science into question. Publishers such as Elsevier can easily screen their catalogue using the indicators we propose to detect outliers such as NMNI and to audit the specific processes in these journals. Independent researchers can explore and refine the index we propose on the basis of an exhaustive study across a broad range of scientific journals to explore its validity and possible variations according to the field. It is indeed time to reward scientific integrity instead of productivity, institutions, journals or publishers. Following the Declaration on Research Assessment (DORA initiative16 or the Hong Kong principles for assessing researchers21 affords a good opportunity to determine which values matter: productivity-based metrics, research quality or the societal consequences of research.