Making sense of the infodemic – A bibliometric analysis of publications on COVID-19 in trauma and orthopedics

INTRODUCTION

COVID-19 was declared a pandemic on March 11, 2020, by the WHO.^[1] Since it started, the number of cases has increased exponentially across the globe. For countering a new viral disease such as this, research should occur at a rapid pace and at a global scale almost equaling the pace and scale of its spread. The literature and number of publications on this topic have swelled at exponential scale and pace across specialties. Publications on overall bibliometrics on this topic do exist as well.^[2,3] However, no bibliometric studies looking at COVID-19 in trauma and orthopedics. To look at how the specialty of trauma and orthopedics has been affected by this “Infodemic,” one needs to study the publications within the journals of his specialty. We studied publications on COVID-19 on the broad subject of orthopedics and trauma across journals as well as those published within the orthopedic journals in PubMed, Scopus, as well as in altmetrics and compared the outputs.

A study with a search strategy looking exclusively into orthopedic journals has not been reported in the literature to our knowledge.

MATERIAL AND METHODS

We did the search in Scopus using a search strategy (TITLE-ABS-KEY (“COVID-19” OR “SARS-Cov-2” OR covid19 OR “new coronavirus”) AND TITLE-ABS-KEY (orthopedic* OR orthopedic* OR trauma OR bone OR joint) AND NOT TITLE-ABS-KEY ( psych* OR brain OR ptsd)) AND PUBYEAR > 2019 on June 19, 2020, and we found 272 articles. In the PubMed on the same day, we searched using a search strategy (((((“COVID-19”[All Fields] OR “SARSCov-2”[All Fields]) OR ((“COVID-19”[Supplementary Concept] OR “COVID-19”[All Fields]) OR “covid19”[All Fields])) OR “new coronavirus”[All Fields]) AND ((((“orthopedic*”[All Fields] OR “orthopedic*”[All Fields]) OR (((((((“injuries”[MeSH Subheading] OR “injuries”[All Fields]) OR “trauma”[All Fields]) OR “wounds and injuries”[MeSH Terms]) OR (“wounds”[All Fields] AND “injuries”[All Fields])) OR “wounds and injuries”[All Fields]) OR “trauma s”[All Fields]) OR “traumas”[All Fields])) OR (((“bone and bones”[MeSH Terms] OR (“bone”[All Fields] AND “bones”[All Fields])) OR “bone and bones”[All Fields]) OR “bone”[All Fields])) OR (((“joint s”[All Fields] OR “joints”[MeSH Terms]) OR “joints”[All Fields]) OR “joint”[All Fields]))) NOT ((“psych*”[All Fields] OR (((“brain”[MeSH Terms] OR “brain”[All Fields]) OR “brains”[All Fields]) OR “brain s”[All Fields])) OR (((“stress disorders, post-traumatic”[MeSH Terms] OR ((“stress”[All Fields] AND “disorders”[All Fields]) AND “post traumatic”[All Fields])) OR “post-traumatic stress disorders”[All Fields]) OR “ptsd”[All Fields]))) AND 2020/1/1:2020/6/18[Date - Create].

This resulted in an output of 887 articles (PubMed Output 1 – PO1).

To search within the orthopedic journals, we obtained a list of 274 orthopedic journals with full names from SCIMAGO.^[4] When we performed a search in PubMed using the full names, the search was not satisfactory. Hence, we found the short names of these journals from PubMed to search. Out of the 274 journals, only 217 were listed in the PubMed database and hence only the names of these journals were included in the search (PubMed Output 2 or PO2). This produced 258 articles. To the best of our knowledge, no previous search was performed to locate articles within the orthopedic journals.

For Scopus, the whole search strategy with 217 journal names exceeded the maximum allowed search field capacity of characters and even in advanced search, the inclusion of these journal names did not give any output. Hence, we were unable to do the search restricted to within the orthopedic journals in Scopus. In PubMed, there was no such restriction and hence the search was completed with an output of 258 articles. Our search strategy in orthopedic journals was essentially the same as the first search strategy (PO1) but except for the inclusion of the short names on 217 journals mentioned above.

Our altmetric search strategy in dimensions website 5 (Free Version) was ((((“COVID-19” OR “SARS-Cov-2” OR COVID19 OR “new coronavirus”) AND (orthopedic* OR orthopedic* OR trauma OR bone OR joint))) NOT (Psych* OR Brain OR PTSD)). It produced 7868 documents and when limited to 2020, the numbers were 7522.

Data were downloaded from PubMed and Scopus and stored and analyzed using Microsoft Excel 365. Citations were analyzed for top authors in PubMed using iCite website.^[6] We have enlisted the most cited articles and protocols published for the benefit of orthopedic surgeons.

RESULTS

There was a total of 887 articles from a PubMed search (PO1), 258 articles from orthopedic journals in PubMed (PO2), 272 articles from Scopus search, and 7522 publications from dimensions. There were only 188 publications that were common to both groups of PubMed (PO1 and PO2). So 699 from PO1 and 164 from PO2 were unique publications. 179 of the 272 listed in Scopus were also listed in either PO1 or PO2. Of the 1417 publications in PO1, PO2 and Scopus together, 761 articles were unique, and 656 articles were common to at least two search strategies. Dimensions include publications from preprint servers and hence the numbers are very high.

We have plotted the articles published (PO1 and PO2) in the past 25 weeks [Chart 1] and we can see that majority of the articles were published in 19^th, 20^th, and 21^stweeks, That is, 88, 102, 99, and 31, 39, and 26 articles, respectively. Publications within orthopedic journals started appearing from 9^th week onward.

Close

Chart 1:

Publications in the past 25 weeks in (PO1 & 2).

Export to PPT

We found that review articles were the most common articles [Chart 2], followed by editorial articles and cohort and surveys.

Close

Chart 2:

Types of studies versus weeks (PO1).

Export to PPT

There were 55 protocols in the list of publications. Of these, 20 were found to be relevant to orthopedic practice. We have given the list of protocols published along with their subjects as we thought it would be useful for the reader [Table 1].

There were a total of 887 articles published in 416 journals (PO1). The Journal of Bone and Joint Surgery (JBJS) American published the most articles, that is, 35 articles, amounting to 3.95% of all the articles, followed by the Journal of Arthroplasty (22, 2.48%) and International Orthopedics (20, 2.25%) [Chart 3]. Out of these, 631 articles from 301 journals are available with free full text. The maximum free full-text articles were from the Journal of Arthroplasty (21), followed by the Journal of Bone and Joint Surgery, (JBJS) American (15).

Close

Chart 3:

Top ten Orthopedic Journals Publishing on COVID-19 (PO1 and PO2).

Export to PPT

In our second search, that is, PO2 (specific orthopedic journals), there were a total of 258 articles from 58 journals. The Journal of Bone and Joint Surgery (JBJS) American published the most articles, that is, 36 articles, amounting to 13.95% of all the articles, followed by the Journal of Arthroplasty (22, 8.53%) and International Orthopedics (20, 7.75%)

There were numerous articles published from different countries on COVID-19 in both PO1 and PO2. After analyzing the most cited articles of first authors from different countries (taken from countries of articles from the top ten publishing authors), we found that there was a topic that each country was more focused on. In PO1, out of 98 articles published by the top ten authors, the majority of them were from China (61) followed by the United States of America (USA) (15), India (21), and Singapore (1). Out of 61 articles Published by China, 22 articles were related to clinical studies in medical issues of COVID-19 and 17 were about Basic Sciences, while USA articles were more focused on epidemiological studies (7). Indian articles were focused on general aspects (4) and Technology and Bibliography (8). In PO2, there were 35 articles published by the top ten authors from which countries were derived. Unlike PO1, India(11), USA(9), Canada(4), German (2), China (1), Singapore(1), and Italy(1) were the countries derived. Indian Articles were more focused on General aspects of COVID-19 in orthopedics (3), Technology and bibliometrics (4), whereas articles from USA were more focused on Arthroplasty(4), artificial intelligence(2), and impact on economy(1).

The maximum citation count from PubMed output 1 (PO1) was 1931 with a total count of 3764 for all articles. The maximum citation count from PubMed output 2 (PO2) was lower with ten with a total citation count from all articles at 162. The most cited article from the Scopus was with 38 citations and maximum citation counts from dimensions were 4700. We have listed the top five most cited articles from each search PO1, PO2, Scopus, and dimensions [Table 2].

There were 887 articles published from 416 journals in 2020 (PO1), and 258 articles from 58 journals in PO2. The total unique authors (non-repetitive and includes all authors) in PO1 and PO2 were 6936 and 1447, respectively [Table 3]. The maximum number of articles was published by Vaishya R (5) in PO1 and by Vaid S(3) in PO2 [Table 4].

From the Scopus search, we found 272 articles from 162 Journals after excluding psychiatry related articles. Journal of Arthroplasty published most of the articles, that is, 12 articles, followed by the Journal of Clinical Orthopedics and Trauma (9). The different types of articles published are given in Chart 4.

Close

Chart 4:

Types of articles in Scopus (n = 272).

Export to PPT

The majority of the articles published in Scopus were in the English language (120), followed by Chinese (6), French (4), and German (2) languages.

Altmetric analysis

When the output from the altmetric search was analyzed, the total number of citations in 2020 was 7836with a mean of 1.98. MedRxiv was the most publishing journal with 781 publications and 1616 citations [Table 5]. Raju Vaishya was the most published author with 20 publications and 59 citations [Table 6].

Table 5: Top ten journals in Dimensions.

Name	Publications	Citations	Citations Mean
medRxiv	781	1,616	2.07
SSRN Electronic Journal	573	227	0.40
Research Square	243	35	0.14
bioRxiv	155	476	3.07
Psychological Trauma Theory Research Practice	65	1	0.02
The BMJ	64	464	7.25
The Lancet	62	7,030	113.39
Science	50	705	14.10
Journal of Medical Virology	49	619	12.63
Mental Health Weekly	47	4	0.09

Table 6: Top authors in Dimensions.

Name Organization, Country	Publications	Citations	Citations mean
Raju Vaishya Indraprastha Apollo Hospitals, India	20	59	2.95
Gerardo Chowell Georgia State University, United Sates	18	401	22.28
Benjamin John Cowling University of Hong Kong, China	15	2,486	165.73
Adel Elkbuli University of Miami, United States	12	22	1.83
Giuseppe Lippi University of Verona, Italy	11	356	32.36
Xin-Miao Fu Fujian Normal University, china	11	16	1.45
Mark George Mckenney University of South Florida, United States	10	22	2.20
Kenji Mizumoto Kyoto University, Japan	10	93	9.30
Karthikayan P Iyengar Southport and Ormskirk Hospital NHS Trust, United Kingdom	9	12	1.33
MohdJavaid Jamia Milia Islamia, India	9	47	5.22
Abid Haleem Jamia Milia Islamia, India	9	47	5.22
Gareth Iacobucci British Medical Association, United Kingdom	9	14	1.56

We listed the top five publications with their altmetric attention scores and citations in Table 7.

Table 7: Top 5 publications according to AAS.

Article	AAS score	Citations
van Doremalen et al.[35]	25224	840
Huang et al.[20]	14242	4700
Wang et al.[38]	12422	110
van Doremalen et al.[35]	11248	53
Sanche et al.[39]	8655	90

We have given the linked page of the publication with maximum AAS score of 25,226 [Figures 1 and 2]. It has resulted in more than 26,411 tweets by 22,572 users on Twitter and followed by more than 24 million followers with 840 citations. The top five articles contained one article (Doremalen et al.) cited twice with two different AAS scores and citations but published by two different publishers.

Close

Figure 1:

Social media attention for Top AAS scored publication (Twitter and Facebook).

Export to PPT

Close

Figure 2:

Metrics scored for the top publication in altmetrics/dimensions.

Export to PPT

In dimensions, there were a total of 7522 articles, one dataset, seven grants, three patents, 111 clinical trials, and 192 policy documents. The 7522 articles were further divided into types given in [Figure 3]. Eighty-five percent of these were journal articles.

Close

Figure 3:

Publication types in dimensions.

Export to PPT

DISCUSSION

Bibliometric studies on COVID-19, on various non-orthopedic specialties, have been published before.^[2,3,39] These studies have been on the overall publications rather than specialty related. We have done a bibliometric analysis related to COVID-19 in trauma and orthopedics by two methods: Orthopedic related articles in general journals as well as those published in exclusively orthopedic journals. We found that there is an overlap of publications between both groups but neither group produced a complete list as there were differences between the groups. Furthermore, to see recent trends, we looked at the altmetrics through dimensions website to find the most popular articles in social media and research on this topic.

We matched the output of orthopedic journals from SCIMAGO, numbering 274, and looked up the list of all journals published by PubMed numbering 8504. Unfortunately, we got only 27 exact matches for the names derived from Scopus that matched with the full names present in the database of journals of PubMed, which makes it only one-tenth of the SCIMAGO list.

We attempted to compare outputs from three search engines using two search strategies resulting in four outcomes. A search specifically within a list of 217 orthopedic journals, and an output with altmetrics, to the best of our knowledge, has not been done before.

The outputs between the two searches were different with some common publications as seen in the author and journal numbers (PO1 and PO2). While the orthopedic search retrieved publications from within orthopedic journals only, the first search (PubMed output 1) retrieved citations from all journals (including orthopedic) related to the orthopedic keywords. The first search retrieved citations from maxillofacial, neurological, and psychiatric journals also, giving out papers related to dental trauma, post-traumatic stress disorders, and mental trauma. This is a drawback while searching through general journals. While the number of citations retrieved will be high for relevant articles, several unrelated citations may also be retrieved. The search among ortho journals was more specific. However, this search has left out some important citations related to trauma and orthopedics as well as general aspects of COVID-19 published in high impact medical journals. This is seen in the list of protocols derived. We found more protocols listed from the PO1 strategy than PO2. We have listed the protocols published on this topic since we felt it would be a useful list for the orthopedic surgeons in this time of crisis. The list is a combination of both search strategies. The majority of them derived were from the first strategy (PO1). Chart 1 shows the weekly publications from each search. PO1 strategy is higher than PO2 but the trends appear similar in both. Scopus gave a total of 272 articles, while dimensions gave 7522 publications. PO2 publications received lower citation numbers than PO1. This could because PO1 journals were more general with higher impact factors and also the fact that PO2 publications started appearing later than PO1 publications in the timeline [Chart 1].

Another option would be to combine the two search strategies. However, the drawbacks associated with the general search remain with such a search strategy. Even though the first search retrieved more articles, we have seen that it has missed some citations from JBJS Am as it retrieved only 35publications while the second search retrieved 36 citations from the journal JBJS Am.

We looked at the citation numbers received by publications by the leading authors on this topic in orthopedic journals. Of the total of 54 papers published by the top ten authors in Scopus, they received a total of 53 citations. On PubMed, we found 34 articles published by the top ten authors and analyzed in iCite. The total citations were 19 for 34 articles with a mean citation per pub of 0.56 and a maximum of four citations for a publication. Normally, it takes few years, on average 7–10 years to reach a peak in the number of citations received by a paper and very few citations are received in the first year. In this case, within the first 3 months, a total number of citations equal to the total number of publications by top authors are a notable finding.

Measuring interest and metrics in a particular branch of research for a new disease like COVID-19 are difficult for three reasons: (a) The rapid rate at which research evolves, (b) the scale at which the research publications are produced, and (c) the simultaneous branches of research that occur to understand and treat such a condition. Traditional ways of measuring performance metrics involve looking at and based on the citation counts for publications. While these may be good measures for slow evolving conditions such as elective or cold orthopedic conditions or procedures, they may not be ideal for rapidly changing diseases since they take time to accumulate.

With the availability of social media and electronic resources, research is disseminated at a rapid rate and altmetrics is used to measure the attention the research gets to give an altmetric attention score (AAS). This was developed to capture the attention a publication gets in the social media (Blogs, Facebook, Twitter, and Reddit), in addition to citations received. Among the current metrics available, AAS would be ideal for showing the immediate attention received by a publication. For a fast-evolving condition like COVID-19, where branches of research become unpredictable and fast-changing, even these metrics may have to be scrutinized on a monthly or even weekly basis to predict directions of research, analogous to the indices used to measure in financial markets where the second to second or minute by minute change is crucial in predicting trends.

The reach of social media sites has increased exponentially since its inception. The prominent sites among these include Twitter, Facebook, and Reddit. Metrics have been designed to assess the popularity of a topic on these sites. The most commonly used tool to improve and study these metrics is the hashtag in the social media. Altmetrics (or Alternate Metrics) are a new way of finding out the popularity of a publication in the social as well as the scientific circles. A score is given to each publication depending on popularity in the social media as well as citation counts obtained. This score has been named altmetric attention score (AAS).

We studied the altmetric scores for the same strategy used to search in PubMed (PO1) and presented the output based on AAS scores. The search field could not accommodate the strategy; we used to search in orthopedic journals (PO2). These scores may change from day to day depending on the interest gained or lost. One caveat about AAS is that not all attention gained is positive in all cases. Even negative attention could contribute to a high AAS. The dimensions website also provides options to view publications in decreasing order of Field Citation Ratio, Relative Citation Ration, as well as Citations of the publications produced by the search strategy.

The publication with maximum AAS score of 25226 has 840 citations, and it was published in the April 2020 issue of New England Journal of Medicine. Within 6 weeks, it has clocked 24 million followers on Twitter. Further studies are needed by tracking publications with such high AAS scores to see if they evolve to be the highest cited publications in the future. If that happens, AAS scores would be valuable in identifying quality publications much before peak citations are seen. However, for this to happen, one would need to see what the AAS scores of these publications were in the early stages. This study could act as a baseline for the assessment of these articles after some time to check on citation counts and altmetrics received.

Altmetrics give quick access to topics and branches of topics that are popular by way of the attention they receive in social media. The difference here from citation counts is that citations are given by the researchers and professionals of the same field of study. In contrast, AAS includes attention from everybody in the society. As researchers increasingly use social media for dissipation and sharing of ideas, major users could be researchers. Whereas with citation numbers, the attention gained by a publication is evaluated by other publications, in AAS, actions such as downloading an article, sharing and following of a study could increase its worth potentially giving important insight into the attention received by a publication from scientists as well as members of the society.

One disadvantage of too much reliance on altmetrics would be that to get higher AAS scores, manuscripts could cater to the general public and get less technical and conversely, manuscripts which are too technical, may not interest general public, and may not be shared as frequently in the social domain and hence could receive lower AAS scores than those which interest the general public.

As surgeons, we need good sources of information that give accurate and reliable information that can be used for our practice to benefit our patients. With evidence-based medicine forming a major component of today’s practice, sources of evidence become important. We endeavor to try and improve our sources to produce evidence, that is, accurate, reliable, reproducible, fast, and easy to access. There are multiple ways to search the literature and each method could produce a different outcome. With multiple search engines on the fray, it is important to know the strengths and weaknesses of each search engine to compose an effective search strategy. The best search strategy should produce an output that contains all the relevant citations that are recent, well-cited, important, and not leave out any article that is related to the search topic. It should also minimize citations that are not relevant to the topic that is being searched.

Bibliometric studies like ours offer several advantages [Table 8] as these are useful tools to measure the research impact quantitatively and hence become an objective way of analysis. It also helps in comparing the impact of a research more readily than with the peer review, which is subjective. Moreover, this type of analysis is transparent, and the results are reproducible, using the same methods. To conduct these studies is inexpensive and takes relatively lesser time, to produce and use.^[40]

CONCLUSIONS

The publications on COVID-19 started from 9^th week of 2020 and have increased exponentially, indicating proportional increase in research activity. Publications in specialist orthopedic journals began to appear later and had lower citation counts. There is a steady rise in the publication of protocols weekly on this topic. Reviews (PubMed) and direct articles (Scopus) were the most published and had been increasing recently. The JBJS (Am) and the J Arthroplasty have published the maximum number of articles on COVID-19 and provided free full-text articles, respectively. Publications from different countries focused on different aspects of the condition and its effects on orthopedics and trauma. We have listed clinically useful protocols from the search. This study should help in quantifying the value of research and publications related to orthopedics and trauma aspects of COVID-19 and therefore helps the readers, researchers, and health-care providers to use this information effectively.

Top five publications from PO1 search output had more citations than orthopedic specific search of PO2. An orthopedic journal-specific search may be useful for specialty-specific condition but for COVID-19, a combination search is useful. Future researchers should note this while designing search strategies. Although the COVID-19 related articles received quick citations in a short period, citations may not be ideal for rapidly changing diseases as they take time to accumulate. Publications with low citation counts could have immense social media attention. Altmetric scores may be useful to find branches of research gaining attention in the social and scientific circles. Standardization of names of journals between search engines would improve output from search strategies.