Making sense of the infodemic – A bibliometric analysis of publications on COVID-19 in trauma and orthopedics
How to cite this article: Kambhampati SB, Vaishya R, Paleti ST, Khanduja V. Making sense of the infodemic – A bibliometric analysis of publications on COVID-19 in trauma and orthopedics. Indian J Med Sci 2020;72(3):155-66.
The COVID-19 pandemic has prompted an explosion of publications to report, understand, further research, and manage this condition. While publications are analyzing the bibliometrics on this condition, there are none available specifically for the impact of COVID-19 on trauma and orthopedics. The aim of this study, therefore, was to perform a bibliometric analysis on COVID-19 and trauma and orthopedics to assess its impact on the specialty. A search for articles on COVID-19 concerning trauma and orthopedics, with the keywords: “COVID-19, New coronavirus, SARS-Cov-2, Orthopedic*, trauma, bone, and joint” were performed on the June 19, 2020, using SCOPUS and PUBMED and this resulted in 272 and 887 articles, respectively. Later, on the same day, we searched for orthopedic journals exclusively and extracted 258 articles from 58 journals. Furthermore, we analyzed the Altmetric data through the dimensions website to find the most popular articles on social media on this topic. After analyzing the data, we found that review articles were the most commonly published articles. The leading journal publishing this content were; The Journal of Bone and Joint Surgery (JBJS) American (35), followed by the Journal of Arthroplasty (22). There were 6936 authors involved in publishing 887 articles in 2020. Most articles were published by Vaishya (5) followed by Liang (5), and Iyengar (5). Analysis of Altmetric data showed a total number of citations of 5000 with a mean of 1.98. MedRxiv with 781 publications and 1616 citations was the preprint server with the most publications on dimensions. We studied details of the article with maximum AAS score of 25226 is with 840 citations. We have listed useful protocols from the search and top five cited articles from each search strategy. Publications on COVID-19 commenced from the 9th week of this year and have increased exponentially. Review articles (PubMed) and articles (Scopus) were the most published. The JBJS (Am) and J Arthroplasty have published the maximum number of articles on COVID-19. We found that for a fast evolving condition and for the short term, altmetrics may be better indicators than citations to follow directions of research. Publications with a low number of citations could have immense social media attention. This study should help in quantifying the value of research and publications related to orthopedics and trauma aspects of COVID-19 and therefore help the readers, researchers, and health-care providers to use this information effectively.
Bibliometrics of COVID-19 in Trauma and Orthopedics
Orthopedic publications on COVID-19
COVID-19 publications in orthopedic Journals
Altmetrics of orthopedic publications on COVID-19
COVID-19 was declared a pandemic on March 11, 2020, by the WHO. Since it started, the number of cases has increased exponentially across the globe. For countering a new viral disease such as this, research should occur at a rapid pace and at a global scale almost equaling the pace and scale of its spread. The literature and number of publications on this topic have swelled at exponential scale and pace across specialties. Publications on overall bibliometrics on this topic do exist as well.[2,3] However, no bibliometric studies looking at COVID-19 in trauma and orthopedics. To look at how the specialty of trauma and orthopedics has been affected by this “Infodemic,” one needs to study the publications within the journals of his specialty. We studied publications on COVID-19 on the broad subject of orthopedics and trauma across journals as well as those published within the orthopedic journals in PubMed, Scopus, as well as in altmetrics and compared the outputs.
A study with a search strategy looking exclusively into orthopedic journals has not been reported in the literature to our knowledge.
MATERIAL AND METHODS
We did the search in Scopus using a search strategy (TITLE-ABS-KEY (“COVID-19” OR “SARS-Cov-2” OR covid19 OR “new coronavirus”) AND TITLE-ABS-KEY (orthopedic* OR orthopedic* OR trauma OR bone OR joint) AND NOT TITLE-ABS-KEY ( psych* OR brain OR ptsd)) AND PUBYEAR > 2019 on June 19, 2020, and we found 272 articles. In the PubMed on the same day, we searched using a search strategy (((((“COVID-19”[All Fields] OR “SARSCov-2”[All Fields]) OR ((“COVID-19”[Supplementary Concept] OR “COVID-19”[All Fields]) OR “covid19”[All Fields])) OR “new coronavirus”[All Fields]) AND ((((“orthopedic*”[All Fields] OR “orthopedic*”[All Fields]) OR (((((((“injuries”[MeSH Subheading] OR “injuries”[All Fields]) OR “trauma”[All Fields]) OR “wounds and injuries”[MeSH Terms]) OR (“wounds”[All Fields] AND “injuries”[All Fields])) OR “wounds and injuries”[All Fields]) OR “trauma s”[All Fields]) OR “traumas”[All Fields])) OR (((“bone and bones”[MeSH Terms] OR (“bone”[All Fields] AND “bones”[All Fields])) OR “bone and bones”[All Fields]) OR “bone”[All Fields])) OR (((“joint s”[All Fields] OR “joints”[MeSH Terms]) OR “joints”[All Fields]) OR “joint”[All Fields]))) NOT ((“psych*”[All Fields] OR (((“brain”[MeSH Terms] OR “brain”[All Fields]) OR “brains”[All Fields]) OR “brain s”[All Fields])) OR (((“stress disorders, post-traumatic”[MeSH Terms] OR ((“stress”[All Fields] AND “disorders”[All Fields]) AND “post traumatic”[All Fields])) OR “post-traumatic stress disorders”[All Fields]) OR “ptsd”[All Fields]))) AND 2020/1/1:2020/6/18[Date - Create].
This resulted in an output of 887 articles (PubMed Output 1 – PO1).
To search within the orthopedic journals, we obtained a list of 274 orthopedic journals with full names from SCIMAGO. When we performed a search in PubMed using the full names, the search was not satisfactory. Hence, we found the short names of these journals from PubMed to search. Out of the 274 journals, only 217 were listed in the PubMed database and hence only the names of these journals were included in the search (PubMed Output 2 or PO2). This produced 258 articles. To the best of our knowledge, no previous search was performed to locate articles within the orthopedic journals.
For Scopus, the whole search strategy with 217 journal names exceeded the maximum allowed search field capacity of characters and even in advanced search, the inclusion of these journal names did not give any output. Hence, we were unable to do the search restricted to within the orthopedic journals in Scopus. In PubMed, there was no such restriction and hence the search was completed with an output of 258 articles. Our search strategy in orthopedic journals was essentially the same as the first search strategy (PO1) but except for the inclusion of the short names on 217 journals mentioned above.
Our altmetric search strategy in dimensions website 5 (Free Version) was ((((“COVID-19” OR “SARS-Cov-2” OR COVID19 OR “new coronavirus”) AND (orthopedic* OR orthopedic* OR trauma OR bone OR joint))) NOT (Psych* OR Brain OR PTSD)). It produced 7868 documents and when limited to 2020, the numbers were 7522.
Data were downloaded from PubMed and Scopus and stored and analyzed using Microsoft Excel 365. Citations were analyzed for top authors in PubMed using iCite website. We have enlisted the most cited articles and protocols published for the benefit of orthopedic surgeons.
There was a total of 887 articles from a PubMed search (PO1), 258 articles from orthopedic journals in PubMed (PO2), 272 articles from Scopus search, and 7522 publications from dimensions. There were only 188 publications that were common to both groups of PubMed (PO1 and PO2). So 699 from PO1 and 164 from PO2 were unique publications. 179 of the 272 listed in Scopus were also listed in either PO1 or PO2. Of the 1417 publications in PO1, PO2 and Scopus together, 761 articles were unique, and 656 articles were common to at least two search strategies. Dimensions include publications from preprint servers and hence the numbers are very high.
We have plotted the articles published (PO1 and PO2) in the past 25 weeks [Chart 1] and we can see that majority of the articles were published in 19th, 20th, and 21stweeks, That is, 88, 102, 99, and 31, 39, and 26 articles, respectively. Publications within orthopedic journals started appearing from 9th week onward.
We found that review articles were the most common articles [Chart 2], followed by editorial articles and cohort and surveys.
There were 55 protocols in the list of publications. Of these, 20 were found to be relevant to orthopedic practice. We have given the list of protocols published along with their subjects as we thought it would be useful for the reader [Table 1].
|Gruskay et al.,J Bone Joint Surg Am||Universal Testing for COVID-19 in Essential Orthopedic Surgery Reveals a High Percentage of Asymptomatic Infections|
|Awad et al.,J Am AcadOrthop Surg||Perioperative Considerations in Urgent Surgical Care of Suspected and Confirmed COVID-19 Orthopedic Patients: Operating Room Protocols and Recommendations in the Current COVID-19 Pandemic|
|Luengo-Alonso et al.,||Critical adjustments in a department of orthopedics through the COVID-19 pandemic|
|Chellamuthu et al.,Int Orthop||Pandemic response protocol of a non-frontline specialty in a multispecialty tertiary health care center a pilot model in orthopedics|
|Keny et al.,J Orthop||Emergency and Urgent Orthopedic Surgeries in non-COVID patients during the COVID 19 pandemic: Perspective from India|
|Randelli et al.Knee Surg Sports Traumatol Arthrosc||Management of orthopedic and traumatology patients during the Coronavirus disease (COVID-19) pandemic in Northern Italy|
|Tanaka et al.J Bone Joint Surg Am||Telemedicine in the Era of COVID-19: The Virtual Orthopedic Examination|
|Sornsa-Ard et al.,Asian Spine J||Management of Traumatic Spinal Fracture in the Coronavirus Disease 2019 Situation|
|Stillman et al.Spinal Cord Ser Cases||COVID-19 and spinal cord injury and disease: Results of an international survey|
|Firstenberg et al. Patient Saf Surg||Isolation protocol for a COVID-2019 patient requiring emergent surgical intervention: case presentation|
|Ducournau et al. Hand Surg Rehabil||COVID-19: Initial experience of an international group of hand surgeons|
|Tang et al., Int J Surg||Avoiding health worker infection and containing the coronavirus disease 2019 pandemic: Perspectives from the frontline in Wuhan|
|Gong et al. AnesthAnalg||Anesthesia Considerations and Infection Precautions for Trauma and Acute Care Cases During the COVID-19 Pandemic|
|Guo F et al. J Med Virol||An effective screening and management process in the outpatient clinic for patients requiring hospitalization during the COVID-19 pandemic|
|O’Reilly et al. Emerg Med Australas||Informing emergency care for COVID-19 patients: The COVID-19 Emergency Department Quality Improvement Project protocol|
|Coccolini et al. World J Emerg Surg||Surgery in COVID-19 patients: Operational directives|
|Bajwa et al. Indian J Anaesth||Perioperative and critical care concerns in coronavirus pandemic|
|Huang et al. Crit Care||Special attention to nurses’ protection during the COVID-19 epidemic|
|Casiraghi et al. Int Orthop||Operational strategies of a trauma hub in early coronavirus disease 2019 pandemic|
|Barnabas et al., Trials||Efficacy of hydroxychloroquine for post-exposure prophylaxis to prevent severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection among adults exposed to coronavirus disease (COVID-19): a structured summary of a study protocol for a randomized controlled trial|
There were a total of 887 articles published in 416 journals (PO1). The Journal of Bone and Joint Surgery (JBJS) American published the most articles, that is, 35 articles, amounting to 3.95% of all the articles, followed by the Journal of Arthroplasty (22, 2.48%) and International Orthopedics (20, 2.25%) [Chart 3]. Out of these, 631 articles from 301 journals are available with free full text. The maximum free full-text articles were from the Journal of Arthroplasty (21), followed by the Journal of Bone and Joint Surgery, (JBJS) American (15).
In our second search, that is, PO2 (specific orthopedic journals), there were a total of 258 articles from 58 journals. The Journal of Bone and Joint Surgery (JBJS) American published the most articles, that is, 36 articles, amounting to 13.95% of all the articles, followed by the Journal of Arthroplasty (22, 8.53%) and International Orthopedics (20, 7.75%)
There were numerous articles published from different countries on COVID-19 in both PO1 and PO2. After analyzing the most cited articles of first authors from different countries (taken from countries of articles from the top ten publishing authors), we found that there was a topic that each country was more focused on. In PO1, out of 98 articles published by the top ten authors, the majority of them were from China (61) followed by the United States of America (USA) (15), India (21), and Singapore (1). Out of 61 articles Published by China, 22 articles were related to clinical studies in medical issues of COVID-19 and 17 were about Basic Sciences, while USA articles were more focused on epidemiological studies (7). Indian articles were focused on general aspects (4) and Technology and Bibliography (8). In PO2, there were 35 articles published by the top ten authors from which countries were derived. Unlike PO1, India(11), USA(9), Canada(4), German (2), China (1), Singapore(1), and Italy(1) were the countries derived. Indian Articles were more focused on General aspects of COVID-19 in orthopedics (3), Technology and bibliometrics (4), whereas articles from USA were more focused on Arthroplasty(4), artificial intelligence(2), and impact on economy(1).
The maximum citation count from PubMed output 1 (PO1) was 1931 with a total count of 3764 for all articles. The maximum citation count from PubMed output 2 (PO2) was lower with ten with a total citation count from all articles at 162. The most cited article from the Scopus was with 38 citations and maximum citation counts from dimensions were 4700. We have listed the top five most cited articles from each search PO1, PO2, Scopus, and dimensions [Table 2].
|Author, Journal, (PO1/PO2)||Title||Citation count|
|Huang et al., Lancet (PO1)||Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China.||1931|
|Yao et al. Infect. Dis. (PO1)||In vitroantiviral activity and projection of optimized dosing design of hydroxychloroquine for the treatment of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).||143|
|Xu et al., Sci China Life Sci (PO1)||Evolution of the novel coronavirus from the ongoing Wuhan outbreak and modeling of its spike protein for risk of human transmission.||117|
|Zheng et al., Nat Rev Cardiol (PO1)||COVID-19 and the cardiovascular system.||112|
|Liu et al., Sci China Life Sci (PO1)||Clinical and biochemical indexes from 2019 to nCoV infected patients linked to viral loads and lung injury.||111|
|Chen et al. J Sport Health Sci (PO2)||Coronavirus disease (COVID-19): The need to maintain regular physical activity while taking precautions||10|
|Zou et al., Asian Spine J (PO2)||Advice on Standardized Diagnosis and Treatment for Spinal Diseases during the Coronavirus Disease 2019 Pandemic.||9|
|Guo et al., J Bone Joint Surg Am (PO2)||Survey of COVID-19 Disease Among Orthopedic Surgeons in Wuhan, People’s Republic of China.||9|
|Stahel et al., Patient Saf Surg (PO2)||How to risk-stratify elective surgery during the COVID-19 pandemic?||9|
|Rodrigues-Pinto et al., J Bone Joint Surg Am (PO2)||Preparing to Perform Trauma and Orthopedic Surgery on Patients with COVID-19.||6|
|Chen et al., Anesthesiology (Scopus)||Perioperative Management of Patients Infected with the Novel Coronavirus: Recommendation from the Joint Task Force of the Chinese Society of Anesthesiology and the Chinese Association of Anesthesiologists||38|
|Zhao et al., Journal of Cardiothoracic and Vascular Anesthesia (Scopus)||Anesthetic Management of Patients with COVID 19 Infections during Emergency Procedures||18|
|Vannabouathong et al, Journal of Bone and Joint Surgery - American Volume (Scopus)||Novel coronavirus COVID-19 current evidence and evolving strategies||11|
|Mi et al., Journal of Bone and Joint Surgery - American Volume (Scopus)||Characteristics and early prognosis of COVID-19 infection in fracture patients||10|
|Guo et al., The Journal of bone and joint surgery. American volume (Scopus)||Survey of COVID-19 Disease Among Orthopedic Surgeons in Wuhan, People’s Republic of China||8|
|Huang et al., Lancet (Dimensions)||Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China.||4700|
|Li et al., NEJM (Dimensions)||Early Transmission dynamics in Wuhan, China of novel Coronavirus infected Pneumonia||2100|
|Doremalen et al., NEJM (Dimensions)||Aerosol and surface stability of SARS-Cov-2 as compared to SARS-Cov-1||840|
|Mehta et al., Lancet (Dimensions)||COVID-19: Consider Cytokine storm syndrome and immunosuppression||683|
|Cao et al., NEJM (Dimensions)||A trial of Lopinavir - Ritonavir in adults hospitalized with severe COVID-19||659|
There were 887 articles published from 416 journals in 2020 (PO1), and 258 articles from 58 journals in PO2. The total unique authors (non-repetitive and includes all authors) in PO1 and PO2 were 6936 and 1447, respectively [Table 3]. The maximum number of articles was published by Vaishya R (5) in PO1 and by Vaid S(3) in PO2 [Table 4].
|Total Free Full-text articles||631||193||-|
|Total number of first authors||772||240||260|
|Total number of Unique authors||6936||1447||1894|
|Total number of journals||416||58||162|
|Total Free full-text journals||301||45||-|
|Vaishya (5)||Vaid S (3)|
|Liang (5)||Chen P (3)|
|Iyengar (5)||Ding BTK (2)|
|Zhang (4)||Hughes D (2)|
|Sen-Crowe (4)||Ducournau F (2)|
|Lakkireddy (4)||Dyer GSM (2)|
|O’Reilly (4)||Thaler M (2)|
|Chakraborty (4)||Haddad FS (2)|
|Kumar (4)||Yadav SK (2)|
|Cioffi (4)||Hernigou J (2)|
From the Scopus search, we found 272 articles from 162 Journals after excluding psychiatry related articles. Journal of Arthroplasty published most of the articles, that is, 12 articles, followed by the Journal of Clinical Orthopedics and Trauma (9). The different types of articles published are given in Chart 4.
The majority of the articles published in Scopus were in the English language (120), followed by Chinese (6), French (4), and German (2) languages.
When the output from the altmetric search was analyzed, the total number of citations in 2020 was 7836with a mean of 1.98. MedRxiv was the most publishing journal with 781 publications and 1616 citations [Table 5]. Raju Vaishya was the most published author with 20 publications and 59 citations [Table 6].
|SSRN Electronic Journal||573||227||0.40|
|Psychological Trauma Theory Research Practice||65||1||0.02|
|Journal of Medical Virology||49||619||12.63|
|Mental Health Weekly||47||4||0.09|
|Name Organization, Country||Publications||Citations||Citations mean|
Indraprastha Apollo Hospitals, India
Georgia State University, United Sates
|Benjamin John Cowling
University of Hong Kong, China
University of Miami, United States
University of Verona, Italy
Fujian Normal University, china
|Mark George Mckenney
University of South Florida, United States
Kyoto University, Japan
|Karthikayan P Iyengar
Southport and Ormskirk Hospital NHS Trust, United Kingdom
Jamia Milia Islamia, India
Jamia Milia Islamia, India
British Medical Association, United Kingdom
We listed the top five publications with their altmetric attention scores and citations in Table 7.
We have given the linked page of the publication with maximum AAS score of 25,226 [Figures 1 and 2]. It has resulted in more than 26,411 tweets by 22,572 users on Twitter and followed by more than 24 million followers with 840 citations. The top five articles contained one article (Doremalen et al.) cited twice with two different AAS scores and citations but published by two different publishers.
In dimensions, there were a total of 7522 articles, one dataset, seven grants, three patents, 111 clinical trials, and 192 policy documents. The 7522 articles were further divided into types given in [Figure 3]. Eighty-five percent of these were journal articles.
Bibliometric studies on COVID-19, on various non-orthopedic specialties, have been published before.[2,3,39] These studies have been on the overall publications rather than specialty related. We have done a bibliometric analysis related to COVID-19 in trauma and orthopedics by two methods: Orthopedic related articles in general journals as well as those published in exclusively orthopedic journals. We found that there is an overlap of publications between both groups but neither group produced a complete list as there were differences between the groups. Furthermore, to see recent trends, we looked at the altmetrics through dimensions website to find the most popular articles in social media and research on this topic.
We matched the output of orthopedic journals from SCIMAGO, numbering 274, and looked up the list of all journals published by PubMed numbering 8504. Unfortunately, we got only 27 exact matches for the names derived from Scopus that matched with the full names present in the database of journals of PubMed, which makes it only one-tenth of the SCIMAGO list.
We attempted to compare outputs from three search engines using two search strategies resulting in four outcomes. A search specifically within a list of 217 orthopedic journals, and an output with altmetrics, to the best of our knowledge, has not been done before.
The outputs between the two searches were different with some common publications as seen in the author and journal numbers (PO1 and PO2). While the orthopedic search retrieved publications from within orthopedic journals only, the first search (PubMed output 1) retrieved citations from all journals (including orthopedic) related to the orthopedic keywords. The first search retrieved citations from maxillofacial, neurological, and psychiatric journals also, giving out papers related to dental trauma, post-traumatic stress disorders, and mental trauma. This is a drawback while searching through general journals. While the number of citations retrieved will be high for relevant articles, several unrelated citations may also be retrieved. The search among ortho journals was more specific. However, this search has left out some important citations related to trauma and orthopedics as well as general aspects of COVID-19 published in high impact medical journals. This is seen in the list of protocols derived. We found more protocols listed from the PO1 strategy than PO2. We have listed the protocols published on this topic since we felt it would be a useful list for the orthopedic surgeons in this time of crisis. The list is a combination of both search strategies. The majority of them derived were from the first strategy (PO1). Chart 1 shows the weekly publications from each search. PO1 strategy is higher than PO2 but the trends appear similar in both. Scopus gave a total of 272 articles, while dimensions gave 7522 publications. PO2 publications received lower citation numbers than PO1. This could because PO1 journals were more general with higher impact factors and also the fact that PO2 publications started appearing later than PO1 publications in the timeline [Chart 1].
Another option would be to combine the two search strategies. However, the drawbacks associated with the general search remain with such a search strategy. Even though the first search retrieved more articles, we have seen that it has missed some citations from JBJS Am as it retrieved only 35publications while the second search retrieved 36 citations from the journal JBJS Am.
We looked at the citation numbers received by publications by the leading authors on this topic in orthopedic journals. Of the total of 54 papers published by the top ten authors in Scopus, they received a total of 53 citations. On PubMed, we found 34 articles published by the top ten authors and analyzed in iCite. The total citations were 19 for 34 articles with a mean citation per pub of 0.56 and a maximum of four citations for a publication. Normally, it takes few years, on average 7–10 years to reach a peak in the number of citations received by a paper and very few citations are received in the first year. In this case, within the first 3 months, a total number of citations equal to the total number of publications by top authors are a notable finding.
Measuring interest and metrics in a particular branch of research for a new disease like COVID-19 are difficult for three reasons: (a) The rapid rate at which research evolves, (b) the scale at which the research publications are produced, and (c) the simultaneous branches of research that occur to understand and treat such a condition. Traditional ways of measuring performance metrics involve looking at and based on the citation counts for publications. While these may be good measures for slow evolving conditions such as elective or cold orthopedic conditions or procedures, they may not be ideal for rapidly changing diseases since they take time to accumulate.
With the availability of social media and electronic resources, research is disseminated at a rapid rate and altmetrics is used to measure the attention the research gets to give an altmetric attention score (AAS). This was developed to capture the attention a publication gets in the social media (Blogs, Facebook, Twitter, and Reddit), in addition to citations received. Among the current metrics available, AAS would be ideal for showing the immediate attention received by a publication. For a fast-evolving condition like COVID-19, where branches of research become unpredictable and fast-changing, even these metrics may have to be scrutinized on a monthly or even weekly basis to predict directions of research, analogous to the indices used to measure in financial markets where the second to second or minute by minute change is crucial in predicting trends.
The reach of social media sites has increased exponentially since its inception. The prominent sites among these include Twitter, Facebook, and Reddit. Metrics have been designed to assess the popularity of a topic on these sites. The most commonly used tool to improve and study these metrics is the hashtag in the social media. Altmetrics (or Alternate Metrics) are a new way of finding out the popularity of a publication in the social as well as the scientific circles. A score is given to each publication depending on popularity in the social media as well as citation counts obtained. This score has been named altmetric attention score (AAS).
We studied the altmetric scores for the same strategy used to search in PubMed (PO1) and presented the output based on AAS scores. The search field could not accommodate the strategy; we used to search in orthopedic journals (PO2). These scores may change from day to day depending on the interest gained or lost. One caveat about AAS is that not all attention gained is positive in all cases. Even negative attention could contribute to a high AAS. The dimensions website also provides options to view publications in decreasing order of Field Citation Ratio, Relative Citation Ration, as well as Citations of the publications produced by the search strategy.
The publication with maximum AAS score of 25226 has 840 citations, and it was published in the April 2020 issue of New England Journal of Medicine. Within 6 weeks, it has clocked 24 million followers on Twitter. Further studies are needed by tracking publications with such high AAS scores to see if they evolve to be the highest cited publications in the future. If that happens, AAS scores would be valuable in identifying quality publications much before peak citations are seen. However, for this to happen, one would need to see what the AAS scores of these publications were in the early stages. This study could act as a baseline for the assessment of these articles after some time to check on citation counts and altmetrics received.
Altmetrics give quick access to topics and branches of topics that are popular by way of the attention they receive in social media. The difference here from citation counts is that citations are given by the researchers and professionals of the same field of study. In contrast, AAS includes attention from everybody in the society. As researchers increasingly use social media for dissipation and sharing of ideas, major users could be researchers. Whereas with citation numbers, the attention gained by a publication is evaluated by other publications, in AAS, actions such as downloading an article, sharing and following of a study could increase its worth potentially giving important insight into the attention received by a publication from scientists as well as members of the society.
One disadvantage of too much reliance on altmetrics would be that to get higher AAS scores, manuscripts could cater to the general public and get less technical and conversely, manuscripts which are too technical, may not interest general public, and may not be shared as frequently in the social domain and hence could receive lower AAS scores than those which interest the general public.
As surgeons, we need good sources of information that give accurate and reliable information that can be used for our practice to benefit our patients. With evidence-based medicine forming a major component of today’s practice, sources of evidence become important. We endeavor to try and improve our sources to produce evidence, that is, accurate, reliable, reproducible, fast, and easy to access. There are multiple ways to search the literature and each method could produce a different outcome. With multiple search engines on the fray, it is important to know the strengths and weaknesses of each search engine to compose an effective search strategy. The best search strategy should produce an output that contains all the relevant citations that are recent, well-cited, important, and not leave out any article that is related to the search topic. It should also minimize citations that are not relevant to the topic that is being searched.
Bibliometric studies like ours offer several advantages [Table 8] as these are useful tools to measure the research impact quantitatively and hence become an objective way of analysis. It also helps in comparing the impact of a research more readily than with the peer review, which is subjective. Moreover, this type of analysis is transparent, and the results are reproducible, using the same methods. To conduct these studies is inexpensive and takes relatively lesser time, to produce and use.
|Quantitative way of measurement of research impact||Distinguish metrics between cited and not cited articles and not necessarily the best quality articles.|
|Research impact is readily comparable||An article may be cited a lot, but for negative reasons.|
|Transparent and reproducible procedure||Metrics can be misused and exploited by the researchers|
|Inexpensive and lesser time consuming||May prompt researchers to publish work which is likely to be cited more, rather than for its value or quality.|
|Scalable (Individual/Institutional/National/International)||Areas of study needs to be taken into consideration, as publications and citations are variable|
|Help in relation to the next REF (Research Excellence Framework)||Not reasonable to compare different areas of specialty like Arts with Medicine.|
Limitations of our study
We have discussed the advantages and drawbacks of PO1 and PO2 search strategies. We have only used the free version of dimensions which is available for public use. If a further analysis on of altmetrics is to be done, a subscription to the altmetrics or dimensions is required.
The publications on COVID-19 started from 9th week of 2020 and have increased exponentially, indicating proportional increase in research activity. Publications in specialist orthopedic journals began to appear later and had lower citation counts. There is a steady rise in the publication of protocols weekly on this topic. Reviews (PubMed) and direct articles (Scopus) were the most published and had been increasing recently. The JBJS (Am) and the J Arthroplasty have published the maximum number of articles on COVID-19 and provided free full-text articles, respectively. Publications from different countries focused on different aspects of the condition and its effects on orthopedics and trauma. We have listed clinically useful protocols from the search. This study should help in quantifying the value of research and publications related to orthopedics and trauma aspects of COVID-19 and therefore helps the readers, researchers, and health-care providers to use this information effectively.
Top five publications from PO1 search output had more citations than orthopedic specific search of PO2. An orthopedic journal-specific search may be useful for specialty-specific condition but for COVID-19, a combination search is useful. Future researchers should note this while designing search strategies. Although the COVID-19 related articles received quick citations in a short period, citations may not be ideal for rapidly changing diseases as they take time to accumulate. Publications with low citation counts could have immense social media attention. Altmetric scores may be useful to find branches of research gaining attention in the social and scientific circles. Standardization of names of journals between search engines would improve output from search strategies.
Declaration of patient consent
Patient’s consent not required as there are no patients in this study.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
- Library Research Support. 2020. London: The Open University; Available from: http://www.open.ac.uk/library-research-support/bibliometrics/pros-and-cons-bibliometrics [Last accessed on 2020 Jul 06]
- [Google Scholar]