Epistemonikos COVID-19 L·OVE can be used as a single source for evidence synthesis

COVID-19 is the topic on which the largest amount of scientific articles has been published in history. Therefore, the synthesis of relevant evidence to timely inform decision-making during the pandemic has become an unprecedented challenge.

 

The L·OVE (Living OVerview of Evidence) platform that contains evidence on COVID-19, developed by the Epistemonikos Foundation, can be used as a single source for research articles to conduct evidence synthesis. So it is reported on a recent publication in The Journal of Clinical Epidemiology ^[1], which concludes that both the comprehensiveness and the currency of the COVID-19 L·OVE repository is 100% for randomized trials.

Traditionally, evidence synthesis begins with the review of multiple bibliographic databases in order to select the articles that will be included. Through manual search, the publication can take on average 15 to 16 months ^[2], an unacceptable delay in the context of a pandemic when informed decisions need to be taken in the shortest time. In 2016, Epistemonikos Foundation developed L·OVE, a tool based on two interrelated components: a repository and a classification platform, that enables the acceleration of evidence synthesis without compromising the rigurosity of the process. The repository contains all the evidence that is relevant to specific health topics, which is assessed by two automated classifiers ^[1].

 

In response to the massive and disperse production of studies about COVID-19, Epistemonikos Foundation launched the COVID-19 L·OVE in June 2020. This tool has been used by the COVID-NMA initiative, the WHO living guideline “Therapeutics and COVID-19”, the COVID LNMA group from McMaster University (Canada), the Living Evidence Framework project coordinated by the Santa Creu i Sant Pau Hospital (Spain) and the PAHO dashboard of pharmacovigilance for COVID-19 vaccines, among other projects.

 

The first validation of the COVID-19 L·OVE repository ^[2] as a single source for the synthesis of COVID-19 studies demonstrated a very high comprehensiveness for different types of articles (randomized trials reports, randomized trials protocols and observational studies). According to the analysis of Pierre O. et al, the COVID-19 L·OVE can be used alone to identify studies assessing interventions to prevent or treat COVID-19 instead of searching in different primary data sources.

 

“Although it was validated by the COVID-NMA initiative and was later incorporated into their methodology as a source for randomized clinical trials, we verified the attributes of the repository component in a transparent manner”, stated Francisca Verdugo, Chief Research Innovation Officer at Epistemonikos Foundation and first author of the publication.

 

“So far, only two specialized databases of COVID-19 publications have been assessed, our COVID-19 L·OVE and the Cochrane COVID-19 Study Register”, Verdugo added. The comprehensiveness of the Cochrane COVID-19 Study Register was calculated to be 77.2% ^[3], whereas the COVID-19 L·OVE repository had perfect comprehensiveness for randomised trials and very high comprehensiveness for other types of studies such as randomized trials protocols and observational studies. Also, the coverage was very high for journal articles and perfect for preprints ^[2].

 

“Even though the extended use of the COVID-19 L·OVE repository is a good indicator, it was necessary to formally evaluate its performance. Our analysis confirmed that the COVID-19 L·OVE repository is highly comprehensive, with a perfect comprehensiveness for randomized clinical trials and and is a particularly important tool for supporting living evidence syntheses”, the Chief Research Innovation Officer commented. Comprehensiveness (or sensitivity was determined through a sampling method based on the relative recall method ^[4]. The sample was composed of all the primary studies included in systematic reviews published during April 2021. A total of 2132 articles were included in 83 systematic reviews; 82 of the articles were randomized trials. Comprehensiveness was then measured as the amount of primary studies in the repository at the time the systematic review was detected. All 82 randomized trials were found on the COVID-19 L·OVE, so the repository had a comprehensiveness of 100% for this type of study. Meanwhile, 2125 of the 2132 total articles were found on the COVID-19 L·OVE, resulting in a general comprehensiveness of 99.67% ^[1].

 

Regarding the methodology used to validate the repository, the team leader explained that “we chose systematic reviews because it is the type of evidence synthesis that conducts high quality searches in a rigorous and transparent way”. Systematic reviews for the sampling method were obtained from the Epistemonikos database ^[5], the world’s largest for health evidence synthesis. Using the Epistemonikos database and the COVID-19 L·OVE replaces the multiple searches in PubMed, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), ClinicalTrials.gov, WHO International Clinical Trials Registry Platform (ICTRP) and medRxiv, so it can significantly reduce the workload of evidence synthesis teams.

 

Currency of the repository was also assessed. It was calculated as the percentage of studies that were available in the COVID-19 L·OVE at the time of the review search in comparison to the total number of articles in the sample. The general currency was calculated to be 96.48%, but 100% for randomized trials ^[1].

 

“Our results confirm that the COVID-19 L·OVE can be used as a single source for the synthesis of different types of studies on COVID-19, thereby substantially reducing the time invested in the identification of primary studies at minimum risk of missing relevant references”, Verdugo concluded.

References:

[1] Verdugo-Paiva F, Vergara C, Ávila C, Castro J, Cid J, Contreras V, Jara I, Jiménez V, Lee MH, Muñoz M, Rojas-Gómez AM, Rosón-Rodríguez P, Serrano-Arévalo K, Silva-Ruz I, Vásquez-Laval J, Zambrano-Achig P, Zavadzki G, Rada G. COVID-19 L·OVE repository is highly comprehensive and can be used as a single source for COVID-19 studies. J Clin Epidemiol. 2022 May 18:S0895-4356(22)00117-2. doi: 10.1016/j.jclinepi.2022.05.001.

[2] Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017 Feb 27;7(2):e012545. doi: 10.1136/bmjopen-2016-012545.

[3] Pierre O, Riveros C, Charpy S, Boutron I. Secondary electronic sources demonstrated very good sensitivity for identifying studies evaluating interventions for COVID-19. J Clin Epidemiol. 2022 Jan;141:46-53. doi: 10.1016/j.jclinepi.2021.09.022.

[4] Metzendorf MI, Featherstone RM. Evaluation of the comprehensiveness, accuracy and currency of the Cochrane COVID-19 Study Register for supporting rapid evidence synthesis production. Res Synth Methods. 2021 Jun 5:10.1002/jrsm.1501. doi: 10.1002/jrsm.1501.

[5] Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006 Jul 18;6:33. doi: 10.1186/1471-2288-6-33.

[6] Rada G, Pérez D, Araya-Quintanilla F, Ávila C, Bravo-Soto G, Bravo-Jeria R, et al. Epistemonikos: a comprehensive database of systematic reviews for health decision-making. BMC Med Res Methodol. 2020 Nov 30;20(1):286. doi: 10.1186/s12874-020-01157-x.