Article Text
PDF
Abstract
Objective To systematically assess the diagnostic accuracy of rapid point-of-care tests for diagnosis of current SARS-CoV-2 infections in children under real-life conditions.
Design Systematic review and meta-analysis.
Data sources MEDLINE, Embase, Cochrane Database for Systematic Reviews, INAHTA HTA database, preprint servers (via Europe PMC), ClinicalTrials.gov, WHO ICTRP from 1 January 2020 to 7 May 2021; NICE Evidence Search, NICE Guidance, FIND Website from 1 January 2020 to 24 May 2021.
Review methods Diagnostic cross-sectional or cohort studies were eligible for inclusion if they had paediatric study participants and compared rapid point-of care tests for diagnosing current SARS-CoV-2 infections with reverse transcription polymerase chain reaction (RT-PCR) as the reference standard. The Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool was used to assess the risk of bias and the applicability of the included studies. Bivariate meta-analyses with random effects were performed. Variability was assessed by subgroup analyses.
Results 17 studies with a total of 6355 paediatric study participants were included. All studies compared antigen tests against RT-PCR. Overall, studies evaluated eight antigen tests from six different brands. Only one study was at low risk of bias. The pooled overall diagnostic sensitivity and specificity in paediatric populations was 64.2% (95% CI 57.4% to 70.5%) and 99.1% (95% CI 98.2% to 99.5%), respectively. In symptomatic children, the pooled diagnostic sensitivity was 71.8% (95% CI 63.6% to 78.8%) and the pooled diagnostic specificity was 98.7% (95% CI 96.6% to 99.5%). The pooled diagnostic sensitivity in asymptomatic children was 56.2% (95% CI 47.6% to 64.4%) and the pooled diagnostic specificity was 98.6% (95% CI 97.3% to 99.3%).
Conclusions The performance of current antigen tests in paediatric populations under real-life conditions varies broadly. Relevant data were only identified for very few antigen tests on the market, and the risk of bias was mostly unclear due to poor reporting. Additionally, the most common uses of these tests in children (eg, self-testing in schools or parents testing their toddlers before kindergarten) have not been addressed in clinical performance studies yet. The observed low diagnostic sensitivity may impact the planned purpose of the broad implementation of testing programmes.
PROSPERO registration number CRD42021236313.
- diagnosis
- pediatrics
- COVID-19
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. The data supporting the findings of this review are available within the article or its appendices The R code used for this review is available from the corresponding author (NFR) on request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Summary box
What is already known on this topic?
Antigen tests are widely used to detect children with current SARS-CoV-2 infection in schools and kindergarten despite an ongoing debate on potential benefits and harms.
Sensitivity estimates of antigen tests in adult populations vary broadly and are substantially lower than reported by manufacturers; however, test performance in paediatric populations remained unknown.
What this study adds?
A systematic literature search and comprehensive author queries allowed the inclusion of 17 studies evaluating the diagnostic accuracy of antigen tests in children.
Real-life performance of current antigen tests for professional use in paediatric populations is below the minimum performance criteria set by WHO, the United States Food and Drug Administration, or the Medicines and Healthcare products Regulatory Agency (UK).
Performance of antigen tests for professional use in paediatric populations is simillar to what has been reported previously for adult populations. No evidence on the performance of self-tests in children was identified.
Summary box
How might it impact clinical practice in the foreseeable future?
The observed low diagnostic sensitivity may impact the intended purpose of antigen tests in children.
Evidence gaps identified in this systematic review demonstrate current research needs to support evidence-based decision making. In particular, evidence is needed on the real-life performance of tests in schools (self-testing performed by children) and kindergarten (sample collection in toddlers by laypersons).
Introduction
Since the beginning of the COVID-19 pandemic caused by SARS-CoV-2, accurate, fast and early detection of people infected with SARS-CoV-2 followed by effective isolation measures of infected individuals has been considered a cornerstone in the global fight against the spread of SARS-CoV-2. Laboratory-based reverse transcription polymerase chain reaction (RT-PCR) testing is the standard for diagnosing current infections with SARS-CoV-2. However, limited testing capacities at many laboratories worldwide and limited availability of laboratories in developing countries has shown the urgent need for novel diagnostic tests that are easy to use, less expensive, widely available and suitable for point-of-care use. Today, such tests—and in particular antigen tests—are increasingly used to complement testing with RT-PCR to extend testing capacities or when a short turnaround time is essential.1 However, the advantages of antigen tests come at the price of lower diagnostic accuracy, most notably a lower diagnostic sensitivity, which increases the risk of missing cases, including those with pre-symptomatic infection who have yet to enter the most infectious period.2
Whether a lower sensitivity can be compensated by frequent testing remains a topic of controversial discussions.3–6 Additionally, the fact that sensitivity and specificity are not inherent test characteristics but are affected by various factors, including population characteristics, sample quality and study design, needs consideration.7 Data on diagnostic accuracy provided by antigen test manufacturers at market access are often overly optimistic and do not necessarily reflect the test’s performance in practice. Sensitivity of antigen tests in adult populations varies considerably across brands,8 with only a few tests meeting the minimum acceptable sensitivity of 80% or higher as defined by WHO or the United States Food and Drug Administration (US FDA).9 10
Because many countries are implementing public health safety measures that involve the use of antigen tests in adults and also in children, such as mass (self-)testing in schools,11 knowledge about how these tests perform in children is of high importance. However, to our knowledge, systematic reviews analysing the diagnostic test accuracy (DTA) of rapid tests in children are lacking. Therefore, in this systematic review and meta-analysis, we aimed to identify, assess and summarise the best available evidence on the real-life performance of rapid tests for diagnosing current SARS-CoV-2 infections in paediatric populations at the point of care.
Methods
The protocol for this systematic review was registered with PROSPERO (ID: CRD42021236313).12 The reporting adhered to the Preferred Reporting Items for Systematic reviews and Meta-Analyses of DTA studies (PRISMA-DTA) guideline13 and two relevant extensions ‘PRISMA-DTA for Abstracts’14 and ‘PRISMA-S for Reporting Literatures Searches in Systematic Reviews’.15
Eligibility criteria
We included diagnostic cross-sectional and cohort studies that evaluated the clinical performance of rapid point-of-care tests for detecting current SARS-CoV-2 infections against the reference standard in paediatric or mixed-age populations. Assessing analytical performance parameters such as the analytical sensitivity (limit of detection) or the analytical specificity (cross-reactivity) was not covered by the current review. Diagnostic case-control studies were excluded because they reflect the test’s performance under ideal conditions and, therefore, often overestimate the diagnostic accuracy.16 Moreover, studies evaluating serological tests were excluded because such tests are not suitable for the initial diagnosis of current SARS-CoV-2 infection.17 We considered a study as eligible if the study population comprised at least 10 paediatric study participants, each identified as positive or negative by the reference standard. In the absence of a true gold standard, laboratory-based RT-PCR alone or in combination with clinical findings or clinical follow-up was defined as the reference standard because it reflects the best available method for diagnosing individuals currently infected with SARS-CoV-2.7 Furthermore, we required reporting of data that allowed constructing a complete 2×2 contingency table. The full set of eligibility criteria is shown in online supplemental table S1 of Appendix 1. The decision rule for author queries is described in online supplemental appendix 2.
Supplemental material
Supplemental material
Information sources
We performed a comprehensive search for primary studies and secondary publications (systematic reviews and Health Technology Assessment (HTA) reports) in the following electronic bibliographic databases: MEDLINE (Ovid), Embase (Ovid), the Cochrane Library (Wiley), and preprint servers (Europe PMC) including medRxiv and bioRxiv (see Hamelers and Parkin18 for a full list of included preprint servers). Here, secondary publications were solely used as sources for potentially relevant studies. In addition, we searched two study registries (ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform (ICTRP)) for relevant clinical studies. Other information sources comprised the International HTA Database, the Foundation of Innovative Diagnostics (FIND) COVID-19 website, and the Evidence Search and Guidance websites of Britain’s National Institute for Health and Care Excellence (NICE).
Search strategy
In accordance with the Cochrane Handbook for DTA Reviews,19 the search strategy included concepts addressing the index test and the target condition. The development of the search strategy followed an objective approach that involved text-analytic procedures to identify candidate search terms based on the method described by Hausner and colleagues.20 Further details are available in online supplemental appendix 2. The last search in bibliographical databases and study registries was conducted on 7 May 2021. Other information sources were last searched on 24 May 2021. All search strategies are provided in online supplemental appendix 3.
Supplemental material
Study selection
The screening of literature retrieved from bibliographical databases involved a two-step screening procedure and was performed independently by two researchers using the web-based Trial Selection Database (webTSDB).21 In a first step, potentially eligible primary studies and secondary publications were identified from screening titles and abstracts of retrieved citations. In a second step, the full texts of these articles were obtained and evaluated. Publications that met the eligibility criteria were included. Any discrepancies were resolved by consensus between the two researchers before finalising each screening step. Reference lists of relevant systematic reviews and HTAs (independent of mentioning paediatric study participants) were manually screened to identify further relevant studies. For the screening of records from study registries, both screening steps were combined. Furthermore, documents identified through searching other information sources were screened for eligibility or information about potentially relevant studies.
Data collection
The individual steps of data collection and data extraction were performed by one researcher. All output was checked by a second researcher to ensure its validity and completeness. Any disagreements were resolved by consensus. See online supplemental appendix 2 for further details.
Quality assessment
We used the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool22 to evaluate the methodological quality and applicability of the included studies at the study level. The tool was tailored to our review by adding one signalling question and review-specific guidance was provided to facilitate judgments; see online supplemental appendix 4. The quality assessment of each included study was performed by one researcher. A second researcher verified all judgments. Any disagreements were resolved by consensus. The results were summarised in the text and visualised as a table and figure.
Supplemental material
Diagnostic accuracy measures and data synthesis
For each included study, diagnostic sensitivity, diagnostic specificity, positive predictive value (PPV) and negative predictive value (NPV) with corresponding 95% CIs were calculated based on the extracted 2×2 tables. Individual study participants were used as the unit of analysis throughout this work. If a study reported repeat testing of individuals only the initial test was included in our analyses. If a study evaluated more than one test in the same study population, we reported all test evaluations, but only one randomly chosen test was included in the meta-analyses to avoid the necessity to adjust for multiplicity.
The meta-analyses were based on recommendations provided in the methodological guideline ‘Meta-analysis of diagnostic test accuracy studies’ by the European Network for Health Technology Assessment (EUnetHTA)23; see online supplemental appendix 2 for further details.
Results
Study selection
Overall, 3011 records were retrieved from five bibliographical databases. The PRISMA flow diagram is shown in figure 1 and outlines the process of identifying relevant studies from different information sources. References that were excluded at the full-text level can be found in online supplemental appendix 5 with the reason for their exclusion. After removing 36 preprint records identified via MEDLINE and 680 duplicate records, 2295 records were screened for eligibility; 2078 records were excluded at the title/abstract level. Full-text publications of 217 records were retrieved for further assessment. Nine studies24–32 met all eligibility criteria for inclusion. Furthermore, 21 studies33–53 were identified as eligible for author queries to obtain study data on paediatric subgroups. The authors of nine studies did not respond to our request for data.33 34 39 40 42 44–46 49 In four cases,36 47 52 53 the authors reported that the required number of individuals who tested positive or negative by the reference standard was not reached, and in one case43 no data on age were recorded. Eventually, author queries led to the inclusion of eight further studies,35 37 38 41 48 50 51 54 resulting in a total of 17 relevant studies for this review (12 peer-reviewed journal articles and five preprints). The full list of included studies is reported in online supplemental table S3 of Appendix 1.
Supplemental material
PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) flow diagram showing the selection process of primary studies included in this systematic review and meta-analysis.
Furthermore, we screened 113 records identified from study registries and 323 records identified from other information sources. The search for studies in study registries allowed us to identify four planned or ongoing and four completed studies with no results posted, see online supplemental table S4 of Appendix 1 for further details. Information retrieval from other information sources included screening 18 records retrieved from the FIND website, 78 records from NICE Evidence Search, 28 records from NICE Guidance, and 23 records from reference lists of six systematic reviews8 55–59 identified via searching bibliographical databases. As a result, no additional study that met the inclusion criteria was identified.
Study characteristics
All 17 included studies (6355 paediatric study participants) evaluated the performance of antigen tests against the reference standard RT-PCR. The main study characteristics for each individual study are summarised in table 1, further details are reported in table 2 and online supplemental table S5 of Appendix 1. Fourteen studies evaluated the test performance in mixed-age populations (adults and children), including 24 to 928 paediatric study participants. Three studies with a sample size between 440 and 1620 individuals exclusively recruited children. In eight studies, the purpose of testing included diagnostic testing of individuals with symptoms suggestive of SARS-CoV-2 infection. Six studies reported the inclusion of individuals who were asymptomatic but were at increased risk of infection due to previous exposure to SARS-CoV-2. Here, ‘asymptomatic’ refers to any individual who is healthy, infected but pre-symptomatic or infected but without symptoms at the time of testing. Symptom status definitions were reported in nine of 16 studies that included individuals who were symptomatic. Individuals with at least one symptom (mostly self-reported) were considered symptomatic. Evaluating the performance of antigen tests in a screening setting (eg, community mass testing) was the main objective of six studies. Eight antigen tests (six lateral flow immunochromatographic assays and two fluorescent immunoassays) from six different brands were used in 18 test evaluations, whereas antigen tests by Abbott were most investigated (Panbio COVID-19 Ag Rapid Test n=6, BinaxNOW COVID-19 Ag Card n=5). In more than half of the test evaluations (n=11), nasopharyngeal samples were collected for the index test. Six test evaluations used anterior nasal specimens for the index test. In all studies, the reference standard was RT-PCR performed in a laboratory setting.
Characteristics of included studies. All studies aimed to identify individuals currently infected with SARS-CoV-2. The reference standard was RT-PCR performed in the laboratory setting (created by the authors)
Characteristics of the studies’ index test and reference standard. (created by the authors)
Risk of bias and applicability
The results of the quality assessment are summarised in online supplemental table S6 of Appendix 1 and figure 2. Quality among studies varied. Only one study was at low risk of bias in all four domains of the QUADAS-2 tool. For patient selection, more than half of the studies were at high (n=1) or unclear (n=12) risk of bias because inadequate exclusion of participants occurred, or it was not clear whether a consecutive or random sample was enrolled into the study. All but one study was judged as having an unclear risk of bias for the reference standard due to insufficient reporting of blinding. Risk of bias in the flow and timing domain was high in three studies due to more than 5% of missing outcome data.
QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2) risk of bias and applicability concerns. Graphical summary showing the review authors’ judgment about each domain as percentages across 18 test evaluations reported in 17 included studies.
Overall applicability concerns were high in three studies due to high concerns in either the patient selection or index test domain. Three studies were of low concern and the remaining 11 were rated unclear due to insufficient reporting in at least one domain.
Results of individual studies
RT-PCR positivity rate, diagnostic sensitivity, diagnostic specificity as well as PPV and NPV (and their 95% CIs) of individual studies based on data from 2×2 contingency tables for paediatric populations are reported in online supplemental table S7 and figure 3. The RT-PCR positivity rate, which corresponds to the SARS-CoV-2 prevalence in the sample population, varied between 4.1% and 50%, with a median of 14,5% over n=17 studies. The sensitivity and specificity ranged from 33.3% to 85.7% and 91.7% to 100%, respectively. PPV and NPV ranged from 60.0% to 98.7% and 73.3% to 98.9%, respectively.
Forest plot of sensitivity and specificity of antigen tests in entire paediatric study populations irrespective of symptoms. The point estimates of sensitivity and specificity from each study (identified by name of first author) are shown as squares; the corresponding 95% CIs are represented as horizontal lines. TP, true positive; FN, false negative; TN, true negative; FP, false positive.
For individual studies, separate analyses for subgroups based on symptom status are reported in online supplemental tables S8-S10 of Appendix 1 and figure 4. Here, populations were defined as symptomatic or asymptomatic if at least 80% of paediatric study participants were reported as being symptomatic or asymptomatic at the time of testing, respectively. Mixed populations refer to populations with no predominant symptom status. RT-PCR positivity rates of the primary analysis population and the different subgroups based on symptom status are presented in online supplemental figure S1 of Appendix 1. Two studies38 41 were performed in high-prevalence populations with RT-PCR positivity rates of 38.7% and 50.0%, respectively. The median RT-PCR positivity rate was 13.2% (n=10 test evaluations) in asymptomatic populations, 13.8% in mixed populations (n=3 test evaluations) and 25.7% in symptomatic populations (n=13 test evaluations). Thus, the median RT-PCR positivity rate in symptomatic study populations was about 12 percentage points higher than in asymptomatic study populations, indicating a slight trend in the RT-PCR positivity rate with respect to the proportion of symptomatic subjects.
Forest plot of sensitivity and specificity of antigen tests in (A) symptomatic, (B) asymptomatic and (C) mixed paediatric study populations. The point estimates of sensitivity and specificity from each study (identified by name of first author) are shown as squares; the corresponding 95% CIs are represented as horizontal lines. TP, true positive; FN, false negative; TN, true negative; FP, false positive.
Synthesis of results
In our primary meta-analyses, we used data from 17 studies evaluating the diagnostic accuracy of antigen tests in paediatric participants. Estimated pooled sensitivity and specificity were 64.2% (95% CI 57.4% to 70.5%) and 99.1% (95% CI 98.2% to 99.5%), respectively. While the estimates for the sensitivity revealed high heterogeneity and thus justified the application of the bivariate model with random effects, the estimates for the specificity were limited to a small range, as shown in figure 5. Consequently, the estimated summary receiver operating characteristic (SROC) curve cannot be meaningfully interpreted. As prespecified in the protocol, we performed subgroup analysis evaluating the diagnostic accuracy according to symptom status. Estimated pooled sensitivity and specificity in asymptomatic children was 56.2% (95% CI 47.6% to 64.4%) and 98.6% (95% CI 97.3% to 99.3%), respectively, based on data from 2439 asymptomatic children in 10 studies. Estimated pooled sensitivity and specificity in symptomatic children was 71.8% (95% CI 63.6% to 78.8%) and 98.7% (95% CI 96.6% to 99.5%), respectively, based on data from 3413 symptomatic children in 13 studies. Estimated pooled sensitivity and specificity in the mixed population of symptomatic and asymptomatic children from three studies including 419 children was 63.4% (95% CI 37.3% to 83.5%) and 98.7% (95% CI 90.8% to 99.8%), respectively. The corresponding SROC curves are shown in online supplemental figure S2 of Appendix 1. The likelihood ratio test (LRT) for differences between the three groups revealed a p-value of pLRT=0.066. Since the bivariate meta-analysis might be influenced by the differences in the prevalence,60 we performed a bivariate meta-regression taking the prevalences within the studies directly into account. The LRT between the models without and with prevalence revealed a statistically significant p-value of pLRT=0.003. Results for the other subgroup analyses did not show relevant differences in the pooled estimates (setting pLRT=0.400; index test sample type pLRT=0.303; reference standard sample type pLRT=0.723; RT-PCR cycle threshold (Ct) cut-off value pLRT=0.105; publication status pLRT=0.551). The prediction regions of these analyses also showed a higher heterogeneity for sensitivity compared with specificity, see online supplemental figures S3-S6 of Appendix 1. Due to insufficient data, we did not perform subgroup analysis with respect to test type (antigen vs molecular) and end-user (layperson (self-testing) vs trained staff/healthcare worker). Except for one study48 61 where the testing procedure involved supervised self-collection of samples by study participants, in all other studies, testing was conducted by trained staff and/or healthcare workers (if reported). Univariate meta-analysis with random effects for sensitivity and specificity in cases where only a few studies were included (mixed population of symptomatic and asymptomatic children) did not show remarkable differences to the bivariate analysis. The results of all bivariate meta-analyses are summarised in table 3.
Summary receiver operating characteristic (SROC) plot of sensitivity and specificity of antigen tests for diagnosis of current SARS-CoV-2 infections in entire paediatric study populations irrespective of symptoms. Each circle represents the point estimate of an individual study, whereas the size of the circle correlates with the number of paediatric study participants (small circle: 500 participants). The pooled estimate (black dot) of the pair of sensitivity (Se) and specificity (Sp) is surrounded by its 95% confidence region (closed curve with short dashes) and prediction region (closed curve with long dashes). The estimation of the SROC curve is based on the bivariate approach by Rutter and Gatsonis.77
Results of the bivariate meta-analyses: pooled sensitivity and specificity with 95% CI (created by the authors)
Discussion
To our knowledge, this is the first systematic review that focused on evaluating the diagnostic accuracy of rapid point-of-care tests for current SARS-CoV-2 infections in paediatric populations. Our review comprises 17 studies with 18 evaluations of eight different antigen tests in children, whereas comprehensive author queries allowed us to include eight studies that did not provide sufficient data on paediatric study participants in their original study publication. We did not identify any evaluations of molecular-based tests that met our inclusion criteria confirming the current dominant role of antigen tests for rapid point-of-care usage.
Sensitivity estimates of antigen tests varied broadly among studies and were substantially lower than reported by manufacturers. However, one should note that the intended use of most tests is limited to symptomatic individuals. Thus, performance data reported by manufacturers usually refer to symptomatic individuals only. Less variation and only minor discrepancies to performance claims by manufacturers were observed for specificity estimates across studies. Taking into account test-specific pooled results, no test included in this review fully satisfied the minimum performance requirements as recommended by WHO9 (minimum sensitivity ≥80% and minimum specificity ≥97%), the US FDA10 (minimum sensitivity ≥80%, whereas a lower bound of the two-sided 95% CI above 70% is required for over-the-counter use self-tests62) or the Medicines and Healthcare products Regulatory Agency (MHRA) in the UK63 (minimum acceptable sensitivity ≥80% with two-sided 95% CI entirely above 70% and minimum acceptable specificity of 95% with two-sided 95% CI entirely above 90%). Limited performance was also observed in a recent laboratory-based study that evaluated the sensitivity of 122 of these antigen tests using common SARS-CoV-2 specimens with varying viral concentrations.64 Even under such ideal conditions, a wide range of sensitivities was observed, whereas 26 tests missed the study’s sensitivity criteria of 75% for specimens with high SARS-CoV-2 concentrations of around 106 SARS-CoV-2 RNA/ml and higher corresponding to a Ct value less than 25.
The bivariate meta-regression with respect to prevalence was statistically significant. This result is mirrored in the results of the subgroup analysis with respect to symptom status. While specificities were similarly high in symptomatic (98.7% with 95% CI 96.6% to 99.5%) and asymptomatic (98.6% with 95% CI 97.3% to 99.3%) populations, we observed a drop in sensitivity by about 15 percentage points in asymptomatic populations (56.2% with 95% CI 47.6% to 64.4%) compared with symptomatic populations (71.8% with 95% CI 63.6% to 78.8%). The better performance in symptomatic populations might be explained by changes in the viral load over the course of infection and the timing of the test: most symptomatic individuals were tested within 7 days of symptom onset in contrast to individuals who were asymptomatic at the time of testing with more variable disease onset, including individuals in the early (pre-symptomatic) or late stages of infection when viral loads are relatively low.65
As expected, the sensitivity increased when the positivity threshold of the reference standard was set to a lower Ct cut-off value of 30 or 25. However, such analyses should not be over-interpreted because Ct values are not standardised across systems or laboratories, making it difficult to directly compare results between different studies. Furthermore, while the Ct value from RT-PCR is a strong indicator of viral load, there is no specific cut-off viral load which allows distinguishing individuals as being infectious or not. As shown in online supplemental tables S11 and S12 of Appendix 1, an increase in sensitivity comes at the cost of a decrease in specificity as antigen tests also identify some individuals with moderate or low viral loads, who would then be considered as false positives. Despite some methodological differences (such as the stringency of inclusion criteria) and neglecting differences between included studies (eg, settings), the findings of our review are similar to those in the recent Cochrane Review by Dinnes et al 8 or the now published living systematic review by Brümmer et al.66 These similarities between paediatric and adult populations might be explained by the findings by Jones et al,67 who only identified minor differences in viral loads across age groups in a comprehensive analysis of more than 25 000 individuals who tested positive for SARS-CoV-2 by RT-PCR.
It is widely recognised that RT-PCR is an imperfect reference standard for identifying current SARS-CoV-2 infection. However, based on the guidance provided by Reitsma et al,68 we assume that this does not play a pivotal part in the context of DTA studies of SARS-CoV-2 antigen tests because antigen-based testing does not outperform RT-PCR-based molecular testing in terms of diagnostic accuracy.2 The observed analytical variability between RT-PCR assays that may affect the false-negative rate is considered negligible as the analytical sensitivity (limit of detection) of RT-PCR assays is several magnitudes higher than the analytical sensitivity of antigen tests. Furthermore, RT-PCR-based testing in low prevalence settings confirms the very high specificity of the method in practice. Any pre-analytic issues such as the quality of specimen collection, which may affect the diagnostic accuracy of RT-PCR, also apply to antigen tests.
For the current version of our review, publication bias is not considered relevant due to the novelty of the topic. No study included in our review was published before November 2020. All four completed studies that were identified through searching study registries were completed within the last 9 months. Of note is that for all but one study26 included in our review no entry in a study registry was reported.
Despite the roll-out of vaccines, testing continues to be a key to pandemic control. Particularly in populations with low vaccination rates or waning immunity, early identification of outbreaks will remain vital for controlling the spread of SARS-CoV-2. Consequently, multi-layered mitigation strategies will continue to involve screening tests of children in schools and kindergarten to avoid further closures. Whether this would still apply in populations with high childhood vaccination rates remains an open point for discussion.
The high specificity of antigen tests and the corresponding PPVs calculated for the paediatric study populations suggest that antigen testing might be a valuable tool to rapidly identify children with SARS-CoV-2 infection in moderate to high prevalence settings. However, at the same time, it is important to raise awareness that antigen tests should not be used to rule out SARS-CoV-2 infection (or infectiousness) because of their limited sensitivity. Whether increasing the frequency of antigen-based testing leads to an improved overall diagnostic accuracy that allows to effectively reduce transmission of SARS-CoV-2 has yet to be demonstrated in practice.69 The latter two aspects and the urgent need for high-quality screening tests probably led to the recent publication of a new target product profile by the MHRA in the UK,70 which includes increased performance requirements for self-tests to be used in national testing programmes that aim at detecting current SARS-CoV-2 infections in individuals without symptoms. Here, the minimum acceptable sensitivity for tests to ‘rule out’ a current infection is at least 97% with two-sided 95% CI entirely above 95%. The minimum acceptable specificity is 99.5% or higher with two-sided 95% CI entirely above 97%. Furthermore, it is stated that performance claims of repeated testing strategies require adequate clinical evidence rather than evidence from modelling studies only.
Other screening testing methods such as molecular-based pool testing, which involves RT-PCR testing of pooled samples and so-called deconvolution testing of individuals belonging to pools tested positive, are currently under investigation71–73 and may complement mere antigen testing, in particular in low prevalence settings. Additionally, novel tests, for example, lateral flow tests based on clustered regularly interspaced short palindromic repeat (CRISPR), hold great promise for a highly sensitive direct detection of SARS-CoV-274 but have yet to gain market access and demonstrate their value.
Limitations
Because we limited our search to studies published in English or German, it is possible that relevant studies were missed. The chosen implementation of the bivariate approach required a continuity correction for studies with zero cells in 2×2 tables. This approach may introduce bias into the results if multiple small studies with zero cells are included. With regard to bias, one should keep in mind that most studies were at unclear risk of bias in at least three of the four domains of the QUADAS-2 tool because of poor reporting. We acknowledge that infectiousness as a target condition is of higher practical relevance than current SARS-CoV-2 infection, which was chosen as the target condition of this review. While RT-PCR as the corresponding reference standard is a highly sensitive method that is used to detect the presence of viral RNA in a specimen, this does not necessarily indicate that infectious virus is present. Therefore, the actual transmission risk from individuals who tested RT-PCR positive remains unknown. Testing for infectiousness would allow to identify (and isolate) exclusively individuals who could pass the virus to others. However, while there have been attempts to use viral load (estimated from Ct values) or virus viability in cell culture as a proxy to determine individuals who are infectious, up to now, there is no adequate reference standard for infectiousness.69 We included only eight different antigen tests in our review. Thus, with more than 500 antigen tests for professional use that gained market access in the EU,75 the performance of most antigen tests under real-life conditions remains unknown. Sample collection in toddlers by laypersons or self-testing in schools performed by children are likely to influence the real-life test performance but were not addressed in any of the studies included in our review. Furthermore, one should keep in mind that diagnostic accuracy is only one factor affecting the effectiveness of testing programmes.76 We emphasise that all included studies were performed before market authorisation of COVID-19 vaccines for paediatric populations and most individuals identified as RT-PCR positive were likely infected with the wild-type of SARS-CoV-2. Diagnostic accuracy estimates reported in this review may not apply to future variants of SARS-CoV-2 or children who are vaccinated.
Conclusion
The performance of current antigen tests in paediatric populations under real-life conditions varies broadly. Relevant data were only identified for very few antigen tests on the market, and the risk of bias was mostly unclear due to poor reporting. Estimates of sensitivity and specificity and their 95% CIs from the bivariate meta-analyses indicate a subpar real-life performance of current antigen tests in children below the minimum performance criteria set by WHO, the US FDA or the MHRA in the UK. This may affect the planned purpose of the broad implementation of testing programmes. Up to now, the most common uses of these tests in children (eg, self-testing in schools or parents testing their toddlers before kindergarten) have not been addressed in clinical performance studies. Thus, it is of high relevance that these use cases are promptly investigated in independent studies. Moreover, the implementation of routine audits of testing programmes may allow monitoring of test performance in practice outside of studies.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. The data supporting the findings of this review are available within the article or its appendices The R code used for this review is available from the corresponding author (NFR) on request.
Ethics statements
Patient consent for publication
Ethics approval
This study does not involve human participants.
Acknowledgments
We thank Dr Siw Waffenschmidt (IQWiG, Germany) for her valuable guidance during the development of the search strategies, the peer review of the final search strategy, and the literature screening technical support. Further, we are grateful to all study authors who replied to our author queries and who provided us with separate data on the paediatric study participants included in their studies. Finally, we thank the two anonymous peer reviewers and the editor for their critical reading of our manuscript and their valuable feedback.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors NFR: idea/conceptualisation, protocol, literature search (search strategies, screening), data extraction, quality assessment, data analysis, manuscript draft [lead author], manuscript review; LB: protocol, data extraction, quality assessment, data analysis, manuscript draft (contributor), manuscript review; YZ: protocol, literature search (screening), manuscript review; AV: supervision, protocol, manuscript review. All authors read and approved the final manuscript.The lead author (NFR) is the manuscript’s guarantor and affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned and registered have been explained. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.