Article Text
PDF
Abstract
Objectives To compare the efficacy of influenza vaccines of any valency for adults 60 years and older.
Design and setting Systematic review with network meta-analysis (NMA) of randomised controlled trials (RCTs). MEDLINE, EMBASE, JBI Evidence-Based Practice (EBP) Database, PsycINFO, and Cochrane Evidence -Based Medicine database were searched from inception to 20 June 20, 2022. Two reviewers screened, abstracted, and appraised articles (Cochrane Risk of Bias (ROB) 2.0 tool) independently. We assessed certainty of findings using Confidence in Network Meta-Analysis and Grading of Recommendations, Assessment, Development and Evaluations approaches. We performed random-effects meta-analysis and network meta-analysis (NMA), and estimated odds ratios (ORs) for dichotomous outcomes and incidence rate ratios (IRRs) for count outcomes along with their corresponding 95% confidence intervals (CIs) and prediction intervals.
Participants Older adults (≥60 years old) receiving an influenza vaccine licensed in Canada or the USA (vs placebo, no vaccine, or any other licensed vaccine), at any dose.
Main outcome measures Laboratory-confirmed influenza (LCI) and influenza-like illness (ILI). Secondary outcomes were the number of vascular adverse events, hospitalisation for acute respiratory infection (ARI) and ILI, inpatient hospitalisation, emergency room (ER) visit for ILI, outpatient visit, and mortality, among others.
Results We included 41 RCTs and 15 companion reports comprising 8 vaccine types and 206 032 participants. Vaccines may prevent LCI compared with placebo, with high-dose trivalent inactivated influenza vaccine (IIV3-HD) (NMA: 9 RCTs, 52 202 participants, OR 0.23, 95% confidence interval (CI) (0.11 to 0.51), low certainty of evidence) and recombinant influenza vaccine (RIV) (OR 0.25, 95%CI (0.08 to 0.73), low certainty of evidence) among the most efficacious vaccines. Standard dose trivalent IIV3 (IIV3-SD) may prevent ILI compared with placebo, but the result was imprecise (meta-analysis: 2 RCTs, 854 participants, OR 0.39, 95%CI (0.15 to 1.02), low certainty of evidence). Any HD was associated with prevention of ILI compared with placebo (NMA: 9 RCTs, 65 658 participants, OR 0.38, 95%CI (0.15 to 0.93)). Adjuvanted quadrivalent IIV (IIV4-Adj) may be associated with the least vascular adverse events, but the results were very uncertain (NMA: eight 8 RCTs, 57 677 participants, IRR 0.18, 95%CI (0.07 to 0.43), very low certainty of evidence). RIV on all-cause mortality may be comparable to placebo (NMA: 20 RCTs, 140 577 participants, OR 1.01, 95%CI (0.23 to 4.49), low certainty of evidence).
Conclusions This systematic review demonstrated efficacy associated with IIV3-HD and RIV vaccines in protecting older persons against LCI. RIV vaccine may reduce all-cause mortality when compared with other vaccines, but the evidence is uncertain. Differences in efficacy between influenza vaccines remain uncertain with very low to moderate certainty of evidence.
PROSPERO registration number CRD42020177357.
- immunization
Data availability statement
Data are available on reasonable request. The full dataset and statistical code are available at https://osf.io/eqb9h/.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Seasonal influenza vaccination of older adults (≥60 years old) is an important societal, cost-effective means of reducing morbidity and mortality.
A multitude of licensed seasonal influenza vaccines for older adults are available in a variety of formulations (such as IIV3, IIV4; prepared in standard and high doses; with and without an adjuvant) relying on production methods including those based on embryonated chicken eggs, or mammalian cell cultures and comprising seasonally selected viral strains or recombinant constructs.
Lack of high-quality analysis of randomised controlled trial (RCT) data pertaining to influenza vaccine production and composition poses challenges for public health clinicians and policy-makers who are tasked with making evidence-based decisions regarding recommendations about choosing optimally efficacious and safe influenza vaccines for older adults.
WHAT THIS STUDY ADDS
This systematic review and network meta-analysis of RCT data found that recombinant influenza vaccines were among the most effective (lowest odds of laboratory-confirmed influenza) and safest (lowest odds of all-cause mortality) of any licensed influenza vaccine type administered to older adults.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Our review points to a potential safety concern regarding increased odds of all-cause mortality associated with older adults receiving adjuvanted influenza vaccines (IIV3-adj and IIV4-adj).
Introduction
Seasonal influenza is a leading cause of infectious respiratory disease globally causing 3–5 million cases of severe illness and 650 000 deaths annually.1 In Canada, influenza ranks among the top 10 causes of mortality, with approximately 3500 deaths and 12 200 influenza hospitalisations per year.2 Annually, influenza results in an economic burden on average of US$14 000–US$20 000 per laboratory-confirmed influenza (LCI) hospitalisation.3
Older adults, particularly those who are frail bare a disproportionate burden of influenza and its complications. Globally, individuals 75 years of age and older experience the highest rates of influenza-associated respiratory deaths (51.3–99.4 deaths per 100 000 vs 13.3–27.8 in those 60–74 years of age and 1.0–5.1 in those less than 60 years of age).4 Annual influenza vaccination is the best way to prevent influenza infection and its complications.5 6
Although there are various influenza vaccines available for older adults, all current vaccines contain two influenza A virus strains and either one (trivalent) or two (quadrivalent) influenza B virus strains.7 Existing randomised controlled trials (RCTs) focus on the safety and efficacy of individual vaccines. The lack of direct comparative vaccine efficacy (VE) evidence poses challenges to public health clinicians and policy-makers to make evidence-based decisions regarding the preferential use of one influenza vaccine over another in older adults regarding vaccine choice. We aimed to systematically assess the comparative efficacy of influenza vaccines (eg, trivalent and quadrivalent at standard-dose (SD), high-dose (HD) or adjuvanted (Adj)) in adults 60 years of age and older. A systematic review with network meta-analysis (NMA) was conducted to fill this critical knowledge gap in comparative VE.
Methods
The protocol was registered in PROSPERO (CRD42020177357), and any deviations are reported in online supplemental appendix 1 (additional file 1). In Canada, national immunisation recommendations of vaccine use are made by the National Advisory Committee on Immunisation (NACI). Our protocol was revised in collaboration with the NACI Influenza Working Group, to improve the feasibility and relevancy of this review for key knowledge users. Below, we briefly describe the methods used in our review; more details are presented in online supplemental appendix 2 (additional file 1). This systematic review follows PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) extension to NMA8 and Guidance for Reporting Involvement of Patients and the Public (GRIPP-2) reporting guidelines9 (online supplemental additional files 2 and 3), as well as the methods outline in the Cochrane Handbook for Systematic Reviews of Interventions.10
Supplemental material
Supplemental material
Knowledge user engagement
We enhanced systematic review process by employing an integrated knowledge translation approach11 from project onset via established partnerships with the NACI Influenza Working Group, which will use these research results to inform the development of guidance for influenza vaccination. Our systematic review included two clinicians (MM and SS), one patient partner (KE), the NACI Secretariat (ASinilaite, PD-P, AG, WS, NM and AStevens) and seven researchers (AAV, MK, IDF, JJY-N, BH, LM and ACT). Throughout the research process, all knowledge users actively participated by contributing their insights in various stages, such as developing and refining the research question, reviewing the search parameters, framing the risk groups, prioritising the outcome measures and interpreting the study findings.
Data sources and searches
We searched Ovid MEDLINE, JBI (formerly the Joanna Briggs Institute) Evidence-Based Practice Database, Embase, PsycINFO and Cochrane Evidence-Based Medicine database from inception to 31 March 2022, with an updated search on 20 June 2022. An expert librarian developed the search strategies with input from the research team. Another expert librarian peer-reviewed the search strategy using the Peer Review of Electronic Search Strategies checklist (see online supplemental appendix 3 (additional file 1) for literature search).12 Grey literature (ie, unpublished or difficult-to-locate studies) was searched using the Canadian Agency for Drugs and Technologies in Health guide (see online supplemental appendix 4 (additional file 1)).13 We consulted content experts from the NACI Influenza Working Group, and scanned the NACI Practices review,14 the National Immunisation Technical Advisory Group websites (eg, SAGE, JCVI, ATAGI, STIKO (https://www.nitag-resource.org), reference lists of included studies and related reviews searching for additional relevant studies (online supplemental appendix 4 (additional file 1)). A library technician searched for article corrections and retractions to inform our retrieval of full-text reports.
Eligibility criteria
Below, we formulate the Population, Intervention, Comparator, Outcome, Study design criteria for our review.
Population
We included RCTs evaluating adults 60 years of age and older. Studies involving the general population that reported stratified data and results for older adults were also included. If stratified data and results were not reported, authors were contacted to retrieve any additional information with up to three reminders. RCTs with wider population characteristics reporting on relevant outcomes of interest in subsets of participants, and including a subcategory of age 60 years and older were also eligible.
Intervention
RCTs comparing any influenza vaccine at any dose that was licensed in Canada or the USA, at time of the review and irrespective of commercial name. Eligible influenza vaccines included but were not limited to egg based, quadrivalent inactivated influenza vaccine (IIV4) at SD or HD or Adj, trivalent IIV (IIV3) at SD or HD or Adj, mammalian cell culture-based trivalent or quadrivalent IIV, and quadrivalent or trivalent recombinant influenza vaccine (RIV4 or RIV3). We determined vaccine licence at time of the study based on the details provided by the studies and/or trial protocols. Intramuscular and subcutaneous routes of administration of relevant licensed vaccines were considered for inclusion.
We excluded intradermal, intranasal and AS03-adjuvanted vaccines, which are not licensed for use in Canada or the USA. Pandemic vaccines were excluded along with experimental vaccines that have not been approved for use.
Comparator
Another influenza vaccine, placebo or any other licensed vaccine was eligible.
Outcomes
Eligible outcomes were determined in collaboration with the NACI Influenza Working Group and clinician experts based on the GRADE (Grading of Recommendations, Assessment, Development and Evaluations)15 methodology for rating the importance of outcomes for decision-making.16 The primary outcomes were LCI and influenza-like illness (ILI). Secondary outcomes were the number of vascular adverse events, hospitalisation for acute respiratory infection (ARI) and ILI, inpatient hospitalisation, emergency room (ER) visit for ILI, outpatient visit and mortality. All outcomes included binary data apart from vascular adverse events that included count data. Other outcomes of interest included ARI cases, LCI hospitalisation, LC-ARI cases, ER visit for ARI or LCI or pneumonia or any cause, hospitalisation for pneumonia due to ILI, ARI, LCI, inpatient or outpatient hospital visit, hospitalisation due to cardiovascular events, LCI-related mortality, LCI-related healthcare interactions, number of hospitalisations due to cardiovascular events, number of participants with vascular adverse events and influenza-related vascular adverse events.
Study design
We included RCTs regardless of publication status, publication date, duration of follow-up, language of publication, geographical region or setting (eg, community, residential facilities, nursing homes).
Selection process and data extraction
The study selection, data extraction and certainty of evidence processes are described in online supplemental appendix 2.
We included both RCTs and cluster RCTs in the review. Following the Cochrane recommendations,17 for cluster RCTs, we initially planned to abstract the effective sample size and/or intraclass correlation. However, since this was not available in the included studies,18–20 we used the adjusted effect estimates considering the cluster design as reported in the publications. Arm-level data were abstracted from multiarm RCTs as reported in the publications, and analysed in an NMA accounting for correlation between intervention arms that share a common comparator.
Within and across study bias assessment
Once a pilot-test was carried out for risk of bias (ROB) assessment on a random sample of five studies, two reviewers (JD, PK, VN and RR) independently and in duplicate appraised the included RCTs using the Cochrane ROB 2.0 tool.21 Conflicts were resolved by a third reviewer (MK). The ROB was rated as low, high or some concerns in each of the following domains:
Randomisation process.
Deviations from the intended interventions.
Missing outcome data.
Measurement of the outcome selection of the reported results.
Timing of identification or recruitment of participants (cluster RCTs only, as per ROB 2.0 extension to cluster RCTs).
We used relevant signalling questions within each domain, as suggested by ROB 2.0 tool, and responded as yes, probably yes, probably no, no or no information, which helped reach overall ROB judgements (https://www.riskofbias.info/).22 ROB was assessed for each outcome included in a meta-analysis separately (completed for primary and secondary outcomes on 7 April 2023 and 15 May 2023, respectively), and based on the reported study information (following the principles of intention to treat, where available).
We visually inspected small-study effects and reporting bias using the comparison-adjusted funnel plot when at least ten studies were available per outcome.23 For this analysis, the interventions were ordered chronologically24 according to when they were licensed in Canada (or USA where applicable).
Synthesis
Study-level information on participant and study characteristics was synthesised descriptively.
Pairwise meta-analyses
We synthesised data in a meta-analysis using the OR for dichotomous outcomes and the incidence rate ratio (IRR) for count outcomes, along with corresponding 95% CIs when at least two studies were available. We performed a random-effects inverse-variance meta-analysis since we expected the studies to be methodologically and clinically different (eg, vaccine dose, participant age). We estimated the overall effect size and its 95% CI using the Hartung-Knapp-Sidik-Jonkman (HKSJ) method for meta-analyses with few studies25–27: specifically, when the estimated heterogeneity was positive (>0) among at least three studies in the meta-analysis, we used the HKSJ approach,28 29 otherwise we applied the Wald-type approach.25–27 30–32 We supplemented estimated overall effects and 95% CIs with prediction intervals (PIs) to capture the interval within which we expected the true intervention effect of a new study to fall. We used the restricted maximum likelihood method33 to estimate the between-study variance (τ2) and the Q-profile approach to calculate its 95% CI.34 35 We also calculated the percentage of variation that can be attributed to heterogeneity rather than chance using the I2 statistic.36
Network meta-analyses
For each outcome, we explored connectivity of evidence and network geometry graphically through network diagrams and summarising interventions, RCTs and intervention comparisons. Node formulation, including lumping or splitting of individual interventions, was decided based on knowledge user expertise, including the clinical, patient and policy-makers with the NACI Secretariat perspectives, and in accordance to our research question. We considered that different decisions on lumping or splitting interventions may impact the validity of our NMA results, and assessed relevant assumptions in all different network geometries. We conducted a frequentist random-effects NMA for each connected network diagram, when the number of studies exceeded the number of intervention nodes. Alternatively, pairwise meta-analysis was conducted for each intervention comparison separately. A common within-network τ2 was assumed for each network, which was a clinically reasonable assumption since networks were composed of influenza-type vaccines; τ2 was estimated using the restricted maximum likelihood method.33 We estimated summary ORs or IRRs and presented them along with 95% CIs and PIs. Interventions were ranked according to their efficacy using the P-score statistic.37
We conducted an NMA under the transitivity assumption, where trial participants are equally likely to be randomised to any of the network interventions and the included trials are sufficiently similar across intervention comparisons with respect to the distribution of effect modifiers. We visually assessed transitivity across intervention comparisons on the a priori selected effect modifiers: overall ROB, participant age, percentage of female participants and year of publication, using mean for continuous and mode for discrete variables. The statistical manifestation of intransitivity can cause inconsistency in the network of interventions. We assessed for consistency using both local (node-splitting)38 and global (design-by-treatment interaction)39 approaches. We planned to assess transitivity for populations with comorbid conditions, frailty and previous influenza vaccination, but these were rarely reported in the included studies.
Exploring the impact of effect modifiers
For the primary outcomes and outcomes displaying evidence of inconsistency, intransitivity or heterogeneity in any NMA, we performed subgroup and sensitivity analyses considering the predefined potential treatment effect modifiers and monitored changes in heterogeneity and effect estimates, whenever possible. In our primary outcomes, we also compared match and mismatch between the circulating and vaccine strains. To determine matched and mismatched studies, we followed previously published methods.40 We categorised a study as matched when the influenza strains were antigenically similar to the vaccine stains. Similarly, we categorised a study as mismatched when the influenza strains were antigenically distinct from vaccine strains. When the information on matching was not reported in the study, we determined the matching using surveillance data (online supplemental appendices 5–7 (additional file 1)). We also combined intervention nodes for placebo and for trivalent and quadrivalent formulation by vaccine type (ie, adjuvant, SD, HD and RIV). At the request of the NACI Influenza Working Group, we excluded tetanus–diphtheria–pertussis vaccine (Tdap) in a sensitivity analysis, where Tdap was included as an intervention. We were unable to explore dose effects in a dose-response NMA due to network sparsity.41 Similarly, we were unable to explore vaccine formulation, frequency, route of administration, platform, type and dose of adjuvant, and licensing product names due to network sparsity or due to incomplete reporting in individual studies which also increased network sparsity further.
Presentation of results
Summary ORs/IRRs for all pairwise comparisons, along with their 95% CIs and 95% PIs, were presented in tables. We also transformed ORs to risk ratios (RRs) and calculated the VE for all pairwise comparisons in the LCI and ILI outcomes using the combined mean baseline risk per outcome. P-scores across interventions and outcomes were presented in a rank-heat plot (https://rankheatplot.com/rankheatplot/).33 Statistical analyses were conducted using the meta 42 and netmeta 43 packages in R.
Assessment of the confidence in the evidence
We assessed confidence in NMA estimates for each primary NMA outcome, using Confidence in Network Meta-Analysis (ie, LCI, number of vascular adverse events, outpatient visits and all-cause mortality outcomes).44 Also, we assessed confidence in pairwise meta-analysis estimates for which an NMA could not be performed using the GRADE approach (ie, in IIV3-HD vs IIV3-SD for ILI, hospitalisation due to ARI, ER visit due to ILI, hospitalisation due to ILI, inpatient hospitalisation, and in IIV3-SD vs placebo for ILI).45 More details are presented in online supplemental appendix 2.
Supplemental material
Supplemental material
Results
Overall, 41 studies (206 032 participants) and 15 companion reports were included,18–20 46–96 after screening 6858 citations and 3425 full-text articles (figure 1). All of the included studies were written in English. Two71 97 of the seven contacted authors71 97–102 responded to our emails asking for additional information regarding study arms (eg, details about the vaccine used in the RCT) and outcomes (eg, outcome data stratified by age or study arm), and one71 provided outcome data stratified by study arm for analysis. Of the 41 RCTs, 26 were included in the quantitative analysis; the remaining 15 RCTs did not report relevant data for the analysis (eg, study size but no event data were available)20 52–54 61 64 66 68 70 73–75 77–79 or compared the same intervention node (eg, Agriflu vs Fluvirin103 both of which are trivalent, inactivated, egg-based influenza vaccines) or were disconnected from the NMA and did not contribute to any pairwise meta-analyses (online supplemental appendix 8). Lists of the studies included in the review and studies excluded after data abstraction can be found in online supplemental appendices 9 and 10 (additional file 1), respectively.
PRISMA flow diagram for identification of eligible studies. JBI, Joanna Briggs Institute; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analyses,
Study, patient and intervention characteristics
The included RCTs were published between 1994 and 2021 with most (36%) published between 2016 and 2020 (table 1). Most of the studies were conducted in the USA (N=19, 46%) and funded by industry (N=23, 56%). Three were cluster-RCTs.18–20 Most RCTs (N=9, 22%) compared IIV3-SD vs IIV3-HD, followed by RCTs comparing IIV3-SD vs placebo or vs IIV3-Adj (N=4, 10%) (table 1, online supplemental appendix 11 (additional file 1). The duration of follow-up and timing of assessment was reported in 24 studies and ranged between 3 months and 1 year.
The reported participant age within most RCTs (n=34, 83%) was ≥65 years; participants reflected racial diversity (n=19, 46%; table 2, online supplemental appendix 12 (additional file 1).
Study characteristics of included studies (N=41)
Participant characteristics of included studies (N=41)
Overall, eight influenza vaccine types were identified across all RCT interventions: IIV3-Adj, IIV4-Adj, IIV3-HD, IIV4-HD, IIV3-SD, IIV4-SD, RIV3 and RIV4. Most influenza vaccines were egg-based (n=47, 46%), and at SD (n=64, 63%). The delay between doses, which, if reported and/or applicable, ranged from 3 to 4 weeks (table 3, online supplemental appendix 13 (additional file 1)). In our analysis, and as recommended by our knowledge users, RIV3 was combined with RIV4 in a single node. Also, the individual intervention groups (IIV3-Adj, IIV4-Adj, IIV3-HD, IIV4-HD, IIV3-SD, IIV4-SD, RIV) assessed in our analysis did not consider multiple doses, modalities or types of adjuvants.
Vaccine characteristics of included studies (N=41)
Of the a priori selected outcomes, 17 were omitted from both pairwise or NMA. This is because in 10 of these outcomes, there was a lack of relevant data within the identified studies. In two outcomes only one study reported relevant data, while for five outcomes, there were multiple studies available, but one study was included per intervention comparison. Of the outcomes analysed in pairwise or NMA, an additional seven outcomes had studies which were not included in the analyses, because intervention comparisons included a single study or were disconnected from the underlying network. See online supplemental appendix 8 (additional file 1) for more details.
Within-study ROB and across-study reporting bias
Within-study bias appraisal suggested that overall low ROB was present for: 2 (22%) RCTs reporting LCI, 9 (45%) RCTs reporting all-cause mortality, 4 (50%) RCTs reporting vascular adverse events, 3 (33%) RCTs reporting ILI and 1 RCT (50%) reporting ER-visits (figure 2, online supplemental appendix 14 (additional file 1)). Overall, ROB for outpatient visits, hospitalisation for ARI and ILI, and inpatient hospitalisations outcomes were judged to be at some or high concerns (online supplemental appendix 15 (additional file 1)).
Cochrane Risk of Bias 2 summary results of primary outcomes (N=26 RCTs). LCI (n=9). ILI (n=9). RCTs, randomised controlled trials.
Reporting bias across studies could be assessed only for the all-cause mortality outcome, due to the small number of studies per outcome. The comparison-adjusted funnel indicated no evidence of publication bias or small-study effects (online supplemental appendix 16 (additional file 1)).
Results of testing assumptions
Assessment of the consistency assumption was possible for LCI and all-cause mortality outcomes (with available closed loops of evidence), where there was no evidence of statistical inconsistency (design-by-treatment interaction model LCI: χ2=2.35, 2 df, p=0.31; all-cause mortality: χ2=0.23, 3 df, p=0.98; online supplemental appendix 17A (additional file 1)). Overall, there were no important concerns regarding average age or ROB, but the mean percentage of female participants varied for LCI (range across comparisons: 32%–59%), outpatient visits (range across comparisons: 26%–55%) and all-cause mortality (range across comparisons: 26%–63%) outcomes, which may challenge the reliability of the transitivity assumption (online supplemental appendix 17B (additional file 1)).
Results of syntheses
Below, we present the analysis results for NMA and pairwise meta-analysis for the included vaccines against placebo or against IIV3-SD if a placebo node was not available for the outcome. Additional results and analyses are presented in online supplemental appendix 18 (additional file 1). The network geometry for each outcome analysed in an NMA is presented in figure 3.
Network-plots of (A) LCI, (B) number of vascular AEs, (C) outpatient visits, (D) all-cause mortality. Adj, adjuvanted; AEs, adverse events; HD, high dosage; IIV3, inactivated influenza vaccine trivalent; IIV4, IIV quadrivalent; LCI, laboratory-confirmed influenza; RIV, recombinant influenza vaccine; SD, standard dosage; Tdap, tetanus–diphtheria–pertussis.
Laboratory-confirmed influenza
The NMA for LCI included 9 RCTs,48 55–57 62 63 69 76 96 6 interventions (IIV3-SD, IIV3-HD, IIV3-Adj, IIV4-SD, RIV, placebo) and 52 202 participants (see online supplemental appendix 19 (additional file 1) for study definitions). The network geometry was informed by eight intervention comparisons with a range of 1–3 RCTs. Results suggested that vaccines may reduce incidence of LCI compared with placebo, but with varying degrees of precision of effect estimates, although with no appreciable between-study network heterogeneity (I2=0%, τ=0.00). IIV3-HD (OR 0.23, 95% CI (0.11 to 0.51), P-score=0.78; low certainty of evidence, VE 72.85 95% CI (43.5 to 86.64)) and RIV (OR 0.25, 95% CI (0.08 to 0.73), P-score=0.73; low certainty of evidence, VE 70.63 95% CI (22.86 to 90.21)) were the most effective vaccines. These were followed by IIV3-Adj vaccine (OR 0.24, 95% CI (0.04 to 1.47), P-score=0.65; low certainty of evidence, VE 71.73 95% CI (−34.45 to 95.06)) (table 4, online supplemental appendices 17C and 20 (additional file 1)).
Summary of results in outcomes with pairwise and network meta-analysis
Results were imprecise in sensitivity analyses restricting to studies with percentage of female above 50% and matched vs mismatched circulating strains (online supplemental appendix 18C (additional file 1)). When formulations were combined (ie, Adj, SD, HD, RIV), RIV performed best (OR 0.22, 95% CI (0.10 to 0.49), P-score=0.78, 95% PI (0.08, 0.63), I2=0%, τ=0.00). Precise results were identified for SD (OR 0.31, 95% CI (0.15 to 0.67), P-score=0.33, 95% PI (0.12 to 0.85)) and HD (OR 0.23, 95% CI (0.11 to 0.50), P-score=0.71, 95% PI (0.08 to 0.64)).
Influenza-like illness
NMA with original coding of interventions was not possible for ILI. Pairwise meta-analysis suggested that IIV3-SD may reduce ILI incidence compared with placebo, but the result was imprecise (2 RCTs,46 96 854 participants; OR 0.39, 95% CI (0.15 to 1.02), I2=8%, τ=0.21 low certainty of evidence, VE 57.79 95% CI (−1.75 to 83.22)) (table 5, online supplemental appendices 21–23 (additional file 1)) for study outcome definitions).
Summary of results in outcomes with pairwise meta-analysis
In a sensitivity analysis, we combined available vaccine formulations from IIV3-Adj, IIV4-Adj, IIV3-HD, RIV, IIV3-SD, IIV4-SD, Tdap and placebo to Adj, HD, RIV, SD, Tdap and placebo. In this case NMA was possible, including 9 RCTs,46 49 55–57 60 62 76 96 5 interventions (Adj, HD, RIV, SD and Tdap) compared with placebo, and 65 658 participants. HD performed best, yet the 95% PI suggested that new evidence may change results (OR 0.38, 95% CI (0.15 to 0.93), P-score=0.77, 95% PI (0.09 to 1.63), I2=0%, τ=0.00). Similar precision of the effect estimate was identified for RIV (OR 0.38, 95% CI (0.15 to 0.94), P-score=0.68, 95% PI (0.09 to 1.66)) and SD (OR 0.38, 95% CI (0.16 to 0.95), P-score=0.57, 95% PI (0.09 to 1.67)), yet PIs were wide (online supplemental appendix 17C (additional file 1)).
Secondary outcomes analysed in both network and pairwise meta-analysis
Number of vascular adverse events
The NMA for the number of vascular adverse events included 8 RCTs,55 56 58 59 71 78 94 5 interventions (IIV3-SD, IIV3-HD, IIV3-Adj, IIV4-HD, IIV4-Adj), and 57 677 participants (see Appendices 17, 18, and 24 (online supplemental appendices 17, 18, and 24 (additional file 1)) for study outcome definitions). The network geometry was informed by four intervention comparisons with a range of 1–4 RCTs. Results suggested that HD and Adj vaccines prevented more vascular adverse events compared with IIV3-SD, but with large dispersion among studies (I2=97%, τ=0.34). IIV4-Adj was ranked the most efficacious intervention and may be associated with lower vascular adverse events (IRR 0.18, 95% CI (0.07 to 0.43), P-score=0.98, 95% PI (0.03 to 1.07); very low certainty of evidence) followed by IIV4-HD (IRR 0.46, 95% CI (0.21 to 1.00), P-score=0.71, 95% PI (0.09 to 2.41); low certainty of evidence) compared with IIV3-SD (table 4, online supplemental appendix 17C (additional file 1)). However, PIs were wide and evidence of low certainty. Results were similar when restricting to RCTs with low overall ROB, and there was low between-study heterogeneity (I2=0%, τ=0.00), but more imprecision around the summary effect estimates.
Outpatient visits
The NMA for the outpatient visits included 4 RCTs,46 55 56 96 3 interventions (IIV3-SD, IIV3-HD, placebo) and 41 995 participants. The network geometry was informed by two intervention comparisons each of which included two RCTs. IIV3-SD (OR 0.64, 95% CI (0.36 to 1.14)) and IIV3-HD (OR 0.65, 95% CI (0.35 to 1.19)) may be superior to placebo in being less associated with visits, but with imprecise effect estimates, moderate between-study network heterogeneity (I2=61%, τ=0.11), and very low certainty of evidence (online supplemental appendix 17C (additional file 1)). Pairwise meta-analysis suggested an imprecise effect for IIV3-SD compared with placebo (OR 0.40, 95% CI (0.07 to 2.14), 2 RCTs, 814 participants, I2=77%, τ=1.08) (table 4).
All-cause mortality
The NMA for all-cause mortality included 20 RCTs,18 19 46 47 49 50 55–60 62 65 67 72 93 95 96 9 interventions (IIV3-SD, IIV3-HD, IIV3-Adj, IIV4-SD, IIV4-HD, IIV4-Adj, RIV, Tdap, placebo) and 140 577 participants (table 4). The network geometry was informed by 11 intervention comparisons with a range of 1–6 RCTs. The effect of RIV on all-cause mortality may be comparable to placebo (OR 1.01, 95% CI (0.23 to 4.49), low certainty of evidence). Both RIV and placebo ranked among the best interventions (P-score: RIV 0.75, placebo 0.71). Despite the low between-study heterogeneity in the network (I2=0%, τ=0.00), effect estimates were highly imprecise with very low to moderate certainty of evidence (online supplemental appendix 17C (additional file 1). Pairwise meta-analysis estimate of IIV3-SD against placebo was highly uncertain (OR 1.47, 95% CI (0.43 to 5.06), 2 RCTs,46 96 854 participants, I2=0%, τ=0.00).
Inconclusive results were provided in sensitivity analyses restricting to studies with percentage of female above 50% and overall low ROB (online supplemental appendix 17C (additional file 1)). One RCT60 (6961 participants) showed that IIV3-Adj was associated with lower odds of influenza-related deaths compared with IIV3-SD (OR 0.75, 95% CI (0.17 to 3.36)), yet the estimate was imprecise. Another RCT93 (1741 participants) suggested that IIV3-SD was associated with higher odds of influenza-related deaths compared with IIV4-SD, yet the result was imprecise (OR 3.01, 95% CI (0.12 to 73.92)) (table 5).
Discussion
This systematic review and NMA provides a comprehensive assessment of influenza VE in older adults. We included a total of 41 RCTs, involving over 200 000 participants. Our results indicated that influenza vaccines were effective in preventing LCI compared with placebo. IIV3-HD showed the most promising results in preventing LCI, followed by RIV, IIV3-Adj and IIV3-SD. However, precision of effect estimates varied among interventions with low overall certainty of evidence, except for the comparison IIV3-HD versus IIV3-SD where certainty was high. The findings highlight the potential benefits of IIV3-HD in preventing hospitalisations for ILI and hospitalisation for ARI, and of IIV4-Adj in reducing vascular adverse events among older adults. Overall, few trials were included in the intervention comparisons in LCI, ILI, vascular adverse events, hospitalisations for ILI and ARI, and influenza-related mortality outcomes. VE may differ across certain participant characteristics, such as age, sex or circulating strains between and within seasons. However, there was insufficient evidence to make definite conclusions or explore heterogeneity in populations with different characteristics (eg, comorbidities, frailty) or study characteristics (eg, circulating strains or between influenza season variability) that could modify the intervention effect. Within-study ROB was predominantly at some or high concerns across outcomes, while small-study effects and reporting bias were only possible to assess in all-cause mortality outcome with no overall concerns. Overall, our results should be interpreted with caution since uncertainty around the point estimates was high and certainty of evidence was primarily low. High certainty of evidence was in LCI (for 1 of the 15 comparisons), ILI-hospitalisation (for the single comparison), and ILI cases (for the single comparison), but moderate in vascular adverse events (for 1 of the 10 comparisons), all-cause mortality (for 15 of the 36 comparisons), ARI-hospitalisation (for the single comparison) and ER-visits (for the single comparison).
Our findings align with the systematic review and NMA conducted by Minozzi et al,104 who assessed the efficacy and safety of influenza vaccines in 220 RCTs of 100 677 children, 329 127 adults, including older adults. Overall, NMA results from their review among the older adult population were consistent with our results in the sense that influenza vaccines are better than placebo for LCI, although with varying levels of precision in effect estimates. Differences in pairwise results between the two reviews were unremarkable. Another review from the Advisory Committee on Immunisation Practices105 included 51 studies of which 27 were RCTs and 24 were non-randomised studies to assess the relative efficacy and safety of influenza vaccines in older adults. They did not conduct any RCT-based meta-analyses that could be compared with those in our review. Relative to both reviews, searches in our review were more comprehensive and up to date including 18 additional RCTs with 71 049 participants not previously included in these reviews. We also evaluated credibility of evidence using both GRADE and CINeMA approaches, which to our knowledge has not been done elsewhere.
The strengths of our study lie in its systematic approach, extensive knowledge user engagement, and rigorous analysis using NMA to compare various influenza vaccines. This integrated knowledge translation approach allowed for valuable insights throughout the research process, from refining the research question to interpreting the findings. We followed the Cochrane Handbook methods for systematic reviews,30 and reviewers worked in pairs and independently for screening, data abstraction and ROB appraisal. We reported the results using the PRISMA-NMA8 and GRIPP-2 reporting statements.9
However, some limitations should be considered. First, the limited number of studies constrained the analysis for specific outcomes, such as ILI, ER-visit for ILI, hospitalisation for ILI and ARI outcomes. The sparsity of networks, in some cases with only one or two studies per intervention comparison, increased the imprecision of effect estimates and reduced statistical power to detect differences in effects. Second, the lack of studies in certain outcomes comparing enhanced influenza vaccines limited the adequate assessment of the NMA assumptions, including exploring sources of heterogeneity (eg, through meta-regression analysis), small-study effects and inconsistency. Also, demographic variables that may explain heterogeneity, such as frailty, comorbidities and season intensities, were absent from the primary studies. RCTs usually were limited to one or two influenza seasons and because of the characteristics of influenza viruses (high mutation) and the history of exposure to influenza viruses in the population (herd immunity, effect of repeated vaccination and protection from previous exposure to influenza viruses) it is very hard to predict the efficacy of influenza vaccines in future seasons. Also, studies usually include relatively healthy populations, and there is a need for high quality RCT among older frail adults. Heterogeneity among studies varied in NMA from low to severe, but in all pairwise meta-analyses it was generally low, except for the inpatient hospitalisation outcome. While no major concerns were identified for small-study effects and publication bias for the all-cause mortality outcome, we could not assess this for any other outcome. For LCI and all-cause mortality outcomes where it was possible to assess, there was no evidence of statistical inconsistency. The transitivity assumption was likely satisfied regarding the overall ROB and participants’ age. We conducted a subgroup analysis of LCI with respect to whether vaccine strains matched the circulating strains. A more targeted analysis would have looked at specific strains; however, there was an insufficient number of studies to undertake such an analysis. Lastly, approximately half of the included RCTs reported races other than white/Caucasian, but these studies included predominantly white participants. Future studies are needed to capture influenza VE in racial diversity, so that they are representative of populations where trials are conducted.
Our study contributes valuable insights to inform evidence-based public health decisions and immunisation guidelines for older adults (eg, through the NACI Influenza Working Group). In particular, the NACI Supplemental Guidance on Influenza Vaccination in Adults 65 Years of Age and Older is anticipated to be published on the Government of Canada website in Spring 2024. Policy-makers and healthcare providers should consider these findings when formulating immunisation strategies to protect older and vulnerable populations from seasonal influenza and its complications. Annual vaccination with any vaccine is the best way to prevent infection and its complications.
Conclusions
Our analysis of RCT data highlights efficacy associated with IIV3-HD and RIV vaccines in protecting elderly persons against LCI. RIV vaccine may reduce all-cause mortality when compared with other vaccines we examined, but the evidence is uncertain. Moreover, our analysis points to a possible association of higher all-cause mortality in elderly persons receiving adjuvant vaccines (IIV3-Adj and IIV4-Adj). The comparative efficacy of individual vaccines regarding LCI, ILI, and secondary outcomes (eg, influenza-related mortality) remains uncertain. Differences among the influenza vaccines are possible and might emerge with accumulation of new evidence.
Our results should be interpreted with caution due to imprecise and heterogeneous summary effects with low certainty of evidence of the RCT studies we included. VE may vary depending on population (eg, frailty, comorbidities) virus strain matching and study characteristics. More high-quality RCTs are necessary that encompass multiple influenza seasons to evaluate the relationship between these factors and VE, such as seasonal vaccines specifically based on RIV and IIV4 formulations.
Data availability statement
Data are available on reasonable request. The full dataset and statistical code are available at https://osf.io/eqb9h/.
Ethics statements
Patient consent for publication
Acknowledgments
The authors wish to thank Huda Ashoor, Areej A. Hezam, Myanca Rodrigues, Ji Yoon Kim, Patricia Rios, Amruta Radhakrishnan and Kelsey Young for their contributions in screening and data abstracting the studies. We thank Brahmleen Kaur, Kiran Ninan and Navjot Mann for their support in formatting the paper and creating the EndNote library. We thank Dr Jessie McGowan for developing the literature searches, Tamara Rader for peer-reviewing the literature search, and Raymond Daniel for conducting the literature search. We thank the National Advisory Committee on Immunisation Influenza Working Group for their content expertise.
References
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.↵
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.↵
- 38.↵
- 39.↵
- 40.↵
- 41.↵
- 42.↵
- 43.↵
- 44.↵
- 45.↵
- 46.↵
- 47.↵
- 48.↵
- 49.↵
- 50.↵
- 51.↵
- 52.↵
- 53.↵
- 54.↵
- 55.↵
- 56.↵
- 57.↵
- 58.↵
- 59.↵
- 60.↵
- 61.↵
- 62.↵
- 63.↵
- 64.↵
- 65.↵
- 66.↵
- 67.↵
- 68.↵
- 69.↵
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 74.↵
- 76.↵
- 77.↵
- 78.↵
- 79.↵
- 80.↵
- 81.↵
- 82.↵
- 83.↵
- 84.↵
- 85.↵
- 86.↵
- 87.↵
- 88.↵
- 89.↵
- 90.↵
- 91.↵
- 92.↵
- 93.↵
- 94.↵
- 95.↵
- 96.↵
- 97.↵
- 98.↵
- 99.↵
- 100.↵
- 101.↵
- 102.↵
- 103.↵
- 104.↵
- 105.↵
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
X @AVeroniki, @AStevens_PhD, @ivand_florez, @76Juano
Contributors AAV, SS and ACT designed the study. ACT and SS obtained funding. SST, MG and JD coordinated the study. SST, JD, MG, PK, VN, MC, RR, AP and CS screened articles, abstracted data and carried out risk of bias assessments. AAV and MK conducted data analyses. IDF and JJY-N conducted GRADE certainty of evidence assessments. AAV drafted the first version of the manuscript. All authors contributed to the manuscript’s revision and interpretation of findings. AAV is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding This work was funded by a Canadian Institutes of Health Research Drug Safety and Effectiveness Network (No. DMC–166263). SS is funded by a Tier 1 Canada Research Chair in Knowledge Translation and Quality of Care, the Mary Trimmer Chair in Geriatric Medicine, and a Foundation Grant (Canadian Institutes of Health Research). ACT holds a Tier 2 Canada Research Chair in Knowledge Synthesis. MC was part supported by the Health Research Board (Ireland) and the HSC Public Health Agency (Grant number CBES-2018-001) through Evidence Synthesis Ireland/Cochrane Ireland.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.