Vaccination has long been recognized as one of the most effective, far-reaching, and cost-effective public health interventions for controlling infectious diseases^[1]. Traditionally, the value of vaccines has primarily been assessed based on direct health outcomes, healthcare cost savings, and short-term productivity gains for patients and caregivers^[2]. However, vaccines commonly yield “broader” benefits and externalities than conventional metrics. These include long-term cognitive and educational improvements, mitigation of antimicrobial resistance (AMR), protection of specific economic sectors (such as tourism), prevention of catastrophic household health expenditures, and fostering industrial innovation^[3].

Accumulating evidence suggests that the value of vaccines is systematically underestimated in Health Technology Assessments (HTAs) because of the omission of these broader dimensions. Failure to capture the full impact of vaccination limits the ability of decision makers to optimize the design, funding, and implementation of national immunization programs. Such an undervaluation can result in suboptimal resource allocation, particularly as vaccination costs increase and competition for healthcare budgets intensifies. Consequently, a comprehensive value assessment framework is essential to link vaccination programs with their multifaceted benefits to public health, the economy, and society.

Building on existing evidence and expert consultation, the present study aims to: (1) develop a consensus-based conceptual framework capturing the "Comprehensive Vaccine Value" and (2) prioritize specific value elements for integration into future HTA processes.

In Phase 1, a protocol was developed in accordance with the methodological principles of the PROSPERO guidelines to ensure systematicity and transparency, although formal registration was not conducted. The inclusion criteria utilized an adapted version of the “PICAR” (P, population, clinical indication, and condition; I, intervention; C, key content; A, attributes of the guidance; R, recommendation characteristics) framework, originally developed for the systematic review of clinical practice guidelines^[4]. Electronic searches were conducted using the MEDLINE, EMBASE, PubMed, China National Knowledge Infrastructure (CNKI), and Wanfang databases on December 28, 2024, focusing on Chinese and English sources that presented vaccine value frameworks or discussed the individual value elements. In addition, gray literature was searched from key international and national health/vaccination websites. The specific search strategies are detailed in the Supplementary Table S1-S6.

All included studies were divided into two types; those with or without framework reporting. Data extracted from each framework encompassed the development objective, vaccine focus, peer-review status, classification hierarchy (categories and value elements), and authors' country of affiliation. A summary is provided in the Supplementary Table S7. Vaccine value frameworks in the same or similar categories were grouped. The final number of groups was not pre-specified. Extraction and grouping were conducted by some of the study members and critically appraised by others, including two experts.

Subsequently, all relevant vaccine value element concepts and measurement indicators were comprehensively extracted from the literature. To synthesize these findings, the study team systematically merged identical or similar indicators. These consolidated metrics were subsequently deductively categorized and summarized into new Value Elements and their corresponding actionable Value Items, thus forming a preliminary three-level hierarchy, with Value Categories, Value Elements, and Value Items classified as Tiers 1, 2, and 3, respectively. To ensure conceptual clarity and practical applicability, the boundaries between these tiers were defined as follows: Value Categories (Tier 1): Broad theoretical domains grouping related impacts based on their overarching societal or economic nature (e.g., Economic and Productivity Value). Value Elements (Tier 2): Specific, distinct impact constructs within a category representing the core conceptual benefits of vaccination (e.g., Value H: Macroeconomic Gains). Value Items (Tier 3): Highly granular, actionable indicators designed for direct measurement, modeling, or data collection in HTA practice (e.g., Item H1: Impact on Gross Domestic Product [GDP] or output of specific economic sectors). The study team and two external experts critically appraised the initial structure.

In Phase 2, two Delphi rounds were conducted using email surveys between August 2025 and November 2025 to: (1) identify the key criteria adopted when assessing the framework, (2) validate the conceptual appropriateness of the framework, and (3) prioritize the value elements in the framework for HTA inclusion. A prioritization scheme based on six criteria—three relating to “Relevance” and three to “Feasibility”—was proposed based on the adaption from a previously established evaluation framework^[5]. The experts were asked to assign a weight to each of the six criteria, with the constraint that the aggregate weight across all six criteria must be equal to 100 per expert. The value concepts from each level were subsequently scored by each expert on a scale from 1 (‘‘Not at all”) to 5 (‘‘Strong yes”). To prioritize the elements, the total weighted score for each value element was calculated as follows:

where $ j $ represents the specific value element; $ n $ is the total number of experts in the final round ($ n $ = 29); $ k $ represents the six evaluative criteria; $ {S}_{i,j,k} $ is the raw score (1–5) assigned by expert $ i $ for value element $ j $ on criterion $ k $; and $ {W}_{i,k} $ is the weight assigned to criterion $ k $ by expert $ i $.

A systematic search identified 5,716 records across the PubMed, Ovid MEDLINE, Embase, CNKI, and Wanfang Data Knowledge Service Platform, supplemented with 61 records retrieved from secondary sources (Figure 1). After removing duplicates, 5,010 unique articles were screened. The preliminary application of the inclusion criteria resulted in the exclusion of 4,312 records, leaving 698 articles for further evaluation. Subsequent screening by title, abstract, and full text led to the exclusion of 555, 75, and 7 articles, respectively. Ultimately, 61 studies met the eligibility criteria for inclusion. Of these, 28 established structured frameworks with the corresponding value concepts or elements, while 33 reported discrete value elements without a formal framework.

Figure 1.  Flowchart of the article review and Delphi consultation.

Among the 28 identified frameworks, 17 were generic and intended for broad applications across all vaccine types. The remaining frameworks targeted specific populations or pathogens: one was designed exclusively for adult vaccines, two focused on COVID-19, two focused on pneumococcal vaccines, and six focused on other specific populations or pathogens. Notably, 67.9% (n = 19) of these frameworks were derived from formal peer-reviewed processes.

Geographically, these frameworks were developed predominantly by researchers in high-income countries. Specifically, seven were authored by United States -based scholars or integrated into HTAs within the context of the United States, while six originated from the United Kingdom (UK) or were applied to UK-based vaccination HTAs. Six additional frameworks intended for global applications were developed without a specific national focus.

Regarding structural taxonomy, the classification methods among the 28 frameworks varied in complexity. Twelve studies utilized a binary (two-category) approach, typically distinguishing between "narrow effects" and "broad effects.” Four frameworks adopted a tripartite (three-category) structure, commonly grouping value elements into "health gains," "productivity gains," and "community externalities.” The remaining studies employed multi-tiered systems; nine utilized four categories, two introduced a five-category system, and one proposed a six-category classification (Figure 2).

Figure 2.  Summary of the identified vaccine value frameworks

A total of 31 experts participated in the initial consultation (round 1), of whom 29 (93.5%) subsequently completed round 2 of the voting process. The participants’ demographics, including gender, age group, highest professional degree, professional title, research field, and years of work, are summarized in Table 1. Notably, all the participating experts were based in mainland China.

The aggregate weight for the six prioritization criteria was set at 100, with experts assigning individual scores to reflect their relative importance. Among these, Criterions 1 (Importance), 2 (Association), and 4 (Evidence/Data Accessibility) emerged as the most critical, with weights of 22.1%, 19.7%, and 17.9%, respectively (Figure 3).

Figure 3.  Average weight of the six criteria to assess value items.

Synthesizing the findings from Phase 1, an initial three-tiered framework (version 1.0) was constructed, comprising five value categories, 20 value elements, and 71 value items. This framework was rigorously refined through two successive rounds of Delphi. Revisions following Round 1 yielded Version 2.0 (5 categories, 21 elements, and 76 items), while Round 2 culminated in the finalized version 3.0, comprising 5 value categories, 21 value elements, and 75 value items.

The expert responses for each criterion across all three tiers are summarized in Table 2. Compared with the initial round, the second Delphi consultation exhibited a clear convergence of expert opinions, as evidenced by the general narrowing of the ranges of the mean scores and standard deviations across the value items. Notably, the maximum coefficient of variation (CV) decreased for several items, indicating reduced dispersion and higher stability of the ratings. Concurrently, an observed increase in the full-score rates further signified an enhanced consensus on the framework's core components. These statistical shifts collectively demonstrated a meaningful progression towards expert agreement, facilitated by the iterative consultation process. The detailed responses for each value category, value element, and value item by round are provided in the Supplementary Table S8-S10.

The initial consensus was defined as > 50% of the expert panel, assigning either a high (4 or 5) or low (1 or 2) rating to both consultation rounds. This relatively inclusive threshold was intentionally selected to capture an extensive 'baseline menu' of emerging value elements in this nascent field, preventing the premature exclusion of any novel concepts that currently lack standardized measurement methods. To ensure methodological rigor, sensitivity analyses were subsequently conducted using more stringent and widely adopted consensus thresholds of 62% and 73% (detailed in Supplementary Figure S1 and S2). The consensus status for each second-tier value element following the second round is shown in Figure 4.

Figure 4.  Consensus on each value element post the 2^nd round consultation.

Regarding expert consensus, value elements A through E (traditional metrics) consistently achieved high-score consensus across all evaluative criteria, with the sole exception of "Uniqueness," where consensus was limited to elements A and C. Beyond these core elements, broader value dimensions exhibited varying degrees of agreement across the six criteria.

Specifically, elements I, J, M, O, and Q reached consensus within the high-score range for "Importance." Under the "Association" criterion, a high-score consensus was observed for H, M, and O, while elements L and P were notably clustered in the low-score range. In terms of "Evidence/Data Accessibility," experts reached a high-score consensus on elements H, I, M, N, and O. Similarly, for "Evidence/Data Quality," elements H, M, and O achieved high-score consensus, while element K was consistently rated in the low-score range. Finally, for "Method Accessibility," a high-score consensus was identified for H, O, and Q, while elements K, P, S, and T were categorized within the low-score range. Following the conclusion of round 2, the Comprehensive Vaccine Value framework was finalized, as presented in Table 3.

To identify the key broader value elements for future integration into HTAs or health economics research, we conducted a prioritization analysis based on both raw scores (ranging from 1 to 5 per criterion) and weighted scores (Supplementary Figure S3 and S4).

The raw score analysis revealed that the highest ratings were predominantly concentrated in elements A–E, which aligned with the traditional metrics currently applied in standard HTAs. Excluding these conventional elements, the five highest-ranking broader value elements by total raw score were as follows: O. Enhancement of Health System Security (672), H. Macroeconomic Gains (646), M. Value of Social Equity and Ethics (642), Q. Prevention of Institutional Disruptions (620), and N. Value of Other Interventions (589). Notably, the weighted-score analysis yielded identical prioritization results, reinforcing the robustness of these findings. The leading broader elements (O, H, M, Q, and N) attained total weighted scores of 113.50, 110.18, 109.48, 104.15, and 99.25, respectively. These elements represent the most critical dimensions for future consideration when expanding the scope of vaccine value assessment.

The primary aim of this study was to develop and validate a robust framework for assessing "Comprehensive Vaccine Value,” to facilitate more comprehensive decision-making and enhance the communication of vaccines' multifaceted benefits. We successfully adapted and validated a framework that reflected the consensus of a multidisciplinary expert panel by synthesizing international evidence through a systematic literature review and a two-round modified Delphi study.

This systematic review identified 61 publications addressing "broader" vaccine value, 28 of which proposed a structured framework. These studies have elucidated dozens of value items beyond conventional health metrics, such as social equity, herd immunity, AMR mitigation, macroeconomic stability, and outbreak control. Notably, the value of social equity has gained significant prominence in recent policy discourse. Similarly, the role of herd immunity^[6] in protecting unvaccinated populations and reducing antibiotic consumption has been increasingly substantiated by diverse regional evidence^[7-10]. Despite persistent methodological heterogeneity, the integration of herd immunity into economic evaluations has risen sharply, from less than 5% in 2001 to approximately 50% in 2015^[11]. Furthermore, the macroeconomic impact of vaccination, encompassing national GDP, employment, and sectors such as travel and tourism, has been acutely highlighted throughout the COVID-19 pandemic^[12-14]. Vaccination has proven to be a cornerstone of economic recovery, particularly in high-coverage nations, while, low coverage in other regions has threatened to stifle economic growth for years, potentially exacerbating global inequities^[15].

The existing vaccine value frameworks typically encompass health system benefits, societal externalities, and behavior-related productivity gains. These frameworks have evolved from a simple dichotomy of "narrow" versus "broad" effects, to more sophisticated multi-category structures. While early frameworks were largely generic, recent years have seen the emergence of pathogen- or population-specific models (e.g., pneumococcal, COVID-19, tuberculosis, and adult vaccines), which significantly enhance the operational utility. However, a geographic imbalance persists: most frameworks originate from high-income countries, leaving a critical gap in the perspectives of developing nations. Furthermore, many existing frameworks are based on scoping or targeted reviews, rather than any rigorous systematic methodologies. Our framework offers three distinct advantages over existing models. First, it introduces 75 actionable Tier-3 indicators, thereby bridging the gap between theoretical value and HTA practice. Second, unlike previous descriptive models, we established six quantitative criteria to prioritize value elements, thus providing a pragmatic roadmap for implementation. Finally, by incorporating the consensus from a major middle-income country, this study mitigates the geographic bias of current high-income-centric frameworks, thereby offering a more globally balanced perspective.

To address these gaps, this study innovatively proposes and defines six criteria for capturing expert perspectives. Notably, “Importance” and “Association” emerged as the highest-weighted criteria, reflecting a consensus on the fundamental requirements for value elements. Our results showed that while traditional elements (A–E) routinely used in Cost-Effectiveness Analysis (CEA) and HTA achieved high consensus across nearly all criteria, five broader value elements were prioritized for future integration.

The values O (enhancement of health system security) and Q (prevention of institutional disruptions) reflect the vaccine value during an infectious disease outbreak and pandemic. The main drivers of the high ranking of the two value elements may result from increased awareness of the negative impact of COVID-19 on societal functions^[16,17]. Further, the recent global emergence of infectious threats such as the MPOX epidemic underscores the critical need for preventative vaccines to avoid localized outbreaks, covert transmission, and subsequent health system disruptions^[18]. Furthermore, the prioritization of these specific elements may also reflect the recent experiences and policy priorities within the Chinese healthcare system. While these findings are highly relevant for middle-income countries with similar public health infrastructures, the weights assigned to these elements may vary across different cultural or institutional settings. Value M (the value of social equity and ethics), which focuses on the equitable distribution of health outcomes^[19], remains the cornerstone of the framework. Indeed, vaccination has been shown to mitigate wealth-related disparities in childhood mortality and prevent household impoverishment, particularly in low-income settings^[20-23]. Additionally, value H (macroeconomic gains) accounts for the role of vaccines in stabilizing national economies during infectious disease crises, as well as the steady contributions of routine immunization to national GDP^[24]. Vaccination can enhance the cost-effectiveness of non-pharmaceutical interventions by lowering disease prevalence to a manageable level, thereby allowing targeted and lower-cost interventions to replace expensive, broad-scale restrictive measures^[25-27], and justifying the Value N (value to other interventions) impact on the cost-effectiveness of other non-vaccine interventions. These five priority value elements require further development (i.e., specifying the kind of vaccine, when each of the elements should be included, and how to measure them) to effectively inform future HTA and policy decisions.

Translating this comprehensive framework into HTA practice will require several implementation barriers, primarily empirical data scarcity, methodological risk of double counting (e.g., conflating individual productivity with macroeconomic gains), and the inherent heterogeneity of vaccines. Given the diverse disease characteristics, immunization strategies, and dynamic epidemiological landscapes, the application of static value weights across all vaccines is methodologically impractical.

To address these challenges, three implementation strategies are proposed. First, the framework should be conceptualized as a deconstructable modular toolkit. Instead of employing a rigid scoring instrument, HTA agencies should dynamically calibrate the value weights tailored to specific pathogenic and institutional contexts. Future research should prioritize the empirical validation of this modular approach through multi-disease case studies. Second, a stepwise integration strategy, transitioning from qualitative appraisals to quantitative modeling as the data infrastructure matures, is recommended. Finally, advancing methodological standardization, such as by integrating dynamic transmission models with macroeconomic models, remains imperative for accurately monetizing broader externalities while mitigating double-counting risks.

Guided by the growing academic consensus that vaccines yield significant value beyond the dimensions traditionally captured in HTA, the present study employed a systematic literature review and a robust, two-round Delphi consultation to develop and validate a "Comprehensive Vaccine Value" framework. Within this framework, value elements are clearly defined and categorized based on their specific attributes. Furthermore, we introduced a pragmatic third-level value item to facilitate granular assessment. By establishing six distinct evaluative criteria, the present study provides a structured prioritization of value elements, offering a roadmap for their integration into future economic evaluations and HTAs.

Despite its contributions, this study had several limitations that warrant consideration. First, the search was restricted to publications in English and Chinese. This language restriction means that we may have inadvertently excluded distinct sociocultural or economic value constructs unique to regions that publish predominantly in other languages, potentially resulting in an underrepresentation of highly localized Tier-3 items. However, as English serves as the primary language for global scientific communication and international health organizations, the core structural domains of the framework are likely to be globally robust. Nonetheless, future iterations will benefit from multilingual scoping to ensure absolute global applicability. Second, while we aimed for mutually exclusive categories, the inherent complexity and interrelated nature of vaccine externalities means that some value elements may resist more strict classification, leading to potential categorical redundancy or overlap. Third, the Delphi panel consisted exclusively of Chinese experts. Although they represent a broad range of disciplines, their perspectives may be shaped by China’s specific national immunization program structure and recent experiences with the COVID-19 pandemic. Consequently, they may be predisposed to prioritize the value elements associated with pandemic resilience and health security. Therefore, while the three-tiered framework is designed to be globally applicable, the specific prioritization of results should be interpreted with caution when applied to other settings with different healthcare financing models. Finally, while our framework identifies "what" to measure, the specific quantitative methodologies for many Tier 3 items currently remain under-defined. Future research should therefore focus on developing standardized quantification methods and investigating the practical mechanisms by which such broad value frameworks can be formally incorporated into national decision-making processes.

Overall, the present study established a validated, three-tiered, comprehensive vaccine value framework, successfully transitioning theoretical broad value concepts into an actionable structure for HTA. This framework provides a standardized roadmap for capturing the multifaceted externalities of vaccination that are often overlooked in conventional evaluations. A pivotal finding of this study is the prioritization of health system security, macroeconomic gains, and social equity, signaling a critical paradigm shift in vaccine valuation in the post-pandemic context. Although traditional health metrics remain the foundation, the inclusion of these broader dimensions is essential to prevent systematic undervaluation and to optimize resource allocation in national immunization programs. Overall, this framework serves as a pragmatic tool for decision makers to facilitate evidence-based policy formulation and transparent value communication. Future research should focus on operationalizing quantitative methodologies for the identified Tier-3 items to ensure that the holistic value of immunization is rigorously integrated into global health economics research.