Introduction

The concept of health literacy has been expanded since it was first defined as the level of ability to obtain, process, and understand basic health information and services necessary for individuals to make appropriate health decisions [1,2]. The World Health Organization announced health literacy as one of the key competencies for promoting individual health, and later emphasized the need for a strategy to expand it into an organizational capacity that enables health equity [3]. Since the 2000s, the United States, Europe, Australia, and China have established and promoted national policies on health literacy at the national level [4-6]. In Korea, eHealth literacy was included as a key task in the 5th Comprehensive National Health Promotion Plan (2030) announced in 2021 [7].

Recent advancements in information and communication technology, alongside the Fourth Industrial Revolution, have introduced a paradigm shift in medical services. The integration of big data, artificial intelligence, the Internet of Things, and medical technology has increased society's reliance on these technologies [8]. In the past, health information was primarily obtained from medical professionals and textbooks. However, with the rapid develop ent of information and communication technology, the ability to search for, understand, and utilize health information online using computers and mobile devices has become crucial [8,9]. This shift has given rise to new terms such as eHealth, digital health, and mHealth, which describe the medical service environment leveraging these technologies [9,10]. eHealth involves providing health-related information online and using information and communication technologies, such as the Internet, to manage health and chronic diseases [10]. As the use of these technologies in healthcare grows, transitioning from traditional health literacy to eHealth literacy has become essential.

eHealth literacy encompasses the ability to effectively access, evaluate, understand, and apply health information from the Internet to address health issues. It includes effective search techniques, such as assessing quality, reliability, validity, and sources, as well as ethical considerations, like protecting one's own information, respecting and protecting others' information, and being a responsible digital citizen [9,10]. eHealth literacy extends beyond basic functional health literacy to include communication skills, interactive literacy, and critical literacy that help individuals interpret diverse information and derive meaningful insights [10,11].

In early studies, eHealth literacy was measured using unstructured questions about whether people could understand information from media such as computers and TV [12]. The first published structured self-report measurement tool was the eight-item eHealth Literacy Scale (eHEALS). Norman and Skinner stated that six abilities—digital literacy, health literacy, information literacy, scientific literacy, media literacy, and computer literacy—interact to enable people to effectively utilize health information [10,13].

Since the development of eHEALS, the use of mobile devices has become more widespread. In response, the scope of measurement tools has recently expanded to include interactive technologies utilizing information and communication technology [11,14], as well as digital content creation [15], in the assessment of eHealth literacy. Lee and her colleagues conducted a systematic review of eHealth literacy measurements published up until March 2021, which included seven measurement tools, covering studies that explored various factor structures of the widely used eHEALS [16]. Although it has been less than 20 years since eHEALS was first published, the concept of digital literacy is rapidly evolving with the advent of advanced digital technologies, such as generative Artificial Intelligence like chat Generative Pre-trained Transformer (ChatGPT), which is quickly becoming accessible to the general population. Consequently, ongoing research on eHealth literacy is anticipated both nationally and internationally.

Therefore, it is crucial to choose a measurement tool that accurately assesses the relevant concepts when evaluating eHealth literacy in the community population. Utilizing the evaluation results can aid in developing appropriate interventions and policies. In this study, we conducted a qualitative evaluation of the characteristics and psychometric properties of tools designed to measure eHealth literacy across all age groups in the community. We used the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist [17] as a framework for evaluating the measurement properties of self-report questionnaires. The findings aimed to offer guideline for selecting appropriate tools in future research and to provide foundational data for exploring research directions in the development and evaluation of eHealth literacy measurement tools.

Methods

Study Design

This study conducted a systematic review of research on questionnaires developed and evaluated to measure eHealth literacy, conducted in accordance with the COSMIN guidelines and the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) [17,18]. We focused on the four key elements suggested for systematic reviews of patient-reported outcome measures: 1) the construct; 2) the population; 3) the type of instrument; and 4) the measurement properties of interest [17,18]. The construct under review was eHealth literacy, and the population included adolescents and adults living in the community. The type of instrument was self-report questionnaires, encompassing all relevant measurement properties of these tools.

Eligibility Criteria

The inclusion criteria for systematic reviews were as follows: (1) studies that reported the development and validation process a self-reported questionnaire measuring eHealth literacy of adolescents and adults within community settings; (2) studies that re-evaluated the validity of an existing tool and categorized the domain differently from previous studies; (3) studies published in English or Korean; and (4) studies published in academic journals. The exclusion criteria were as follows: (1) studies for which the full text could not be found; (2) studies that measured only health literacy, excluding digital attributes; (3) studies that measured only digital literacy, excluding health-related attributes; (4) studies that measured eHealth literacy targeting patients with specific diseases; (5) studies that aimed to verify the cross-cultural validity of an existing tool; and (6) review studies.

Data Collection

Data collection included a comprehensive search of both domestic and international databases for literature published up until June 2024. The search was limited to English and Korean languages and used keywords and MeSH terms relevant to prior research. Databases searched included PubMed, CINAHL, Google Scholar, and RISS, with continuous manual searches for newly identified literature. Keywords such as ‘digital health literacy,’ ‘ehealth,’ ‘health literacy,’ ‘mhealth,’ ‘measurement development,’ and ‘psychometrics’ were employed. The search strategy used Boolean operators (AND/OR) and truncation (*) to enhance the sensitivity and specificity of the search. The search strategy outlined in the systematic review protocol for eHealth literacy measurement tools [19] was applied, along with the measurement properties and exclusion filters for PubMed [20] from COSMIN, to identify any tools that may have been omitted in previous search results.

Data Extraction

A total of 413 studies were identified through various academic sites: 83 from PubMed, 32 from CINAHL, 269 from Google Scholar, 13 from RISS, and 16 from manual searching. After excluding 47 duplicates, 366 studies were selected in the first round. Following a review of titles and abstracts, 318 studies were excluded for not meeting the research topic and selection criteria, leaving 48 studies for the second round. After reviewing the full texts of 48 studies, we excluded 27, including those that were translated versions of the original tool or had the same construct validity results as the original tool (n=12), studies that measured eHealth literacy for specific diseases such as diabetes or COVID-19 (n=5), studies that did not report enough statistical analysis results or failed to report the questionnaire development process (n=3), and review papers (n=5). Ultimately, we selected 21 studies for the review (Figure 1).

To examine the characteristics of the tools included in the final analysis, data were extracted on the author and published year, instrument name, target population and location, total number of items, domains and the number of items for each domain, scoring system, reliability, and validity. The extracted data was organized into a table format. To ensure the reliability of the data description, the information was compiled according to a standardized data description format, and the consistency of the data was reviewed separately by two researchers (Table 1).

Data Evaluation

The quality assessment of the psychometric properties of the questionnaire was evaluated as ‘sufficient (+)’, ‘insufficient (-)’, or ‘indeterminate (?)’ based on the updated COSMIN checklist criteria [17,18]. First, internal consistency was evaluated as (+) if Cronbach's alpha(s) were ≥ 0.70 for each unidimensional scale or subscale, and (-) if they were lower. Second, reliability was evaluated using intra-class correlation (ICC), test-retest reliability, or weighted Kappa, with a threshold of ≥ 0.70 for a (+) rating. Third, measurement error was assessed following several criteria: whether the two tests were conducted independently, whether the time interval between tests was appropriate, whether the subject's condition and the test environment remained stable without changes during the interval, and whether the smallest detectable change or limits of agreement were below the minimal important change.

Fourth, content validity was evaluated based on the clarity of the measurement goal, the definition of the target population, the measurement concept, item selection, and the understandability of the tool. It was also assessed whether subject and expert validity investigations were included. Fifth, construct validity was evaluated by examining whether exploratory factor analysis (EFA) and/or confirmatory factor analysis (CFA) were conducted within the framework of classical test theory, and whether Rasch analysis was performed within the framework of item response theory. For EFA, a rating of (+) was given if the explanatory power was 50% or higher. For CFA, a rating of (+) was given if the Comparative Fit Index (CFI) or Tucker-Lewis Index (TLI) was greater than 0.95, the Root Mean Square Error of Approximation (RMSEA) was less than 0.06, or the Standardized Root Mean Residuals (SRMR) was less than 0.08. Sixth, in hypothesis testing for construct validity, evaluations were based on whether hypotheses were set in advance and whether the direction and size of the hypotheses were presented and consistent.

Seventh, cross-cultural validity/measurement invariance was assessed to determine if the translated measurement tool accurately reflected the items of the original tool, considering cultural characteristics. A (+) rating was given if there were no differences in factors such as age, gender, and language between groups, or if the differential item functioning (DIF) was less than 0.02 (McFadden’s R²). Eighth, criterion validity was evaluated based on the degree of correlation with a gold standard. A rating of (+) was assigned if the correlation with the gold standard was 0.70 or higher, or if the Area Under the curve (AUC) was 0.70 or higher.

Responsiveness was assessed by evaluating whether hypotheses regarding time intervals, directions, and magnitudes were set in advance. A (+) rating was given if the correlation between tests was 0.50 or higher, if the degree of agreement between results and hypotheses testing was 75% or higher, or if the AUC was 0.70 or higher. Additionally, a (+) was assigned if the result value conformed to the hypothesis or if a cut-off value was determined and met the criteria. The evaluation of floor and ceiling effects was based on whether each was assessed at 15%.

Results

Characteristics of Included Questionnaires

The general characteristics of the questionnaires included in the study are shown in Table 1. A total of 21 studies were reviewed, analyzing 19 distinct questionnaires. Among these, three studies (A3, A12, A18) focused on the eHEALS developed by Norman and Skinner, analyzing it as a single factor [13], two factors—information acquisition and application [21], and three factors—awareness, skill, and evaluation [22]. The studies were published from 2006 onward, with 1 study by 2010, 4 by 2015, 10 by 2020, and 6 up to June 2024. The number of domains varied from one to nine, and the diversity in domain names made quantification challenging, as they were often divided or grouped into multiple categories. Many questionnaires included domains such as ‘access to digital services,’ ‘evaluating mHealth information,’ ‘digital/operational skills,’ and ‘understanding.’ Other domains included were ‘communication,’ ‘motivation,’ ‘privacy,’ ‘application,’ ‘content creation,’ and ‘confidence.’

The number of items in each questionnaire ranged from 8 to 35. Participants in the studies were between 13 and 93 years old and lived in the community. Research was conducted in the United States, China, and the United Kingdom with approximately 3 to 4 studies in each country, followed by Korea, Taiwan, and Denmark with 2 studies each. Most questionnaires used Likert scale responses, with 12 studies employing a 5-point Likert scale. Regarding reliability, 21 studies reported Cronbach’s α, 4 reported test-retest reliability, 3 reported composite reliability, and 1 reported ICC. For validity, 14 studies reported EFA, 11 reported CFA, 6 reported both EFA and CFA, 1 study used Rasch analysis, and 1 study did not report the validity results.

Evaluation of Psychometric Properties

The results of the quality assessment of the psychometric properties of the eHealth literacy measurement tools are shown in Table 2. Cronbach’s α was reported in all studies for internal consistency assessment. Of these, 11 studies reported Cronbach’s α of .70 or higher for each domain, while 10 studies reported results below .70 or provided reliability data only for the entire questionnaire, which were evaluated as unsatisfactory,. Regarding reliability, five studies reported test-retest reliability or ICC, with only two studies (A7, A9) meeting the criterion of .70 or higher [23,24]. No studies reported measurement errors.

Content validity was reported in 15 studies, but two studies (A11, A12) did not meet the criteria, lacking adequate evaluation through external experts or pilot studies despite meeting the measurement purpose and research targets [13,25]. All studies except one study (A15), which did not report structural validity [26], conducted and reported EFA, CFA, or both. A total of 11 studies were rated as (+) for structural validity, while 9 studies were rated as (-). Statistical issues included factor loadings below .50 in four studies (A3, A6, A14, A17) [21,27-29], EFA explanatory power below 50% in one study (A1) [30], insufficient statistical analysis results in two studies (A8, A11) [25,31], and unmet model fit criteria in two studies (A16, A20) [15,32]. For hypothesis testing, 14 studies were rated as (+), 3 as (-), and 4 as (?). In studies rated as (+), hypotheses were set based on variables such as gender, age, education level, health level, or criterion validity, and the results supported these hypotheses.

Cross-cultural validity was rated as (+) in 6 studies, (-) in 7 studies, and (?) in 8 studies. Studies that met the criteria verified differential item functioning for age and gender and included procedures such as translation and back-translation and cognitive interviews for foreign language versions. Criterion validity was reported in six studies (A7, A9, A13, A19-21) [14,15,23,24,33,34], with one study (A13) [14] reporting criterion validity but receiving a (-) rating. Responsiveness was rated as (+) in 12 studies, (-) in 7 studies, and (?) in 2 studies. Six studies reported floor and ceiling effects (A3, A5-7, A10, A19) [21,23,27,33,35,36], while 15 studies did not report these effects.

In summary, 6 instruments had 5 or more sufficient (+) items, with A7 having the most, followed by A9, A21, A6, A19, and A20 [15,24,27,33,34]. There were 5 instruments with 0-1 sufficient (+) items, specifically A1, A8, A11, A14, and A15 [25,26,28,30,31]. However, among these, A15 had 1 insufficient (-) item, A8 and A14 had 3 insufficient (-) items, and A1 and A11 had 5 insufficient (-) items (Appendix 1).

Discussion

In this study, we systematically examined the characteristics and psychometric properties of 19 questionnaires from 21 studies, developed to measure eHealth literacy through self-reported assessments targeting the general population in community settings. The COSMIN checklist was utilized for the evaluation [16,18]. Analysis of the publication years of the studies revealed that five studies, including eHEALS developed in 2006 [13], were published in the past 10 years, ten studies in the following 10 years, and six studies in the past four years. This indicates a growing interest in digital literacy and research around measurement tool development. Notably, many of these studies were published in countries like the United States, China, and the United Kingdom, likely due to the implementation of related policies in these regions [3,4,6,7]. Since the last systematic review of eHealth literacy instruments, papers have been published in six countries: Korea [15,23], China [24,34], Indonesia [26], and Belgium [32]. This shift confirms that research, which was previously conducted primarily in the United States, Canada, and European countries, is now being actively pursued in Asia.

Some studies focused on adolescents and young adults [13,21,24-26], but most included all age groups, demonstrating that the newly developed digital literacy tools are applicable across various age groups in the community. The sample sizes in these studies were sufficient for statistical reliability and validity verification [37]. The domains of the tools, as presented in Table 1, showed varying levels of distinction, making categorization complicated. It has been noted that questionnaires developed within the past 10 years have diversified, encompassing concepts of eHealth literacy. Consequently, various domains have emerged, such as ‘privacy’ [22,33], ‘mobile-based patient–doctor communication’ [25,26], and ‘content creation’ [27,28].

eHEALS was the first questionnaire developed among eHealth literacy measurement instruments, and it was translated into 18 languages and used in over 26 countries [16]. In this study, we included three studies [13,21,22] that analyzed eHEALS into one to three factors. For the single-factor study, both reliability and validity were sufficient. However, in the EFA analysis of the two-factor study, the factor loading was insufficient, ranging from .32 to .91, although internal reliability was sufficient [21]. In the study comparing the factorial validity of baby boomers in the US, UK, and New Zealand across three factors, structural validity was supported by CFA. However, internal reliability was insufficient as they only reported the Cronbach’s α for the entire tool and failed to demonstrate it for each factor [22].

Boston University operates the Health Literacy Tool Shed (tuftsmedicine.org), a database that includes various health literacy measurement tools, including eHealth literacy tools. As of July 2024, 280 health literacy tools were registered, nine of which are eHEALS related, including specific targets and translations [38]. Given that not all published tools are listed, the actual number of existing health literacy and digital literacy measurement tools is likely higher. While developing various measurement tools is crucial, the significance of this study lies in building a comprehensive database and analyzing the tools through systematic reviews, enabling future researchers to select appropriate tools and conduct suitable validation analyses.

In this study, among the reliability evaluation indices for eHealth literacy measurement tools in the 10 quality assessment areas suggested by COSMIN, the psychometric index that most consistently reported was internal reliability, with Cronbach’s α reported in all studies. There were three studies where only the reliability of the entire tool was reported without reporting by subscale, and seven cases did not meet the COSMIN standard of ≥.70 for each unidimensional subscale [17,39]. On the other hand, for test-retest and ICC, which indicate the stability of the instrument, and measurement error, the number of studies that reported these standards was small or absent, suggest the need to report Cronbach’s α for each subscale and report on test-retest test results and measurement error in relation to reliability reporting in future questionnaire development and validation studies.

In terms of validity, construct validity and content validity were reported in most studies. For construct validity, EFA, CFA, or both verifications were performed, except for one study [26]. The results of the quality assessment were insufficient in cases where statistical analysis results were not sufficiently reported. In most cases, factor loadings by item were not reported. In cases where factor loadings are presented in a figure, low pixel resolution made it difficult to read the numbers. Therefore, it is necessary to include factor loadings by item as an appendix or provide a figure with appropriate resolution. For instrument tools that are rated insufficient due to low factor loadings or poor model fit, future validation studies using the same tool are necessary to re-evaluate the psychometric quality, as there are currently no comparative studies available.

The quality assessment of content validity generally explained the aim of instrument use and target population, but two studies lacked sufficient process or explanation for expert validity or pilot surveys that could confirm the relevance, comprehensiveness, and comprehensibility of the items [40]. Six studies did not provide any such explanation. In this study, only six studies met the criteria for quality assessment. Since criterion validity requires a gold-standard tool or a short version of the instrument, it is often difficult to find a verification tool [41]. Accordingly, more studies establish hypotheses about relevant variables based on prior research and then verify those hypotheses. In this study, quality assessments for hypothesis testing were conducted, with 14 studies meeting the criteria, while four did not provide any hypotheses.

Only five studies met the quality assessment criteria for cross-cultural validity. Given the nature of digital-related topics, differences in results by age are expected, so differential item functioning analysis can be conducted to determine whether there are differences in results by gender, nationality, and language, excluding age [35]. However, in most cases, the relevant content was either missing or insufficient. Some studies considered and reported on floor and ceiling effects, but only six studies did so, indicating a need for more active reporting.

One limitation of this study is the potential for publication bias, as it cannot rule out biases from sources such as dissertations, conference presentations, and research reports. Furthermore, this study does not include any questionnaires other than eHEALS, limiting opportunities for a comprehensive quality appraisal and restricting the potential to make consistent recommendations for specific measurement tools.

We recommend using measurement tools such as A6, A7, A9, and A19-21, which have shown strong methodological evaluation results in various studies [15,23,24,27,33,34]. Additionally, conducting systematic reviews that compile the findings from these studies is recommended. As more research is conducted, quality appraisals of the same tool will become feasible. An exception is eHEALS, which was included in three studies in this review. It has demonstrated satisfactory quality in previous research [16] and is easy to apply due to its eight-item format. However, its limitation in assessing only a narrow scope of eHealth literacy should not be overlooked [11].

Today, eHealth literacy extends beyond basic digital device usage and Internet navigation. It now includes the ability to search for trustworthy information, engage in bi-directional communication [31,34], contribute to content creation within a community [26,33], and protect privacy [31,33]. Therefore, researchers should carefully select eHealth literacy measurement tools that align with their research objectives, rather than relying solely on tools that are easy to use or widely used in the past. Recent studies have incorporated these broader aspects of eHealth literacy. Notably, the number of items in six studies with a high number of sufficient (+) ratings ranged from 24 to 44, excluding shortened versions. For example, A7, which had the highest number of sufficient (+) ratings, was derived from a 47-item tool and shortened to 11 items. Since its validity and reliability were well-established, it should be repeatedly used and validated in future research. The field of eHealth literacy measurement is expected to expand significantly, particularly in performance-based methods [37] that go beyond self-report formats. We also recommend review studies on tools focusing on performance measurement rather than self-report formats.

Conclusions

In this study, a systematic review was conducted on questionnaires developed to evaluate eHealth literacy among adolescents and adults in community settings using the 2018 COSMIN guidelines. The review highlighted that while aspects such as construct validity, internal consistency, hypothesis testing, cross-cultural validity, criterion validity, and responsiveness were frequently verified, important areas like measurement error, test-retest reliability, and floor or ceiling effects were often overlooked. Based on these findings, we provide guidance for the selection and use of appropriate measurement tools in future research. Future studies should focus on addressing these gaps by adhering closely to the COSMIN methodology, which would enhance the robustness of eHealth literacy assessments. Additionally, psychometric evaluations should aim for comprehensive reporting of their properties to ensure the tools are applicable and reliable across diverse populations and settings.

Figure 1.

PRISMA flowchart of the search process.

References

• 1. Simonds SK. Health education as social policy. Health Education & Behavior. 1974;2(1):1–10. https://doi.org/10.1177/10901981740020S102Article
• 2. Nutbeam D. The evolving concept of health literacy. Social Science & Medicine. 2008;67(12):2072–2078. https://doi.org/10.1016/j.socscimed.2008.09.050ArticlePubMed
• 3. World Health Organization. Shanghai declaration on promoting health in the 2030 agenda for sustainable development: 9th Global Conference on Health Promotion. Shanghai, China: World Health Organization; 2016 November. Report No.: WHO/NMH/PND/17.5.
• 4. European Commission. European citizens’ digital health literacy. Brussels: European Commission; 2014. 221 p.
• 5. Nutbeam D. Health literacy as a population strategy for health promotion. Japanese Society of Health Education and Promotion. 2017;25(3):210–222. https://doi.org/10.11260/kenkokyoiku.25.210Article
• 6. U.S. Department of Health and Human Services. Healthy People 2030: Understanding and improving health [Internet]. Maryland: Office of disease prevention and health promotion. [cited 2024 Jul 31]. Available from https://health.gov/healthypeople
• 7. Park DJ, Koh KW, Lee JY. Direction of national policy for health literacy in Korea. Korean Journal of Health Education and Promotion. 2022;39(4):1–14. https://doi.org/10.14367/kjhep.2022.39.4.1Article
• 8. Bhavnani SP, Narula J, Sengupta PP. Mobile technology and the digitization of healthcare. European Heart Journal. 2016;37(18):1428–1438. https://doi.org/10.1093/eurheartj/ehv770ArticlePubMedPMC
• 9. World Health Organization. Global strategy on digital health 2020-2025. Geneva: World Health Organization; 2021. 60 p.
• 10. Norman C. Ehealth literacy 2.0: Problems and opportunities with an evolving concept. Journal of Medical Internet Research. 2011;13(4):e125. https://doi.org/10.2196/jmir.2035ArticlePubMedPMC
• 11. Karnoe A, Kayser L. How is eHealth literacy measured and what do the measurements tell us? A systematic review. Knowledge Management & E-Learning. 2015;7(4):576–600. Article
• 12. Haun JN, Valerio MA, McCormack LA, Sørensen K, Paasche-Orlow MK. Health literacy measurement: An inventory and descriptive summary of 51 instruments. Journal of Health Communication. 2014;19(2):302–333. https://doi.org/10.1080/10810730.2014.936571ArticlePubMed
• 13. Norman CD, Skinner HA. eHEALS: The eHealth literacy scale. Journal of Medical Internet Research. 2006;8(4):e27. https://doi.org/10.2196/jmir.8.4.e27ArticlePubMedPMC
• 14. Paige SR, Stellefson M, Krieger JL, Miller MD, Cheong J, Anderson-Lewis C. Transactional eHealth literacy: Developing and testing a multi-dimensional instrument. Journal of Health Communication. 2019;24(10):737–748. https://doi.org/10.1080/10810730.2019.1666940ArticlePubMedPMC
• 15. Yoon J, Lee M, Ahn JS, Oh D, Shin SY, Chang YJ, et al. Development and validation of digital health technology literacy assessment questionnaire. Journal of Medical Systems. 2022;46(2):13. https://doi.org/10.1007/s10916-022-01800-8ArticlePubMedPMC
• 16. Lee J, Lee EH, Chae D. Ehealth literacy instruments: Systematic review of measurement properties. Journal of Medical Internet Research. 2021;23(11):e30644. https://doi.org/10.2196/30644ArticlePubMedPMC
• 17. Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research. 2018;27(5):1147–1157. https://doi.org/10.1007/s11136-018-1798-3ArticlePubMedPMC
• 18. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097ArticlePubMedPMC
• 19. Délétroz C, Allen MC, Sasseville M, Rouquette A, Bodenmann A, Gagnon1 MP. Ehealth literacy measurement tools: A systematic review protocol. Systematic Reviews. 2022;11(1):205. https://doi.org/10.1186/s13643-022-02076-2ArticlePubMedPMC
• 20. Terwee CB, Jansma EP, Riphagen II, de Vet HC. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Quality of Life Research. 2009;18(8):1115–1123. https://doi.org/10.1007/s11136-009-9528-5ArticlePubMedPMC
• 21. Holch P, Marwood JR. EHealth literacy in UK teenagers and young adults: exploration of predictors and factor structure of the eHealth literacy scale (eHEALS). JMIR Formative Research. 2020;4(9):e14450. https://doi.org/10.2196/14450ArticlePubMedPMC
• 22. Sudbury-Riley L, FitzPatrick M, Schulz PJ. Exploring the measurement properties of the eHealth Literacy Scale (eHEALS) among baby boomers: A multinational test of measurement invariance. Journal of Medical Internet Research. 2017;19(2):e53. https://doi.org/10.2196/jmir.5998ArticlePubMedPMC
• 23. Kim S, Cho KW, Kim M. Development of a Korean mobile based digital health literacy instrument: validity and reliability assessment. The Korean Journal of Health Service Management. 2024;18(2):75–89. https://doi.org/10.12811/kshsm.2024.18.2.075Article
• 24. Liu HX, Chow BC, Liang W, Hassel H, Huang YW. Measuring a broad spectrum of eHealth skills in the web 3.0 context using an eHealth literacy scale: Development and validation study. Journal of Medical Internet Research. 2021;23(9):e31627. https://doi.org/10.2196/31627ArticlePubMedPMC
• 25. Niemelä R, Ek S, Eriksson-Backa K, Huotari M-L. A screening tool for assessing everyday health information literacy. Libri. 2012;62(2):125–134. https://doi.org/10.1515/libri-2012-0009Article
• 26. Rachmani E, Haikal H, Rimawati E. Development and validation of digital health literacy competencies for citizens (DHLC), an instrument for measuring digital health literacy in the community. Computer Methods and Programs in Biomedicine Update. 2022;2:100082. https://doi.org/10.1016/j.cmpbup.2022.100082ArticlePubMedPMC
• 27. Kayser L, Karnoe A, Furstrand D, Batterham R, Christensen K, Elsworth G, et al. A multidimensional tool based on the eHealth literacy framework: development and initial validity testing of the eHealth literacy questionnaire (eHLQ). Journal of Medical Internet Research. 2018;20(2):e36. https://doi.org/10.2196/jmir.8371ArticlePubMedPMC
• 28. Petrič G, Atanasova S, Kamin T. Ill literates or illiterates? Investigating the eHealth literacy of users of online health communities. Journal of Medical Internet Research. 2017;19(10):e331. https://doi.org/10.2196/jmir.7372ArticlePubMedPMC
• 29. Seçkin G, Yeatts D, Hughes S, Hudson C, Bell V. Being an informed consumer of health information and assessment of electronic health literacy in a national sample of internet users: Validity and reliability of the e-HLS instrument. Journal of Medical Internet Research. 2016;18(7):e161. https://doi.org/10.2196/jmir.5496ArticlePubMedPMC
• 30. Chinn D, McCarthy C. All Aspects of Health Literacy Scale (AAHLS): developing a tool to measure functional, communicative and critical health literacy in primary healthcare settings. Patient Education and Counseling. 2013;9(2):247–253. http://dx.doi.org/10.1016/j.pec.2012.10.019Article
• 31. Koopman RJ, Petroski GF, Canfield SM, Stuppy JA, Mehr DR. Development of the PRE-HIT instrument: patient readiness to engage in health information technology. BMC Family Practice. 2014;15(18):1–9. https://doi.org/10.1186/1471-2296-15-18ArticlePubMedPMC
• 32. Scherrenberg M, Falter M, Kaihara T, Xu L, van Leunen M, Kemps H, et al. Development and internal validation of the digital health readiness questionnaire: prospective single-center survey study. Journal of Medical Internet Research. 2023;25:e41615. https://doi.org/10.2196/41615ArticlePubMedPMC
• 33. Van der Vaart R, Drossaert C. Development of the digital health literacy instrument: Measuring a broad spectrum of health 1.0 and health 2.0 skills. Journal of Medical Internet Research. 2017;19(1):e27. https://doi.org/10.2196/jmir.6709ArticlePubMedPMC
• 34. Zhang L, Li P. Problem-Based mHealth Literacy Scale (PB-mHLS): Development and validation. JMIR Mhealth and Uhealth. 2022;10(4):e31459. https://doi.org/10.2196/31459ArticlePubMedPMC
• 35. Karnoe A, Furstrand D, Christensen KB, Norgaard O, Kayser L. Assessing competencies needed to engage with digital health services: Development of the eHealth literacy assessment toolkit. Journal of Medical Internet Research. 2018;20(5):e178. https://doi.org/10.2196/jmir.8347ArticlePubMedPMC
• 36. Liu P, Wang JY, Lee ST. Relationship between levels of digital health literacy based on the Taiwan digital health literacy assessment and accurate assessment of online health information: Cross-sectional questionnaire study. Journal of Medical Internet Research. 2020;22(12):e19767. https://doi.org/10.2196/19767ArticlePubMedPMC
• 37. Kline RB. Principles and practice of structural equation modeling. 4th ed. New York: The Guilford Press; 2015. 494 p.
• 38. Tufts Medical Center. Health Literacy Tool Shed. [Internet]. Vermont: A database of health literacy measures. [cited 2024 Jul 31]. Available from https://healthliteracy.tuftsmedicine.org/
• 39. Mokkink LB, Prinsen C, Patrick DL, Alonso J, Bouter L, De Vet HC, et al. COSMIN methodology for systematic reviews of patient-reported outcome measures (PROMs). User manual, 2018.
• 40. Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: A delphi study. Quality of Life Research. 2018;27(5):1159–1170. https://doi.org/10.1007/s11136-018-1829-0ArticlePubMedPMC
• 41. Lee EH. Psychometric properties of an instrument 3: Convergent, discriminant, known-groups, and criterion validity. Korean Journal of Women Health Nursing. 2021;27(3):176–179. https://doi.org/10.4069/kjwhn.2021.08.18ArticlePubMedPMC

Appendix

Figure & Data

References

Citations

Citations to this article as recorded by  

• Current status of systematic review studies on patient-reported outcome measures published in Korean journals
  Duck-Hee Chae, Jiyeon Lee, Eun-Hyun Lee
  Research in Community and Public Health Nursing.2025; 36: 1.     CrossRef
• Neonatal nurses’ e-health literacy and technology‑mediated clinical practice: a cross-sectional analysis of digital health competencies and practice patterns
  Osama Mohamed Elsayed Ramadan, Nadia Bassuoni Elsharkawy, Alaa Hussain Hafiz, Nouran Essam Katooa, Areej Abunar, Dena Marwan A. Attallah, Mashael Dewan, Majed Mowanes Alruwaili, Abeer Nuwayfi Alruwaili, Afrah Madyan Alshammari, Nourah Alsadaan, Enas Mahro
  BMC Nursing.2025;[Epub]     CrossRef
• Understanding Patient Rights: A Pilot Study Assessing Health Literacy in Written Pre-Appointment Letters
  Julie Dalgaard Guldager, Lotte Christina Waldhauer, Carsten Kronborg Bak
  International Journal of Environmental Research and Public Health.2025; 22(10): 1518.     CrossRef
• Translation and Psychometric Validation of the Amharic eHealth Literacy Questionnaire (eHLQ): A Cross-Sectional Study (Preprint)
  Gesine Meng, Nuhamin Tekle Gebre, Abel Shita, Eyerusalem Getachew, Alemnew Destaw, Nicola Cera Schroeder, Marcus Heise, Kay Brauer, Patrick Jahn, Eva Johanna Kantelhardt, Adamu Addissie, Muluken Gizaw, Sefonias Getachew, Eric Sven Kroeber
  Journal of Medical Internet Research.2025;[Epub]     CrossRef