Skip to main content
Download PDF
Download ePub

Gothenburg Empowerment Scale (GES): psychometric properties and measurement invariance in adults with congenital heart disease from Belgium, Norway and South Korea

Health and Quality of Life Outcomes volume 20, Article number: 145 (2022) Cite this article

Abstract

Background

Patient empowerment is associated with improvements in different patient-reported and clinical outcomes. However, despite being widely researched, high quality and theoretically substantiated disease-generic measures of patient empowerment are lacking. The few good instruments that are available have not reported important psychometric properties, including measurement invariance. The aim of this study was to assess the psychometric properties of the 15-item Gothenburg Empowerment Scale (GES), with a particular focus on measurement invariance of the GES across individuals from three countries.

Methods

Adults with congenital heart disease from Belgium, Norway and South Korea completed the GES and other patient-reported outcomes as part of an international, cross-sectional, descriptive study called APPROACH-IS II. The scale’s content (missing data) and factorial validity (confirmatory factor analyses), measurement invariance (multi-group confirmatory factor analyses), responsiveness (floor and ceiling effects) and reliability (internal consistency) were assessed.

Results

Content validity, responsiveness and reliability were confirmed. Nonetheless, metric but not scalar measurement invariance was supported when including the three countries, possibly because the scale performed differently in the sample from South Korea. A second set of analyses supported partial scalar invariance for a sample that was limited to Norway and Belgium.

Conclusion

Our study offers preliminary evidence that GES is a valid and reliable measure of patient empowerment in adults with congenital heart disease. However, cross-country comparisons must be made with caution, given the scale did not perform equivalently across the three countries.

Registration ClinicalTrials.gov: NCT04902768.

Background

Patient empowerment is a concept associated with increasing peoples’ ability to manage their condition and their lives [1]. It initially emerged in the health promotion field and later demonstrated relevance in the care of persons with chronic conditions (CCs) [2]. It has been suggested that by increasing patient empowerment, patients may eventually become more actively engaged in their care, develop more disease-related knowledge, and improve their quality of life and well-being [3].

There is a considerable amount of research on patient empowerment, both theoretically and empirically [4]. However, to date, there is no consensus on how to define patient empowerment or which instrument is best at capturing this construct. There are approximately 30 different definitions available and more than 40 instruments [4]. Moreover, a detailed assessment of current instruments has indicated that a vast majority lack a clear theoretical ground and have poor validity and reliability [5, 6].

Given the methodological limitations of existing instruments, a new measure, the Gothenburg Young Persons Empowerment Scale (GYPES), was developed [7]. Scale construction reflected the theoretical work of Small and colleagues who defined patient empowerment as “an enabling process or outcome arising from communication with the healthcare professional and a mutual sharing of resources over information relating to illness, which enhances the patients’ feelings of control, self-efficacy, coping abilities and ability to achieve change over their condition” [8]. From semi-structured interviews with adults with CCs, they concluded that patient empowerment comprises five dimensions [8]. First, knowledge and understanding, related to the level of knowledge patients need to manage their illness and lives. Second, personal control, given that each patient should have the ability to manage their disease. Third, identity, which entails how much the illness influences the patients’ lives and their sense of self. Fourth, shared decision-making, the ability and possibility to make decisions together with the healthcare provider. Fifth, enabling others, referring to the ability to share experiences and coping strategies with other persons who are experiencing a similar situation.

GYPES was developed for use with adolescents with CCs because at the time there were no existing instruments developed specifically for this age group. The scale’s content and factorial validity, internal consistency, and responsiveness were evaluated in two samples of young persons with congenital heart disease (CHD) and type I diabetes. GYPES was found to have good psychometric properties in these groups [7].

Given that the GYPES was found to be valid and reliable for its use with adolescents with CCs, and because it was based on sound conceptual grounds, we developed a slightly modified version to measure empowerment in adults with CCs: the Gothenburg Empowerment Scale (GES). In the evaluation of the scale’s psychometric properties reported herein, we also chose to investigate measurement invariance, given potential use of the GES in international studies. Measurement invariance, also known as measurement equivalence, is often neglected in psychometric evaluation [9], yet it is important to investigate whether an instrument measures the same construct in different contexts. If invariance is not supported, this suggests that the instrument triggers different response mechanisms in different groups, making score comparisons invalid [10]. Invariance is particularly important if cross-cultural comparisons are to be made [11].

Therefore, the aim of this study was to assess the psychometric properties of the GES, including the level of measurement invariance among adults with CHD from different cultures.

Methods

Development of GES

Development of the GES was modeled after the 15-item GYPES, which includes three items each to assess the five dimensions of patient empowerment: (1) personal control; (2) knowledge and understanding; (3) identity; (4) shared decision-making; and (5) enabling others [8]. Scale items are purposefully disease-generic, to facilitate use within different populations of CCs. Originally created in English, the GYPES has currently been translated to other languages including Swedish, Dutch, Mandarin, and Turkish. Additional information on the development and evaluation of GYPES has been published [7].

The GES includes the same five dimensions and number of items as GYPES. The only modification was changing the term “young persons” by “persons” throughout the scale. To translate the GES to Dutch, Norwegian and Korean for the present study, the translation process followed the guidelines from the World Health Organization [12]. This process entailed a forward-backward translation, pre-testing the translated questionnaires in a few patients, proofreading the questionnaire and finalizing the translation. To assure consistency, substantial changes from the English version were not permitted.

Design

This methodological study is part of a larger international, cross-sectional descriptive study known as APPROACH-IS II, which aims to increase the understanding of patient-reported outcomes in adults with CHD, by enrolling adults with CHD from 32 different countries across the world [13]. APPROACH-IS II is registered at ClinicalTrials.gov: NCT04902768.

Sample

APPROACH-IS II is enrolling participants from over 50 adult CHD centers worldwide. Data collection is scheduled to be finished by the end of August 2022. For the present study, data from participants of Belgium, Norway and South Korea were included, since the data collection of these three countries was undertaken before the covid-19 pandemic.

Participants were eligible for the study if they fulfilled the following criteria: (i) diagnosis of CHD; (ii) aged 18 years or older at the moment of inclusion; (iii) diagnosed before the age of 10 years, (because we wanted participants to have experience living with CHD); (iv) in follow-up at an adult CHD center or included in a national/regional registry; and (v) having the physical, cognitive and language abilities to complete the self-report questionnaires. Patients with prior heart transplantation were ineligible.

In the present study, there are a total of 850 people enrolled, 497 were from Belgium, 144 from Norway, and 209 from South Korea. The median age for the total sample was 30 years, and the proportion of men and women was fairly equal. Moderate CHD was found in 56.4% of the participants. Demographic and clinical information of the included participants is detailed in Table 1. There were significant differences in age (p < 0.001, eta square: 0.044) and complexity (p < 0.001, Cramer’s V: 0.227) between the samples, with moderate effect sizes. Indeed, the Norwegian sample was slightly older, and the proportion of severe CHD was larger in South Korea.

Table 1 Sociodemographic and clinical aspects
Full size table

Procedure

Participants completed a set of self-reported questionnaires, including the GES, that were administered during outpatient clinic visits (in Norway, South Korea and Belgium) and/or mailed to their home (in Belgium). Data for this sub-study were collected between August 2019 and February 2020, hence before the COVID-19 pandemic emerged.

Statistical analyses

The psychometric evaluation of GES entailed an assessment of the content validity, factorial validity, measurement invariance, internal consistency and responsiveness. Content validity was assessed by the proportion of missing values and invalid scores [7]. While this is not a common approach for evaluating content validity, missing values can be considered an indicator of whether the items in the scale are perceived as relevant by the participants and whether the items are intelligible. Less than 5% missing data was deemed acceptable in this study.

Factorial validity was evaluated through confirmatory factor analyses (CFA) to assess the hypothesized five-factor structure of the scale and the overall factor of patient empowerment [14]. CFA was performed for the entire sample. A good model fit was obtained if the comparative fit index (CFI) was > 0.90, standardized root mean square residual (SRMR) < 0.08 and root mean square error of approximation (RMSEA) < 0.08 [15]. These fit indices were chosen based on Kline’s suggestion for fit evaluation of CFA [14]. Standardized factor loadings are also reported.

Measurement invariance was examined through multigroup CFA (MGCFA) across countries [11]. The MGCFA included three models: (1) configural model (i.e., separates the sample into three subgroups, but no parameter constraints are imposed); (2) metric model (i.e., constrains the factor loadings to be equal across subgroups); and (3) scalar model (i.e., constrains the factor loadings and the item intercepts to be equal across subgroups) [16]. These models are tested sequentially and the process begins with a configural model that is well-fitting. Measurement invariance is based on how well the model fits the data as indicated by the fit indices mentioned before (i.e., CFI, RMSEA, SRMR). Additionally, change fit statistics are used to determine whether measurement invariance is present or not. This change refers to how the fit indices increase or decrease as more constraints are added. As per current recommendations, decreases below 0.01 in CFI (∆CFI) and increases below 0.015 for RMSEA (∆RMSEA) and below 0.03 for SRMR (∆SRMR) are deemed acceptable [17]. If measurement invariance was not achieved at some point, modification indices were assessed to determine whether model fit could be improved or partial invariance could be established [18]. Partial invariance occurs when some items that are different across groups are estimated freely, while keeping at least two indicators per latent construct to be equal across groups [9, 10, 19]. For the GES, this meant that only one item per dimension could be freed to attempt to establish partial invariance. If items were released, this was done following a backward method, based on the items with the highest modification indices [10]. Lastly, if no configural, metric or scalar invariance could be achieved, the factor structure was assessed separately in each group [18].

Internal consistency was assessed by calculating the Cronbach’s alpha coefficient [20]. Coefficients were calculated for each dimension and for the overall scale. Besides Cronbach’s alpha coefficients, composite reliability values were calculated as an additional method to evaluate the inter-item consistency of the scale. The Cronbach’s alpha value as well as the composite reliability values should be above 0.70 to be considered acceptable [21]. Floor and ceiling effects were calculated as a way to assess issues regarding responsiveness. This is an indirect way to evaluate a scale’s sensitivity to detect change [22]. Floor and ceiling effects were considered present if more than 15% of the participants achieved the lowest (i.e. 15) or highest score (i.e. 75) [22].

Statistical analyses were performed with the Lavaan package in R [23] and IBM SPSS Statistics for Windows version 27.

Ethics and informed consent

APPROACH-IS II has its coordinating center at KU Leuven, Belgium. Therefore, ethical approval was granted from the Institutional Review Board of the University Hospitals Leuven/KU Leuven. Additionally, ethics approval was granted by the local ethics committees of the included centers (i.e., Norway and South Korea). All participants included in this study provided verbal and written informed consent.

Results

Content validity

The proportion of missing values ranged from 0 to 0.8% (Table 2). “Knowledge and understanding” was the dimension with the highest proportion of missing values.

Factorial validity

The five-factor structure as well as the overall factor of patient empowerment of the GES were evaluated through CFA in the entire sample. The five-factor model had an acceptable model fit based on the fit indices (x2(80) = 326.296; CFI = 0.948; RMSEA = 0.060; and SRMR = 0.039). Factor loadings for this model ranged from 0.515 to 0.864 and were all significant with p < 0.001 (Table 2). Item 6 from the “personal control” dimension had the lowest factor loading (i.e., 0.515). A second-order factor model to test the overall construct of patient empowerment also showed an acceptable model fit ( x2(85) = 358.916; CFI = 0.942; RMSEA = 0.062; and SRMR = 0.044). The first-order factor loadings in this model ranged from 0.523 to 0.863 and all were significant (p < 0.001). The second-order factor loadings had the following values: 0.903 (knowledge and understanding), 0.859 (personal control), 0.638 (identity), 0.773 (shared decision-making) and 0.583 (enabling others).

Table 2 Proportion of missing values and factor loadings for GES
Full size table

Measurement invariance

The configural model had a CFI (0.899) and RMSEA (0.085) near the cut-off values for an acceptable model (Table 3). To improve model fit, modification indices were evaluated, and based on this, the residuals of items 6 and 15 were allowed to covary. Even though these items belong to different dimensions (personal control and enabling others, respectively), it is reasonable to expect that persons who are actively involved in their care, also feel more capable of sharing their experiences with others [7]. A second configural model was evaluated with this error correlation, and model fit indices reached the expected threshold (x2(252) = 695.911; CFI = 0.913; RMSEA = 0.079; and SRMR = 0.061). By achieving a well-fitting configural model, we proceeded to test metric and scalar invariance, also including this covariation.

The metric model fitted the data well (Table 3). Additionally, changes in model fit indices (∆CFI 0.005; ∆RMSEA 0.002; ∆SRMR 0.006) were within the expected values, indicating metric invariance was supported. An evaluation of scalar invariance came along with slightly worse model fit indices (Table 3). In comparison to the metric model, fit indices were not within the allowed change range (∆CFI 0.034; ∆RMSEA 0.010; ∆SRMR 0.007). Therefore, full scalar invariance could not be established.

Given the lack of full scalar invariance, we proceeded to test partial scalar invariance by evaluating modification indices and the equality constraints across groups. Constraints were relaxed sequentially until an acceptable fit was achieved. Fixing the intercept of item 9 (identity) contributed the most to the observed misfit across groups, so this item was estimated freely in a partial scalar invariance model. While model fit improvements were obtained, the indices were below the acceptable threshold. Therefore, a new partial scalar invariance model was tested with the intercepts of items 9 (identity) and 4 (personal control) set free. While improvements were identified (Table 3), model fit indices were still not within an acceptable range. We continued to release items sequentially to achieve acceptable model fit. Models were tested with intercepts of items 9 (identity), 4 (personal control), 12 (shared decision-making) and 2 (knowledge and understanding) unconstrained and while fit indices were almost within the recommended range, it was not possible to achieve acceptable model fit. While models with unconstrained items from the “enabling others” dimension were assessed, none of them led to changes in the model fit indices. Additionally, given that not more than one item per dimension could be estimated freely, models with more than 5 free items where not tested. Therefore, partial scalar invariance was rejected.

Since partial scalar invariance was not achieved, an assessment of the three groups independently through CFA was undertaken to understand why this was the case. While fit indices for Norway (x2(85) = 117.694; CFI = 0.964; RMSEA = 0.053; and SRMR = 0.074) and Belgium ( x2(85) = 348.280; CFI = 0.916; RMSEA = 0.079; and SRMR = 0.053) indicated the model fitted these groups well, it appears this was not the case for the data from South Korea (x2(85): 307.753; CFI: 0.788; RMSEA: 0.112; and SRMR: 0.079). Factor loadings for this country ranged between 0.407 and 0.772, with low factor loadings in the “shared decision-making” dimension, though significant (p < 0.001).

Measurement invariance: norwegian and belgian samples

As the CFA models for Norway and Belgium had an acceptable model fit. An evaluation of measurement invariance within these two countries was undertaken. Configural invariance was supported by a well-fitted model (x2(170) = 463.510; CFI = 0.928; RMSEA = 0.074; and SRMR = 0.057). An evaluation of metric invariance also showed acceptable model fit indices (Table 3) and changes in them were also within the acceptable ranges (∆CFI 0.001; ∆RMSEA 0.002; ∆SRMR 0.004). Given that metric invariance was supported, scalar invariance was evaluated next. Fit indexes for this model were within the recommended values (Table 3). However, changes in the CFI were outside of the acceptable range to support measurement invariance (∆CFI 0.013; ∆RMSEA 0.004; ∆SRMR 0.002). Therefore, full scalar invariance was not supported. Modification indices indicated that freeing the intercept of item 11 (shared decision-making) could lead to model improvements. Hence, partial scalar invariance was evaluated with the intercept of item 11 free. This model had acceptable model fit indexes (x2(192) = 519.18; CFI = 0.920; RMSEA = 0.073; and SRMR = 0.063). Additionally, in comparison to the metric model, fit indices suggested worse model fit, but changes were still within acceptable ranges (∆CFI 0.007; ∆RMSEA 0.002; ∆SRMR 0.001). These results support partial scalar invariance for the samples of Norway and Belgium.

Table 3 Model fit indexes of multi-group confirmatory factor analyses
Full size table

Internal consistency

The Cronbach’s alpha for the overall scale was 0.873, indicating that it was internally consistent. The alpha values for the subscales were: 0.73 (knowledge and understanding); 0.693 (personal control); 0.680 (identity); 0. 774 (shared decision-making); and 0.863 (enabling others). The Cronbach’s alpha for each country is given in Table 4. An evaluation of the scale’s composite reliability showed that most of the values are above the expected range of 0.7 and those below such threshold were relatively near to this acceptable value.

Responsiveness

The mean patient empowerment score for the entire sample was 59.36 ± 8.47. None of the participants had the lowest score (i.e., 15) and only 2.5% had the highest attainable score of 75. Hence, no floor or ceiling effects were identified.

Table 4 Cronbach’s alpha values, composite reliability and mean empowerment scores
Full size table

Discussion

Recent research highlights the need to improve and evaluate patient empowerment in persons with CCs [4]. However, the availability of instruments with a strong theoretical background and/or acceptable psychometric properties is limited [6]. Therefore, the present study evaluated a slightly modified version of a previously validated patient empowerment in a group of adults with CHD. Results indicate that GES proved reliable, meaning the scale’s items altogether are consistent. Additionally, no floor or ceiling effects were identified, which is indirectly related to the scale’s sensitivity to measure change [22]. Whereas these results are in favor of the GES, a psychometric assessment of the scale’s validity revealed that the scale appears valid in two of the three included countries.

Results from the CFA of the entire sample indicate the five-factor structure fits the data well. However, due to the lack of measurement invariance, CFAs were performed separately for patients from each country. Analyses revealed that an acceptable model fit was only achieved for the samples from Belgium and Norway. For the South Korean data, the CFI and the RMSEA were above the acceptable ranges although factor loadings from the “shared decision-making” dimension were low. This is an indication that the relations among the latent constructs of GES are not in line with the data, i.e., the data do not support the model [24]. Even though it is common in CFA to conduct an evaluation of modification indexes to improve model fit, changes should be theoretically justified [25]. In this case, we could identify no strong theoretical reasons to allow for model changes that could potentially improve the model fit.

The poorly fitted model to the South Korean data might be the reason why invariance was not achieved when comparing the three countries. A lack of invariance suggests that patient empowerment may have a different meaning in different countries. Therefore, making mean comparisons of this construct across countries using the GES should be made with caution. Nonetheless, while cross-country comparisons cannot be made, it is possible to compare how the GES relates to other constructs (i.e., variables) within countries. For example, one could undertake a study to compare how the GES relates to patient functioning in Belgium, vs. how the GES relates to patient functioning in South Korea. Another potential research aim would be to determine whether changes in GES relate to psychological distress and whether these changes have the same magnitude in Belgium, Norway and South Korea.

The absence of invariance across the three countries can be an indicator of patient empowerment being interpreted differently across countries. Perhaps individuals in South Korea have different interpretations and values of patient empowerment compared to patients in Belgium and Norway, two European countries that are likely more culturally similar. It has been suggested that patient empowerment is determined by its context and that individuals within a particular environment, organization or country may have a different perception of empowerment [26]. Hence, a measure that fits all persons (or contexts), might be hard to develop. Findings from the present study partially support this notion. It is plausible to consider that different countries (and cultures) have a different understanding of patient empowerment domains, values, and skills.

Evidence on how patient empowerment may differ between countries is limited. However, authors of a systematic review concluded that although patients do share many perspectives of this construct, there might be variations on how certain aspects of patient empowerment are understood and research has yet to address structural elements of patient empowerment [27].

Consideration of structural elements is relevant for the current study because the “shared decision-making” dimension had the lowest factor loadings for the South Korean sample, as well as poor reliability. Shared decision-making involves healthcare professionals and patients working together on a care plan [28]. It entails patients who are willing to participate as well as clinicians who want to collaborate with the patient. Hence, it is greatly influenced by the healthcare structure. Within Asian cultures, for example, it is more commonly believed that patients prefer to take a less independent role and that a family-centered (rather than individual-centered) approach towards decision-making is preferred [29,30,31]. It has been suggested that shared decision-making reflects values associated with western cultures and that this might be different in other cultures [28, 31, 32]. The value placed on shared decision-making and the structure of the healthcare system might therefore potentially impact the value of patient empowerment as perceived by study participants. Therefore, the comparison of this construct between different cultures and regions of the world should be made with caution.

Although the GES appears to be a valid and reliable instrument for the Norwegian and Belgian sample, it is worth noting that future research should evaluate whether the low psychometric properties found in the South Korean sample are indeed applicable to other Asian or non-European countries. Comparatives studies evaluating this construct (and similar constructs) between Western and Asian countries are needed to comprehend this phenomenon, the types of research questions that can be addressed and the conclusions that can be drawn when evaluating outcomes such as patient empowerment. Such studies could be undertaken once the data collection from APPROACH-IS II is finalized.

Methodological considerations

There are several strengths of the study that merit mentioning. First, GES development was based on a previously validated questionnaire, which in turn has a strong theoretical foundation and followed a rigorous development process. Therefore, it is plausible to conclude that GES also has a strong theoretical foundation. Second, data were collected as part of an international study, which allows for cross-country comparisons. Third, the data were collected before the COVID-19 pandemic, therefore there was no concern about the potential for survey responses to be impacted by the pandemic [33].

There are, however, also some methodological limitations associated with this study. First, some aspects associated with validity and reliability were not assessed. For instance, it was not possible to directly evaluate responsiveness with cross-sectional data. While floor and ceiling effects are an indirect way to measure responsiveness, this is better achieved through longitudinal studies. Future longitudinal studies should evaluate this psychometric property. Second, the study only includes individuals with CHD, who were in current follow-up and who had the abilities to answer the questionnaires. Therefore, the generalizability of the results might be limited.

Conclusion

This study provides evidence on the GES’s validity, reliability, and responsiveness in adults with CHD. GES seems to be valid and reliable for the sample of Belgium and Norway. However, this is not the case for South Korea, because shared decision-making seems to have a different meaning in Asian cultures. Hence, cross-cultural comparisons using GES should be made cautiously. It is possible that in countries who are more culturally different from Norway or Belgium, the scale might not perform as well.

Data Availability

The data analyzed in the current study are available from the corresponding author on reasonable request.

Abbreviations

CCs:

chronic conditions.

CFA:

confirmatory factor analyses.

CFI:

comparative fit index.

CHD:

congenital heart disease.

GES:

Gothenburg Empowerment Scale.

GYPES:

Gothenburg Young Persons Empowerment Scale.

MGCFA:

multigroup confirmatory factor analyses.

RMSEA:

root mean square error of approximation.

SRMR:

standardized root mean square residual.

References

  1. Bravo P, Edwards A, Barr PJ, Scholl I, Elwyn G, McAllister M, et al. Conceptualising patient empowerment: a mixed methods study. BMC Health Serv Res. 2015;15(1):252.

    Article  Google Scholar 

  2. McAllister M, Dunn G, Payne K, Davies L, Todd C. Patient empowerment: the need to consider it as a measurable patient-reported outcome for chronic conditions. BMC Health Serv Res. 2012;12:157.

    Article  Google Scholar 

  3. Falk-Rafael AR. Empowerment as a process of evolving consciousness: a model of empowered caring. Adv Nurs Sci. 2001;24(1):1–16.

    Article  CAS  Google Scholar 

  4. Acuña Mora M, Sparud-Lundin C, Moons P, Bratt E-L. Definitions, instruments and correlates of patient empowerment: A descriptive review. Patient Educ Couns. 2021;105(2):346–55.

    Article  Google Scholar 

  5. Pekonen A, Eloranta S, Stolt M, Virolainen P, Leino-Kilpi H. Measuring patient empowerment – A systematic review. Patient Educ Couns. 2020;103(4):777–87.

    Article  Google Scholar 

  6. Barr PJ, Scholl I, Bravo P, Faber MJ, Elwyn G, McAllister M. Assessment of patient empowerment–a systematic review of measures. PLoS ONE. 2015;10(5):e0126553-e.

    Article  Google Scholar 

  7. Acuña Mora M, Luyckx K, Sparud-Lundin C, Peeters M, van Staa A, Sattoe J, et al. Patient empowerment in young persons with chronic conditions: Psychometric properties of the Gothenburg Young Persons Empowerment Scale (GYPES). PLoS ONE. 2018;13(7):e0201007.

    Article  Google Scholar 

  8. Small N, Bower P, Chew-Graham CA, Whalley D, Protheroe J. Patient empowerment in long-term conditions: development and preliminary testing of a new measure. BMC Health Serv Res. 2013;13:263.

    Article  Google Scholar 

  9. Wang S, Chen C-C, Dai C-L, Richardson GB. A Call for, and beginner’s guide to, measurement invariance testing in evolutionary psychology. Evol Psychol. 2018;4(2):166–78.

    Article  Google Scholar 

  10. Putnick DL, Bornstein MH. Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Dev Rev. 2016;41:71–90.

    Article  Google Scholar 

  11. Brown GTL, Harris LR, O’Quin C, Lane KE. Using multi-group confirmatory factor analysis to evaluate cross-cultural research: identifying and understanding non-invariance. Int J Res Method Educ. 2017;40(1):66–90.

    Article  Google Scholar 

  12. World Health Organization. Process of translation and adaptation of instruments. Geneva: World Health Organization; 2016.

    Google Scholar 

  13. Van Bulck L, Kovacs AH, Goossens E, Luyckx K, Zaidi A, Wang J-K, et al. Rationale, design and methodology of APPROACH-IS II: International study of patient-reported outcomes and frailty phenotyping in adults with congenital heart disease. Int J Cardiol. 2022;363:30.

    Article  Google Scholar 

  14. Kline R. Principles and practices of strucutal equation modeling. Fourth edition ed. United States: Guilford Publications; 2016.

  15. Lt Hu, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1–55.

    Article  Google Scholar 

  16. Vandenberg RJ, Lance CE. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ Res Methods. 2000;3(1):4–70.

    Article  Google Scholar 

  17. Cheung GW, Rensvold RB. Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Struct Equ Modeling. 2002;9(2):233–55.

    Article  Google Scholar 

  18. Fischer R, Karl JA. A Primer to (Cross-Cultural) Multi-Group Invariance Testing Possibilities in R. Front Psychol. 2019;10(1507).

  19. Byrne BM, Shavelson RJ, Muthén BJPB. Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychol Bull. 1989;105:456–66.

    Article  Google Scholar 

  20. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53–5.

    Article  Google Scholar 

  21. Hair JF, Ringle CM, Sarstedt M. PLS-SEM: indeed a silver bullet. J Mark Theory Pract. 2011;19(2):139–52.

    Article  Google Scholar 

  22. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42.

    Article  Google Scholar 

  23. Rosseel Y. lavaan: An r package for structural equation modeling. J Stat Softw. 2012;48(2):1–36.

    Article  Google Scholar 

  24. Lewis TF. Evidence regarding the internal structure: confirmatory factor analysis. Meas evaluation Couns Dev. 2017;50(4):239–47.

    Article  Google Scholar 

  25. Jackson DL, Gillaspy JA, Purc-Stephenson R. Reporting practices in confirmatory factor analysis: an overview and some recommendations. Psychol Methods. 2009;14(1):6–23.

    Article  Google Scholar 

  26. Zimmerman MA. Psychological empowerment: Issues and illustrations. Am J Comm Psychol. 1995;23(5):581–99.

    Article  CAS  Google Scholar 

  27. Agner J, Braun KL. Patient empowerment: a critique of individualism and systematic review of patient perspectives. Patient Educ Couns. 2018;101(12):2054–64.

    Article  Google Scholar 

  28. Ismail MA, Midin M. Shared decision-making and role preference among patients with schizophrenia in malaysia: a cross-sectional study. Front Psychiatry 2021;12(1228).

  29. Corrigan PW, Lee EJ. Family-centered decision making for east asian adults with mental illness. Psychiatric Serv. 2021;72(1):114–6.

    Article  Google Scholar 

  30. Ambigapathy R, Chia YC, Ng CJ. Patient involvement in decision-making: a cross-sectional study in a Malaysian primary care clinic. BMJ Open. 2016;6(1):e010063.

    Article  Google Scholar 

  31. Gao S, Corrigan PW, Qin S, Nieweglowski K. Comparing Chinese and European American mental health decision making. J Ment Health. 2019;28(2):141–7.

    Article  Google Scholar 

  32. Härter M, Moumjid N, Cornuz J, Elwyn G, van der Weijden T. Shared decision making in 2017: International accomplishments in policy, research and implementation. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen. 2017;123–124:1–5.

  33. Moons P, Goossens E, Luyckx K, Kovacs AH, Andresen B, Moon JR, et al. The COVID-19 pandemic as experienced by adults with congenital heart disease from Belgium, Norway, and South Korea: impact on life domains, patient-reported outcomes, and experiences with care. Eur J Cardiovasc Nurs. 2021. https://doi.org/10.1093/eurjcn/zvab120.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This study is supported by KU Leuven – University of Leuven, and the Research Foundation Flanders through grant 1154719 N and 12E9819N.

Open access funding provided by University of Gothenburg.

Author information

Authors and Affiliations

  1. Faculty of Caring Science, Work Life and Social Welfare, University of Borås, Borås, Sweden

    Mariela Acuña Mora

  2. Institute of Health and Care Sciences, University of Gothenburg, Gothenburg, Sweden

    Mariela Acuña Mora & Philip Moons

  3. School Psychology and Development in Context, Leuven, KU, Belgium

    Koen Raymaekers, Koen Luyckx & Jessica Rassart

  4. Research Foundation Flanders (FWO), Brussels, Belgium

    Koen Raymaekers, Liesbet Van Bulck & Jessica Rassart

  5. Department of Public Health and Primary Care, KU Leuven, Kapucijnenvoer 35, B-3000, Leuven, Box 7001, Belgium

    Liesbet Van Bulck, Eva Goossens & Philip Moons

  6. Faculty of Medicine and Health Sciences, Centre for Research and Innovation in Care, Division of Nursing and Midwifery, University of Antwerp, Antwerp, Belgium

    Eva Goossens

  7. UNIBS, University of the Free State, Bloemfontein, South Africa

    Koen Luyckx

  8. Knight Cardiovascular Institute, Oregon Health & Science University, Portland, OR, USA

    Adrienne H. Kovacs

  9. Department of Cardiothoracic Surgery, Department of Cardiology, Oslo University Hospital, Oslo, Norway

    Brith Andresen

  10. Department of Nursing, Samsung Medical Center, Seoul, South Korea

    Ju Ryoung Moon

  11. Congenital and Structural Cardiology, University Hospitals Leuven, Leuven, Belgium

    Alexander Van De Bruaene

  12. Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium

    Alexander Van De Bruaene

  13. Department of Paediatrics and Child Health, University of Cape Town, Cape Town, South Africa

    Philip Moons

Authors
  1. Mariela Acuña Mora
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Koen Raymaekers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liesbet Van Bulck
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eva Goossens
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Koen Luyckx
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Adrienne H. Kovacs
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Brith Andresen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ju Ryoung Moon
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Alexander Van De Bruaene
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jessica Rassart
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Philip Moons
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MAM analyzed and interpreted the data and drafted the manuscript. KR interpreted the data and substantively revised the manuscript. LVB planned the conception of the study, managed the data collection and revised the manuscript. EG planned the conception of the study and revised the manuscript. KL substantively revised the manuscript. AK planned the conception of the study, assisted in the data collection and revised the manuscript. BA planned the conception of the study, assisted in the data collection and revised the manuscript. JRM planned the conception of the study, assisted in the data collection and revised the manuscript.

AVB planned the conception of the study, assisted in the data collection and revised the manuscript. JR planned the conception of the study, assisted in the data collection and revised the manuscript. PM planned the conception of the study, secured funding, managed the data collection, interpreted the data and revised the manuscript. All authors read and approved the final manuscript.

APPROACH-IS II consortium.

Navaneetha Sasikumar (Kochi, Kerala, India, Amrita Institute of Medical Science), Junko Enomoto, Yoshiko Mizuno (Chiba, Japan, Chiba Cerebral and Cardiovascular Center), Ming Chern Leong, Izzatun Nafsi Binto Sabran (Kuala Lumpur, Malaysia, Paediatric & Congenital Heart Center, IJN), Laila Akbar Ladak, Babar Hasan, Ayat Siddiqui (Karachi, Pakistan, Aga Khan University), Ju Ryoung Moon, June Huh (Seoul, South Korea, Samsung Medical Center, Heart Vascular Stroke Institute), Hsiao-Ling Yang, Jou-Kou Wang, Chun-Wei Lu (Taipei City, Taiwan, National Taiwan University Hospital), Fatma Demir, Tuğba Öden (Bornova/İZMİR, Turkey, Ege University Health Application and Research Center), Endale Tefera, Julius Mwita (Gaborone, Botswana, Princess Marina Hospital), Jean-Claude Ambassa, Charles Mvondo, Marcel Fanka (Yaoundé, Cameroon, Clinique Medical Le Jourdain), Dejuma Yadeta (Addis Ababa, Ethiopia, Addis Ababa University Faculty of Medicine), Mohamed Leye, Khadija Gueye (Thiés, Senegal, University of Thiés, UFR des sciences de la santé), Harald Gabriel, Matthias Svhneider, Selina Seeliger (Vienna, Austria, Medical University of Vienna), Werner Budts, Alexander Van De Bruaene, Philip Moons, Eva Goossens, Liesbet Van Bulck, Jessica Rassart, Koen Luyckx (Leuven, Belgium, University Hospitals Leuven), Michele De Hosson, Julie De Backer (Ghent, Belgium, Univeristy Hospital Ghent), Anna Kaneva, Mila Markova (Sofia, Bulgaria, National Heart Hospital), Birgitte Lykkeberg, Miriam Biyai (Copenhagen, Denmark, Copenhagen University Hospital), Magalie Ladouceur, Anissa Boubrit (Paris, France, Hôpital Européen Georges Pompidou), Jean-Benoît Thambo, Amandine Ruissel, Cecile Jore (Bordeaux, France, Hôpital cardiologique Haut-Leveque), Pascal Amedro, Hamouda Abassi, Sonia Soltani, Charlene Bredy (Montpellier, France, Montpellier University Hospital), George Giannakoulas, Diamantis Kosmidis, Despina Ntiloudi, (Aristotle University of Thessaloniki, Medical School, Greece), Edward Callus, Silvana Pagliuca, Enrico Giuseppe Bertoldo, Valentina Fiolo (Milan, Italy, Policlinico San Donato & University of Milan), Maryanne Caruana, Justine Swain, Neil Grech, Rachel Xuereb, Sarah Craus, Maria Bonello, Darren Borg, Paul Farrugia, Joanne Cardona (L-Imsida, Malta, Mater Dei Hospital), Mette-Elise Estensen, Brith Andresen, Katrine Eriksen (Oslo, Norway, Oslo University Hospital, Rikshospitalet), Maria Emília Areias, Filipa Manuela Ribeiro Nunes, Ana Maria Barbosa de Bessa, Sara Mafalda Espírito Santo, Paula Brito, Marisa Pereira, Cristina Cruz, Filipe Macedo, José Carlos Areias, Joana Miranda, Fátima Pinto, Sofia Teixeira, Brenda Marques (Porto, Portugal, CHUSJ), Bengt Johansson, Camilla Sandberg, Helena Cronesten, Mikael Dellborg, Eva Mattson, Zacharias Mandalenakis, Christina Christersson (Gothenburg, Sweden, Sahlgrenska University Hospital & Umeå, Sweden, Umeå University Hospital & Uppsala, University Hospital Uppsala & Lund, Skånes Universitetssjukhus & Stockholm, Karolinska Universitetssjukhuset), Corina Thomet, Javier Ruperti (Bern, Switzerland, University Hospital Bern), Judith Bouchardy, Fanny Brand, Tobias Rutz, Coralie Blanche (Lausanne, Switzerland, Centre Hospitalier Universitaire Vaudois & Genève, Switzerland, Hôpitaux Universitaires de Genève), Joost P van Melle, Anna Boer (Groningen, the Netherlands, UMCG), Andrew Constantine, Michael A Gatzoulis, Konstantinos Dimopoulos (London, United Kingdom, Royal Brompton Hospital, Guy’s and St Thomas’ NHS Foundation Trust), Louise Coats, Debbie McParlin (Newcastle, United Kingdom, Newcastle Upon Tyne Hospitals), Paul Khairy, Anna Proietti (Montreal, Canada, Montreal Heart Institute, Université de Montréal), Jonathan Windram (Edmonton, Canada, Mazankowski Alberta Heart Institute), Susan Fernandes, Melissa Major (Stanford, California, United States, Lucile Packard Children’s Hospital and Stanford Health Care), Vaikom Subremanian Mahadevan, Thomas Nguyen, Shabir Sarwary (San Francisco, California, United States, University of California in San Francisco (UCSF)), Arwa Saidi, Leah Breault, Dalia Lopez Colon (Gainesville, Florida, United States, University of Florida Health), Ari Cedars, Shelby Kutty, Jong Ko (Baltimore, Maryland, United States, Taussig Heart Center of John Hopkins Hospital & Dallas, Texas, United States, University of Southwestern Medical Center), Anne Valente, Sarah Brainard, Jeffrey Reichman (Boston, Massachusetts, United States, Boston Children’s Hospital / Brigham and Women’s / Harvard Medical School), Ali Zaidi (New York, New York, United States, Mount Sinai Heart), Jamie Jackson, Taylor Swenski, Steven Neville (Columbus, Ohio, United States, Nationwide Children’s Hospital), Adrienne Kovacs, Lidija McGrath, Marshall Taunton (Portland, Oregon, United States, Oregon Health and Science University), Yuli Kim, Jessica Carducci, Saffy Bashey (Philadephia, Pennsylvania, United States, Philadelphia Penn Medicine and Children’s Hospital of Philadelphia), Luis Alday, Marina Lousararian, Héctor Maisuls, Marisa Quinteros, Mónica Lucero (Cordoba, Argentina, Hospital de Niños), Lucia Ortiz (Buenos Aires, Argentina, Hospital San Juan De Dios De La Plata), Fernando Amaral, Andre Schmidt (Ribeirão Preto, Brazil, Hospital das Clinicas da Faculdade de Medicina de Ribeirão Preto), Fernando Baraona Reyes (Santiago, Chile, Intituto Nacional Del Torax), John Jairo Araujo, Alberto Hernadez Súarez (Meintegral-Clinic, Manizales, Colombia), Samuel Menahem, Qifeng Wang, Pasangi Madhuka Wijayarathne (Melbourne Children’s Cardiology/Adult Congenital Heart, Monash University).

Corresponding author

Correspondence to Philip Moons.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was granted by the local ethical committees of the participating centers. Additionally, participants were asked to provide verbal and written consent.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Acuña Mora, M., Raymaekers, K., Van Bulck, L. et al. Gothenburg Empowerment Scale (GES): psychometric properties and measurement invariance in adults with congenital heart disease from Belgium, Norway and South Korea. Health Qual Life Outcomes 20, 145 (2022). https://doi.org/10.1186/s12955-022-02056-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12955-022-02056-z

Keywords

  • Adults
  • Congenital heart defects
  • Chronic conditions
  • Measurement invariance
  • Patient empowerment
  • Psychometrics
  • Validity
  • Reliability

Health and Quality of Life Outcomes

ISSN: 1477-7525

Contact us