An international field study for the reliability and validity of the EORTC communication questionnaire EORTC QLQ-COMU26

Abstract

Background

The EORTC Quality of Life Group has developed a questionnaire to evaluate cancer patients’ perception of their communication with healthcare professionals (HCPs): the EORTC QLQ-COMU26. In this study we test the validity and reliability of this novel measure in an international and culturally diverse sample of cancer patients.

Methods

Cancer patients completed the following EORTC questionnaires at two time points (before and during treatment): the QLQ-COMU26 (including a debriefing questionnaire), the QLQ-C30, and specific IN-PATSAT32 scales. These data were used to assess: the cross-cultural applicability, acceptability, scale structure, reliability, convergent/divergent validity, known-groups validity, and responsiveness to change of the QLQ-COMU26.

Results

Data were collected from 498 patients with various cancer diagnoses in 10 European countries, Japan, Jordan and India (overall 5 cultural regions). At most, only 3% of patients identified an item as confusing and 0.6% as upsetting, which indicates that the questionnaire was clear and did not trigger negative emotional responses. Confirmatory factor analysis and multi-trait scaling confirmed the hypothesised QLQ-COMU26 scale structure comprising six multi-item scales and four single items (RMSEA = 0.025). Reliability was good for all scales (internal consistency > 0.70; test–retest reliability > 0.85). Convergent validity was supported by correlations of ≥ 0.50 with related scales of the IN-PATSAT32 and correlations < 0.30 with unrelated QLQ-C30 scales. Known-groups validity was shown according to sex, education, levels of anxiety and depression, satisfaction with communication, disease stage and treatment intention, professional evaluated, and having a companion during the visit. The QLQ-COMU26 captured changes over time in groups that were defined based on changes in the item of satisfaction with communication.

Conclusion

The EORTC QLQ-COMU26 is a reliable and valid measure of patients’ perceptions of their communication with HCPs. The EORTC QLQ-COMU26 can be used in daily clinical practice and research and in various cancer patient groups from different cultures. This questionnaire can help to improve communication between patients and healthcare professionals.

Background

Communication between patients and health professionals is key in oncological care [1,2,3].

Carlson et al. [4] present two aims of communication based on Feldman-Stewart’s theoretical framework for patient–professional communication [5]: primary aims, which are directly linked to the communication process, such as offering support [6], providing better information [7, 8], and enhancing patient education and understanding; and secondary aims, which are indirect consequences of effective communication: patient reported outcomes such as psychological functioning [9] and Quality of Life (QoL) [10,11,12].

A shift has occurred in recent years (especially in Western countries) in models of care from a paternalistic [13], asymmetrical relationship where the professional occupied the dominant position by virtue of their specialist medical knowledge and the patient merely cooperated, to patient-centred cancer care, where patients’ preferences, experiences and needs are the main focus [14].

Patient-centred care has also been called client-centred care [15]. One major component of patient-centred cancer care (client-centred cancer care) is Patient-Centred Communication (PCC) since the relationship between the provider and the client is at the heart of ensuring that a client’s goals are being addressed and met. According to Epstein and Street [16], the six core functions of PCC comprise fostering healing relationships, exchanging information, responding to emotions, managing uncertainty, making decisions, and enabling patient self-management. Evidence supports a PCC approach in situations related to communication, such as delivering bad news and navigating difficult conversations [17].

Cross-cultural differences exist in communication between patients and professionals [18]. The communication model of some countries may be more paternalistic, whereas the model of others may be more patient-centred.

Communication between cancer patients and professionals and improving this communication is an important research field [19]. Studies on communication have addressed aspects such as PCC components (such as empathy, dealing with uncertain feelings) [20] and communication in cancer-related genetic and genomic testing [21], survivorship [22], and supportive and palliative care settings [23].

The European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Group (QLG) has developed a cancer-specific communication questionnaire – a stand-alone measure that, depending on the aims of researchers, could be administered in combination with the EORTC Quality of Life Questionnaire Core 30 (QLQ-C30). This cancer-specific communication questionnaire aims to assess some of the primary goals of communication presented by Carlson et al. [4]. It has been developed in a cross-cultural setting and is applicable across different cultures. Except for the ‘making decisions’ function, which we believe requires a questionnaire of its own [24], Epstein and Street’s core functions of the PCC model [16] also provided a starting point for developing the communication questionnaire.

We also did not consider the ‘content of information’ area of the ‘exchanging information’ function since it can be evaluated from the EORTC information questionnaire (QLQ-INFO25) [25]. This EORTC communication questionnaire is recommended for use in clinical trials, cross-cultural research in communication, daily practice [26] and clinical studies to improve communication in a centre or to evaluate new interventions in the area of communication, e.g. communication skills training, an area in which it has already been administered [12].

The EORTC QLG has implemented a four-phase methodology to develop questionnaires in a cross-cultural setting that is mainly based on the Classical Test Theory [27]. Phases I-III of the communication questionnaire have been completed. Phase I involved a search of the literature and interviews with professionals and patients to determine the key issues. Phase II involved rephrasing the issues as items. In Phase III a study was conducted to pre-test the provisional questionnaire in 140 patients from five cultural areas. A revised version of the questionnaire – the EORTC QLQ-COMU26 – was then created [28, 29].

The aim of the present study was to test the EORTC QLQ-COMU26 communication questionnaire to determine its cross-cultural applicability and acceptability as well as its psychometric properties. This represents Phase IV of the questionnaire-development process [27].

Materials and methods

Study sample

Patients were included in the study if they were 18 years of age or over, had been diagnosed with (primary or recurrent) cancer at any cancer site, their cancer was at an initial or advanced stage, they were undergoing any treatment (e.g., radiotherapy (RT) or chemotherapy (CT), having previously undergone or not undergone surgery) at any treatment line, and were able to understand the language of the questionnaire and to complete it. Patients were excluded if they had any concurrent malignancies or if they were participating in QOL trials during the period of our study since this could lead to the treatment team (professionals involved in their care) having a different structure, which could interfere with the study.

To ensure cross-cultural applicability, patients were enrolled from an English-speaking country; northern, southern, and eastern European countries; and other regions (India, Japan and Jordan) [27].

Study design

The patients recruited for this prospective longitudinal study were approached when they began a treatment line. Two main patient groups were created based on whether the treatment intent was 1) radical/curative, or 2) palliative/symptom relief (with no prospect of a cure).

Communication was assessed with doctors and nurses, whose regular timetables for patient consultations can usually be compared. We decided that patients should assess their communication mainly with doctors, since our experience in the previous phase of the questionnaire development shows that this is the general preference of patients when it comes to evaluating communication with professionals [29]. However, we also aimed to adequately represent patient communication with nurses.

Assessment time points

Patients were approached twice, i.e. at two time points related to their treatment process. The first assessment for all patients was conducted around the first day of a treatment line, while the second assessment was performed in an interview conducted with the patient either around the end of their RT or three months after their initial CT (see Fig. 1). Patients had at least one conversation with their healthcare professional between the two assessment time points. The first day of treatment is an important moment for the patients’ emotional reactions and, consequently, for the patient-professional relationship. Also on this first day of treatment, patients had not had any prior treatment-related intervention and so any interference with the current rating due to prior experience was avoided.

Fig. 1

Assessment time points. Assessment points. The first assessment and retest were common for all patients. The second assessment was organised depending on the treatment modality: radiotherapy, chemotherapy or concomitance

A subsample of patients participated in a third assessment three days after the second assessment to measure test–retest reliability and correlations between the second and third assessments were studied [30].

Patients were asked to assess their communication with one doctor or a group of nurses during the treatment period only (i.e. not at diagnosis or during follow-up). This is because, at our participating centres, while just one doctor tends to be responsible for each patient’s treatment, several nurses tend to intervene.

Questionnaires

Patients completed the provisional EORTC QLQ-COMU26 communication questionnaire, the EORTC QLQ-C30, and the scales of the EORTC IN-PATSAT32 related to communication with doctors. All questionnaires were completed at the first two time points, while the QLQ-COMU26 was also completed at a third time point in order to measure test–retest-reliability. Scores from each of the three instruments range from 0 to 100.

The EORTC QLQ-COMU26 evaluates patients’ communication with healthcare professionals. It is applicable across cancer diagnoses and treatment stages (diagnosis, treatment or follow-up), including palliative care. It allows patients to indicate which professional category they are evaluating, i.e. Doctor(s), Nurse(s), Psychologist(s), or Other professional(s). Patients also indicate which specific treatment period they are evaluating, i.e. diagnosis, treatment or follow-up. As we explain later, in the present study the patients evaluated their treatment period only.

The EORTC QLQ-COMU26 comprises 26 items divided into six scales, plus 4 individual items that mainly assess behaviours related to communication [29]. A high score means better communication on all scales and items. The structure of the questionnaire is presented in Table 1.

Table 1 EORTC QLQ-COMU26. Distribution of scale scores and individual items. Measurement properties

The EORTC QLQ-COMU26 is available in English and fourteen other language versions (each version was developed in accordance with EORTC translation procedure), including the languages of the participating centres [31]. The questionnaire and its scoring instructions are available from the EORTC at https://qol.eortc.org/form/#1.

The EORTC QLQ-C30 covers QoL aspects relevant to most cancer patients [32]. It consists of 30 items divided into five functional scales, three symptom scales, one global QoL scale, and six single items. A score of 100 indicates good QoL on the function scales, whereas on the symptom scales it indicates a heavy burden (see supplementary Table 1).

The scales of the EORTC IN-PATSAT32 patient satisfaction module [33] that assess doctors’ interpersonal skills (items 4–6) and doctors’ availability (items 10, 11) were also administered. A high score means a high level of satisfaction.

When completing the retest assessment, all patients were given an individual item to evaluate whether their general perception of their communication with the professional they had assessed had changed since the second assessment (‘yes/no’).

At the first assessment, a short debriefing questionnaire surveyed the patients’ acceptability of the QLQ-COMU26, including the time it took to complete, whether they needed help to complete it, and whether any QLQ-COMU26 items were upsetting, difficult or confusing.

Socio-demographic and clinical data were also collected at the first time point. At the second time point, data were collected on current treatment modality and whether the patient was accompanied by a significant other.

Statistical analysis

Statistical analysis focused on scale structure, reliability, and aspects of validity. Scale structure, internal consistency and descriptive analysis on scores relied on data from the first time point.

Confirmatory factor analysis was performed to confirm the hypothesised scale structure based on data from phase III of module development [29]. In this analysis, standardised factor loadings for each item on the corresponding scale were expected to be > 0.60 [34, 35].

Goodness of model-data fit indices indicated good fit if they were above 0.95 for Comparative Fit Index (CFI) and Tucker-Lewis index (TLI) and below 0.05 for Root Mean Square Error of Approximation (RMSEA) [36]. Residual correlations indicating local independence were expected to be below 0.20 [37].

Multi-trait scaling analysis was performed to further investigate scale structure [38], with items expected to correlate > 0.40 (corrected for overlap) with their hypothesised scale [39].

Internal consistency of the QLQ-COMU26 scales was determined based on Cronbach’s alpha and test–retest reliability based on intra-class correlation (ICC). For both of these parameters, correlations > 0.80 indicated good reliability and correlations > 0.90 indicated excellent reliability [39].

Convergent validity was investigated through correlations of the QLQ-COMU26 scales with the IN-PATSAT32 scales included in the study (only in patients assessing communication with doctors), while divergent validity was analysed through correlations with the QLQ-C30. Spearman rank correlations > 0.50 and < 0.30 indicated sufficient convergent [40, 41] and divergent validity [41], respectively.

Known-groups validity was performed by comparing the QLQ-COMU26 scales and items using T-tests for independent samples and one-way analysis of variance analyses of differences between patient groups defined by the QLQ-C30 emotional functioning scale and the QLQ-COMU26 satisfaction item (both dichotomized), age (< 70– ≥ 70 years), sex, education, treatment intention, professional assessed (doctor, nurse), and having a companion at the visit.

Higher scores in communication were expected on the QLQ-COMU26 in patients with higher levels of emotional functioning [42, 43], higher satisfaction with communication, older patients [44], males [45, 46], higher education level [47, 48], palliative treatment intention [49], communication with nurses [50], and having a companion at the visit [51].

Responsiveness to change between the first two assessments was tested through T-tests for paired samples. All analyses were performed using IBM SPSS Statistics 27 [52] and R-Studio [53].

The overall sample size for this study was 500 patients, 100 of whom also took part in our test–retest analyses. The calculation of these sample sizes was based on EORTC Quality of Life Group Module Development Guidelines [27].

Results

Characteristics of the sample

A total of 498 patients were recruited at 15 centres in 13 countries representing 5 cultural areas. The mean age of these patients was 62 years. Of these patients, 54.9% were females. Various education levels, cancer sites and oncology treatment modalities were represented. Details of the patients are shown in Table 2 and supplementary Table 2.

Table 2 Demographic and clinical characteristics of the sample (N = 498)

A total of 346 patients (69.5%) completed the QLQ-COMU-26 at the second assessment (201 of these patients assessed doctors while 145 assessed nurses). Reasons for non-completion were: declined (12 patients; 2.4%), deceased (28 patients; 5.6%), and administrative failure (46 patients; 9.2%). In some cases (66 patients; 13.2%), the reasons were not recorded. A total of 219 patients (44%) performed the re-test assessment (120 of these patients assessed doctors while 99 assessed nurses). All questionnaires had over 70% of their items answered at each assessment.

The total number of missing items from all the QLQ-COMU-26 questionnaires gathered at the first assessment was 86 (0.7%): the number of patients who failed to complete these missing items ranged from 0 to 13 (2.7%) per item.

Debriefing questionnaire on patient acceptability

Most patients (78%) completed the QLQ-COMU-26 in 15 min or less; 43.1% of patients were given help when doing so, with 26.9% being given practical help.

A total of 70 patients (14.3%), from 10 centres, found at least one item confusing. The highest frequencies were for item 17 (the professional listened when the patient expressed emotions), which was considered confusing by 15 patients (3.0% of the sample), and item 18 (help with managing emotions), which was confusing for 14 patients (2.8%). For both items, the patients were distributed among the various centres.

A total of 14 patients (2.9%) found at least one item upsetting. Three patients (0.6%) found item 16 (the professional tried to understand the patient’s situation) upsetting, while three patients found item 17 (the professional listened when the patient expressed emotions) upsetting. The other items were mentioned by fewer patients. See supplementary Table 3.

Scale structure

Goodness of fit measures in confirmatory factor analysis confirmed the hypothesised scale structure of the questionnaire (CFI = 1.00, TLI = 1.00, RMSEA = 0.025). All factor loadings were above 0.82 and all residual correlations but one were below 0.22. See Fig. 2 for details on the investigated model. Scale structure was also supported by the results from multi-trait scaling analysis, which showed that all items had an item-own-scale correlation above 0.40 (corrected for overlap) and that all items but four had the highest correlations with their hypothesised scale (Table 3).

Fig. 2

Confirmatory Factor Analysis (CFA) path diagram of the estimated model. Coefficients (from left to right): Residuals, Standardized Regression Weights (SRW) and Covariances between Scales. I1 to I24: number of items in the questionnaire. Sc1 to Sc6: scales of the questionnaire

Table 3 Multi-trait scaling analysis

QLQ-COMU-26 descriptive statistics and reliability

Mean scores and standard deviations of the scales and individual items at first assessment are shown in Table 1.

The percentages for floor effects were low: item 19 showed the highest floor effect (11%). All scales except two had ≥ 50% of patients at ceiling (highest ceiling effect = 69.4%).

Cronbach’s alpha coefficients for all scales were between 0.85 and 0.91, while test–retest reliability was between 0.86 and 0.92 (see Table 1).

Convergent and divergent validity

Supplementary Table 4 shows the correlations between the EORTC QLQ-COMU26 areas and the selected doctors’ items of the EORTC IN-PATSAT32 that evaluated convergent validity.

Correlations ranged from 0.50 to 0.70. The hypothesised relationships between the selected doctors’ items of the EORTC IN-PATSAT32 and the scales and items of the QLQ-COMU26 that were expected to be more conceptually related showed correlation coefficients of > 0.60 for convergent validity. The correlations between QLQ-C30 and QLQ-COMU26 areas were < 0.30 in all cases, which confirms the QLQ-COMU26 divergent validity (see supplementary Table 1).

Known-groups validity

The results of known-groups comparisons are shown in Table 4. Communication scores in all communication areas were higher in patients with higher levels of emotional functioning in the EORTCQLQ-C30 scales and higher satisfaction with communication in item 26 of EORTC QLQ-COMU26. No significant differences in communication scores were found between age-based groups.

Table 4 Construct validity: known-groups comparisons

Males showed higher communication scores than females in seven QLQ-COMU26 areas. Patients with lower education levels showed higher communication scores than those with higher education levels (nine QLQ-COMU26 areas). Better management of the patient’s emotions by professionals and greater satisfaction with communication were shown in patients who received treatment with palliative intention. Communication with nurses showed higher scores than communication with doctors in ten QLQ-COMU26 areas. Sometimes having a companion during the visit showed higher communication scores than having no companion in four QLQ-COMU26 areas, while sometimes having a companion showed higher sores than always having a companion in three QLQ-COMU26 areas.

Responsiveness to change

Mean changes between the two assessment points for the whole sample showed significant differences in three areas ranging from 2.5 to 3 points but the Effect Size showed no effect (both were lower than 0.2). [54]

For a more in-depth analysis, three groups (from the whole sample) were created based on changes (or no changes) in the score of item 26 (satisfaction with communication improved, remained stable or worsened). These groups were studied independently. Patients whose score in satisfaction item 26 worsened showed significant worsening in nine QLQ-COMU26 areas (between 16.1 and 25.4 points, with medium to large Effect Size in all of these scales). Patients whose score remained stable showed just two significant differences towards worsening (between 2.4 and 2.9 points), but the Effect Size showed no effect (both were lower than 0.2). Patients whose score in satisfaction item 26 improved showed significant improvement in nine QLQ-COMU26 areas (between 14.7 and 24.3 points, with medium to large Effect Size) (see Table 5).

Table 5 Sensitivity to change on QLQ-COMU26 scales and individual items for patient subgroups

Discussion and conclusion

Discussion

This study evaluated the psychometric properties of the EORTC Communication questionnaire EORTC QLQ-COMU26 in a large international multilingual and multicultural sample of cancer patients.

The demographic and clinical characteristics of the sample may be considered adequate since sex, age groups, main tumour sites and treatment modalities were widely represented. The variety of countries and cultural areas included support cross-cultural validity.

Patients’ acceptance of the QLQ-COMU26 was high. The low percentage of patients who found any item confusing was distributed across various countries, which indicates that the questionnaire was well understood. The low percentage of patients who found any item upsetting indicates that there were no major problems with the questions and that the questionnaire did not trigger negative emotional responses.

The hypothesised structure of the questionnaire was confirmed in the confirmatory factor analyses and multi-trait analysis.

The distribution of scores in the QLQ-COMU26 questionnaire areas was adequate since they covered the whole range of communication levels. Despite having a high ceiling effect, the fact that the response range is from 0 to 100 indicates that the questionnaire is able to discriminate even if the communication level is low or high.

The internal consistency of the questionnaire scales was very positive since all scales had good or excellent Cronbach’s alpha levels. Test–retest reliability was satisfactory and in line with the high percentage of patients who indicated that their general perception of their communication with their medical professional did not change between the second and the third assessment.

Results of convergent validity analyses were satisfactory, which indicates that the contents of the QLQ-COMU26 and those of the IN-PATSAT32 communication with doctor areas are related. Correlations between the scales and items of the QLQ-C30 and the QLQ-COMU26 supported the questionnaire’s divergent validity since the concepts assessed by the two instruments were shown to be weakly related.

Results of group comparisons were satisfactory and support the questionnaire’s known-groups validity: higher communication scores in patients with better emotional functioning were also found in other studies [42] and may be related to the idea that better communication with the professional helps to achieve a more positive management of emotional reactions [43]. As hypothesised, higher communication scores were found in patients with greater satisfaction with their communication with the professional. Contrary to our hypothesis but in line with other studies [55], no differences in communication were found between age-based groups. This lack of differences could be because elderly cancer patients are not a homogenous group when it comes to communication with medical professionals due to the influence of other variables, such as perceived competence in communication with doctors [56, 57]. As in other studies, males reported higher communication scores [45, 46]. These sex differences may be related to factors such as differences in communication preferences: for example, gender differences have been found in aspects such as preferences for being a more active participant in patient-physician communication or the amount of information received [58, 59].

Unlike in the present study, others found higher communication scores in patients with higher education levels [48]. It would be interesting to use multivariate analysis to examine these differences in patients with different education levels in our study to determine whether factors such as age distribution or cultural differences account for them. Better communication was found in patients with palliative treatment intention. This may be related to greater attention being paid to psychosocial aspects offered to patients who receive treatment with a palliative rather than a curative intention [49]. As other studies also found, communication with nurses showed higher scores than communication with doctors [50].

As other studies also found [51], sometimes having a companion during the visit showed higher communication scores than having no companion. We also found that sometimes having a companion showed higher sores than always having a companion. This may be because patients have the chance to talk more openly with their professional about their disease and treatment when they are alone.

The results of responsiveness to change may be considered satisfactory when the sample is divided into three groups: patients whose level of satisfaction worsened or improved showed significant worsening or improvement in a large number of areas and with a high effect size. On the other hand, significant changes appeared in few questionnaire areas and the magnitude and effect size of those differences were very small in patients whose satisfaction did not change. We believe that several factors, especially the fact that no intervention was conducted to improve communication, may have influenced the limited differences in the global sample.

A strength of this study is the participation of a large international group of unselected patients. This reflects usual clinical practice in countries from different cultural areas. A limitation, however, is that it would have been helpful to include more patients from Eastern Europe and English-speaking countries. Also, there was no external observation of the communication between patients and professionals that could be compared with the patients’ scores on the communication questionnaire.

It would be interesting to perform future psychometric studies with patients who assess their diagnoses, follow-up periods and communication with other medical professionals such as psychologists, physiotherapist and RT technicians. It would also be helpful to perform psychometric studies with just one professional group, such as nurses.

It would also be useful to perform studies on cross-cultural differences in communication as differences have also been found in information among cultural areas [60].

Conclusion

In conclusion, the overall positive results in this large linguistically, culturally and clinically diverse sample of cancer patients support the psychometric properties of the EORTC QLQ-COMU26. This questionnaire is a valid measure of patients’ perceptions of their communication with professionals. The EORTC QLQ-COMU26 is a cancer-specific instrument that has been internationally developed. The views of cancer patients and healthcare professionals as well as cross-cultural differences have been considered in the development of the instrument. The EORTC COMU26 can be used in daily clinical practice and research and in various cancer patient groups from different cultures. This questionnaire can provide guidance on how to improve communication between patients and healthcare professionals.