Questionnaire Development
Our qualitative research conducted in Singapore identified 29 QOL themes in family caregivers of patients with advanced cancers [7]. We used this as the basis for the generation of questionnaire items. A panel consisting of 7 investigators, including two health outcomes researchers (YBC and HLW), two physicians (GY and SN), a clinical psychologist (IT), a social worker (GLL), and a linguist/translator (WC), generated 54 questionnaire items based on the themes identified in the qualitative study to form the draft version of the new questionnaire. Six of the 7 panelists were bilingual in English and Chinese. The English and Chinese versions were developed simultaneously, instead of developed in one language and then translated into another. The co-development focused on making the semantics of the two language versions comparable. The Chinese version is in simplified Chinese characters. A 5-point scale – Not at All (0), A Little (1), Somewhat (2), Quite a Lot (3) and Very Much (4) – was employed. Four items included a “Not Applicable” response: one of them was about support from religious groups and the other three were about employment. This response was included because in the Singapore population it is not uncommon to have no religious affiliation and many family caregivers were expected to be homemakers or have retired.
Study Setting
The National Cancer Center Singapore (NCCS) is the largest public provider of cancer care in Singapore, with about 152,000 outpatient clinic attendances per year in 2016/2017 [11]. Cancer patients who required inpatient care are admitted to the Singapore General Hospital (SGH). The Singapore Health Services Centralized Institutional Review Board approved the study (#2016/2243).
Pilot Study
A pilot study was conducted to assess the readability of the scale. Six English-speaking and six Chinese-speaking caregivers of patients with advanced solid cancers were recruited from the NCCS. The draft version of the caregiver QOL scale was administered. Open-ended questions were included in the questionnaire package to seek feedback on the readability of the questions and on whether there were other important QOL concerns that had not been covered by the scale. The interviewers were trained to probe for feedback. Based on the pilot study, 3 items were modified in both the English and Chinese versions to improve semantic clarity. For example, the item “I feel appreciated as a caregiver” was modified to “I feel appreciated as a caregiver by the patient” in both the English and Chinese versions. Furthermore, 5 items in the Chinese version were modified for choice of words/grammar that did not affect the semantics. For example, the Chinese phrase lijia (leave home, in Chinese phonetic symbol) was replaced by the phrase chumen (get out of the door), which is more ordinary language. The respondents indicated sufficient coverage of the scale. No item was added or dropped after the pilot study.
Validation Study
Study design and measurements
The study comprised a baseline and a follow-up survey of caregivers of patients with advanced solid cancers. The baseline survey included the new caregiver QOL scale, the Brief Assessment Scale for Caregivers (BASC), which is a multi-factor measure of caregiver outcomes [12], two items on financial concerns from a modified version of the Caregiver Reaction Assessment (CRA) for use in Singapore [13], and a demographic, caregiving and health background section, which included a caregiver rating of patient’s performance status. The performance status score ranged from 0 (without symptoms) to 4 (bedridden), excluding the score 5 (death), which was not applicable in the baseline survey [14]. When patients were at the study sites to receive medical care, accompanying caregivers were invited to participate in the study. The research aims and procedures, which involved a baseline and a follow-up survey, were explained to the participant in the language they preferred (either English or Chinese). Written informed consent was obtained from all participants prior to the baseline survey. Consented caregivers were invited to self-administer the questionnaire. Sixty one caregivers requested interviewer-administration.
The purpose of the follow-up survey was to assess test-retest reliability. The follow-up survey comprised the caregiver QOL scale, a question on the caregiver’s self-perceived change in QOL since the baseline survey on a 7-point scale [15], and a question on the patient’s performance status. The questionnaire together with a postage-paid return envelope was sent to the caregivers about one week after the baseline interview. Only participants who had self-administered the baseline survey were included.
Eligibility
Family caregivers of patients with advanced cancers who were receiving care from the outpatient clinics of NCCS or oncology wards of SGH were recruited. In this study, we defined a family caregiver as a family member who was taking direct care of the patient’s day-to-day and healthcare needs, or ensuring provision of care to meet the needs, or who was the decision maker with regard to the patient’s needs and healthcare. Participants must be 21 years of age or older, able to communicate in either English or Chinese (Mandarin), and aware of the patient’s diagnosis, and the patients must have stage III or IV solid cancers. Caregivers in the bereavement stage were not recruited. Only one eligible caregiver was recruited per patient. If there was more than one eligible caregiver willing to participate, we recruited the caregiver who was most involved in the care of the patient.
Statistical Analysis
Baseline data
All the QOL items and the items in the BASC and CRA were recoded so that a higher score indicated a better outcome. We conducted exploratory factor analysis (EFA), with Quartimin rotation, using robust least squares method for data with missing values (ULSMV) to include all 612 participants [16]. The Root Mean Square Error of Approximation (RMSEA) and Comparative Fit Index (CFI) were used for model selection and assessment of goodness-of-fit [17, 18]. While there is no golden rule to determine what cutoff values are optimal, we plotted reference lines of RMSEA 0.05 and CFI 0.95, which are often discussed in the literature, to facilitate graphical inspection [17, 18]. Furthermore, we conducted a parallel analysis [19, 20], with 200 simulation replicates, based on 5-point scale data and polychoric correlation. The probability distribution of the 5 categories followed that of our baseline data pooled across items. The observed eigenvalues were compared against the 95th percentile obtained from the simulated data [20].
Using the solution from the EFA, we conducted confirmatory factor analysis (CFA) to test for group invariance in factor loadings between ethnic Chinese and non-Chinese among participants who used the English version of the questionnaire package [16]. We also tested group invariance between the English- and Chinese-speaking samples among the ethnic Chinese participants. Furthermore, the sample consisted of two major groups in terms of relationship with patients: Spouses and adult children of the patients. These two groups are of interest because they do not only differ in terms of relationship with the patients, but also in terms of age and possibly (unobserved) covariates such as experience with critical life events. So we also tested for group invariance between the two groups of caregivers.
Upon finding a satisfactory factor structure, the “half rule”, also called “simple mean imputation”, was used to replace item non-responses [21]. The QOL domain scores were calculated as the simple mean of the relevant item scores, which were on the 0 to 4 scale, and then multiplied by 25 to re-scale them to the 0 to 100 scale. The QOL total score was a weighted average of the QOL domain scores, using the number of items in the domains as the weights. It is equivalent to a simple summation of all the item scores after applying the half-rule and rescaling to the 0 to 100 scale.
The BASC and its 5 factor scores, the sum of the scores on the two financial concerns items from the CRA, referred to as CRA (Finance) in this report for brevity, and patient’s performance status were used as validity criteria. Pearson’s correlation coefficient (r) was calculated between the QOL scores and BASC and CRA (Finance) scores to assess convergent/divergent validity. Analysis of variance was used to compare groups defined by patients’ performance status ≤ 1 versus ≥ 2 to assess known-group validity. Cronbach’s alpha was used to determine internal consistency.
Follow-up data
Participants who had returned the follow-up survey within 28 days of the baseline survey, whose patients had not passed away by the date of filling in the follow-up questionnaire, and who had reported no change in self-perceived QOL and patient’s performance status were included in test-retest reliability assessment.
Sample size determination
For validity assessment by evaluation of the Pearson’s correlation coefficients between QOL scores and validation criteria, i.e. BASC and CRA (Finance) scores, a sample size of 200 per language gave over 80% power, at 5% 2-sided type 1 error rate, to detect a correlation coefficient of 0.3 against a trivial correlation coefficient of 0.1 (PASS software, version 13). Further considering the recommendation of Comrey and Lee that, for factor analysis, a sample size of 300 is “good” [22], we targeted a sample size of about 300 per language.