Skip to main content
Download PDF
Download ePub

Reliability and validity of a new HIV-specific questionnaire with adults living with HIV in Canada and Ireland: the HIV Disability Questionnaire (HDQ)

Health and Quality of Life Outcomes volume 13, Article number: 124 (2015) Cite this article

Abstract

Background

Our aim was to assess internal consistency reliability, construct validity, and test-retest reliability of the HDQ with adults living with HIV in Canada and Ireland.

Methods

We recruited adults 18 years of age or older living with HIV from hospital clinics and AIDS service organizations in Canada and Ireland. We administered the HDQ paired with reference measures (World Health Organization Disability Assessment Schedule, SF-36 Questionnaire, Medical Outcomes Study Social Support Survey), and a demographic questionnaire. We calculated HDQ disability presence, severity and episodic scores (scored from 0–100). We calculated Cronbach’s alpha and Intraclass Correlation Coefficients (ICC) (Canada only) for the disability severity and episodic scores and considered coefficients >0.80 and >0.70 as acceptable, respectively. To assess construct validity, we tested 40 a priori hypotheses of correlations between scores on the HDQ and reference measures and two known group hypotheses comparing HDQ presence and severity scores based on age and comorbidity. We considered acceptance of at least 75 % of hypotheses as demonstrating support for construct validity.

Results

Of the 235 participants (139 Canada; 96 Ireland), the majority were men (74 % Ireland; 82 % Canada) and were taking antiretroviral therapy (88 % Ireland; 91 % Canada). Compared with Irish participants, Canadian participants were older (median age: 48 versus 41 years) and reported living with a higher median number of comorbidities (4 versus 1). Cronbach’s alpha for Irish and Canadian participants were 0.97 (95 % confidence interval (CI): 0.97–0.98) and 0.96 (95 % CI: 0.95–0.98), respectively, for the severity scale and 0.98 (95 % CI: 0.97–0.98) and 0.96 (95 % CI: 0.95–0.98), respectively, for the episodic scale. Of the 40 construct validity correlation hypotheses, 32 (80 %) and 22 (55 %) were supported among the Canadian and Irish samples respectively; both (100 %) known group hypotheses were also supported. ICC values for Canadian participants ranged from 0.80 (95 % CI: 0.71, 0.86) in the cognitive domain to 0.89 (95 % CI: 0.83, 0.92) in the social inclusion domain.

Conclusions

The HDQ demonstrates internal consistency reliability and a variable degree of construct validity when administered to adults living with HIV in Canada and Ireland. The HDQ demonstrates test-retest reliability when administered to adults with HIV in Canada. Further validation of the HDQ outside of Canada is needed.

Background

In developed countries such as Canada and Ireland where individuals living with HIV have access to treatment, HIV is increasingly experienced as a chronic illness. At the end of 2011, over 71,000 people were living with HIV in Canada and 7800 in Ireland [1, 2]. In both countries, the majority of new infections continue to include men who have sex with men (46 % Ireland; 47 % Canada) with 23–25 % of new diagnoses occurring among women [3]. As people with HIV in these countries are living longer, they are facing similar issues related to comorbidity and aging, such as bone and joint disorders, mental health conditions, cardiovascular disease, cancer, and neurocognitive decline [4–8]. Forming international partnerships in countries where individuals experience similar issues related to HIV and aging is essential to addressing clinical and research priorities in this emerging field.

The health-related challenges experienced with these comorbidities in addition to HIV and aging may be termed, disability. Disability is defined as a combination of physical, cognitive, mental and emotional symptoms and impairments; difficulties carrying out day-to-day activities; challenges to social inclusion; and uncertainty about future health that may be experienced as episodic in nature, fluctuating on a daily basis and over the extended course living with HIV [9].

Measuring disability in the context of HIV is important for determining the prevalence and impact of disability associated with HIV, comorbidities and aging, and for determining the effectiveness of interventions to reduce the presence and severity of disability. Dimensions of disability may overlap with other health-related concepts such health-related quality of life (HRQL) measured in the context of HIV. In an earlier content analysis we assessed the extent to which items in existing HIV-specific HRQL questionnaires captured the health-related challenges experienced by adults living with HIV according to the Episodic Disability Framework, a conceptual framework developed from the perspective of adults living with HIV [9, 10]. Many of the instruments included in this review were developed prior to the advent of combination antiretroviral therapy which dramatically changed the course of HIV infection from a progressively terminal illness to a chronic illness characterized by periods of wellness and illness [11]. Other recent developed HIV-specific HRQL questionnaires more broadly included items related to social relationships and employment [12, 13]. However these instruments were not developed to assess the episodic nature of disability; and components of HRQL such as personal goals, values and expectations go beyond the scope of disability. Furthermore, evidence demonstrating weak relationships between symptoms and function (domains of disability) and overall health-related quality of life suggest HRQL and disability while related, are distinct concepts, hence both are important to consider in the context of HIV [14, 15].

Originally developed in Canada, the HIV Disability Questionnaire (HDQ), is a self-administered instrument that describes the presence, severity and episodic nature of disability experienced by adults living with HIV [16]. Developed from the Episodic Disability Framework, the HDQ is the first known HIV-specific measure of disability developed and assessed for use with people living with HIV [9, 17]. Distinct features of the HDQ include its description of the daily episodic nature of disability, and inclusion of uncertainty as a domain of disability. The HDQ possesses sensibility, including face and content validity and ease of use for people living with HIV in Ontario [18]. However, the reliability and validity of the HDQ for use with people living with HIV in Canada and other developed countries is unknown.

Our aim was to assess the internal consistency reliability, construct validity, and test-retest reliability of the HDQ. We specifically focused on Ireland and Canada due to a funding opportunity to promote collaborative research between these two countries. Results from this study will help to establish the role of the HDQ as a future measure of disability in countries such as Canada and Ireland where HIV is considered a chronic illness and individuals experience similar issues related to HIV, comorbidity and aging.

Methods

We conducted two cross-sectional studies to assess the measurement properties of the HDQ in Canada and Ireland. All aspects of this research were conducted in collaboration with a Community Advisory Committee based in Canada comprised of four members including adults living with HIV, representatives from AIDS Service Organizations and a representative from the Ontario Ministry of Health and Long-Term Care. This research was approved by Research Ethics Boards at McMaster University, Hamilton, Ontario, University of Toronto, St. Michael’s Hospital, Toronto, Ontario, Canada, and St. James’s Hospital, Dublin, Ireland.

Participants and recruitment

We included adults 18 years of age or older living with HIV who self-identified as having experienced an episode of illness attributed to HIV and who were able to read and understand English. We recruited participants in southern Ontario, Canada and Dublin, Ireland from hospital clinics, HIV community-based service organizations and a specialty hospital (Casey House).

Data collection

We administered the HDQ with seven additional instruments: the World Health Organization Disability Assessment Schedule 2.0 (WHODAS-2.0), SF-36 Health Questionnaire (SF-36), Centers for Epidemiologic Studies Depression Scale (CES-D), Medical Outcomes Study Social Support Survey (MOS-SSS), HIV Symptom Index, Brief COPE, and HIV Stigma Scale [19–25]. We also administered a demographic questionnaire. We administered the HIV Disability Questionnaire (HDQ) a second time one week later to the participants in Ontario to assess test-retest reliability.

HIV disability questionnaire

The HDQ consists of 69 items that describe the presence, severity and episodic nature of disability experienced by adults living with HIV. The HDQ is comprised of six domains: physical, cognitive and, mental and emotional health symptoms and impairments, uncertainty, difficulty with day-to-day activities and challenges to social inclusion [26]. An additional item asks participants to classify living with HIV as a ‘good day’ or ‘bad day’. Each item consists of a statement about a health-related challenge and has both a five point ordinal response scale asking the respondent to rate the challenge on the day of administration (from 0 to 4) and a nominal response scale asking whether the challenge fluctuated (or changed) over the past week (‘Yes’ or ‘No’). Disability presence scores are calculated by summing the number of health challenges experienced for each HDQ domain and transforming them to a score out of 100. Disability severity scores are calculated by summing individual item scores from each domain and then linearly transforming them into HDQ domain severity scores out of 100. Episodic scores are calculated by summing the number of challenges identified as fluctuating in each domain and then transforming them to a score out of 100. All HDQ scores range from 0–100. Higher scores indicate a greater presence, severity, and episodic nature of disability.

World Health Organization Disability Assessment Schedule 2.0 (WHODAS-2.0)

The WHODAS-2.0 contains 36 items that measures disability across six domains (cognition, mobility, self-care, getting along, life activities, and participation). Scores ranges from 0 to 100, where 0 means no disability and 100 means full disability [19]. The WHO-DAS 2.0 has been used with people living with HIV [27] and possesses reliability, validity and responsiveness among people with chronic disease [28].

Brief COPE

The Brief COPE contains 28 items about coping responses to stressful events living with a health condition [21]. Brief COPE scoring is divided into an Adaptive and Maladaptive Coping Score. The Adaptive Coping Score is the sum of 16 positive coping items related to active coping, planning, emotional support, instrumental support, positive reframing, acceptance, religion and humor (ranging from 16–64) with higher scores indicating better coping skills, and the Maladaptive Coping Score is the sum of 12 negative coping items related to venting, denial, substance use, behavioral disengagement, self-distraction and self-blame (ranging from 12–48) with higher scores indicating worse coping skills [21].

HIV stigma scale

The HIV Stigma Scale contains 40 items which ask about social and emotional aspects of having HIV [20]. The HIV Stigma Scale includes four subscales: personal stigma, disclosure, negative self-image and public attitudes (scores range from 40–160). Higher scores indicate higher levels of stigma [20]. The HIV Stigma Scale possesses construct validity and reliability with people living with HIV [20].

Medical Outcomes Study (MOS) social support scale

The MOS-Social Support Survey contains 19 items that ask how often companionship, assistance and other types of support are available if needed [24]. The MOS-Social Support Score ranges from 0 to 100, where 0 means no support available when needed, and 100 means support is always available when needed [24]. The MOS-Social Support Scale possesses construct validity and reliability when used with people living with HIV [24].

HIV symptom index

The HIV Symptom Index is a 20-item questionnaire that describes the presence and burdensome nature of symptoms experienced by adults with HIV. Each item is scored using a five point ordinal scale ranging from ‘I do not have this symptom’ to ‘I have this symptom and it bothers me a lot’. HIV Symptom Index scoring involves summing the number of symptoms that are present and the number that are considered bothersome. The HIV Symptom Index demonstrates good construct validity with people living with HIV [22].

CES-D

The Centers for Epidemiologic Studies for Depression Scale (CES-D) is a 20-item instrument that measures depressive symptomatology [23]. The Scale possesses high internal consistency, adequate test-retest reliability and concurrent and construct validity in the general population [23] and has been used extensively with people living with HIV [29–33]. Item scores range from 0–60 and higher scores indicate more depressive symptoms [23].

SF-36 questionnaire

The SF-36 questionnaire is a generic health-related quality of life instrument [25, 34]. The measurement properties of the SF-36 are well established, demonstrating high internal consistency reliability, content, construct, and predictive validity [35, 36]. Domain scores for the SF-36 include physical function, role limitation physical, bodily pain, vitality, role limitation emotional, mental health, social function, and general health perception. Physical Health Component Summary (PCS) and Mental Health Component Summary (PCS) scores are calculated from the domain scores. All domain and summary scores range on a scale from 0–100 with higher scores indicating better quality of life.

Analysis

All data were entered into a database and 20 % of cases were independently manually verified for accuracy. We computed missing response rates for the disability and episodic sections of the HDQ and the health status measures. We removed cases with more than 10 % missing item responses in the HDQ. For all other cases with missing responses, we performed mean imputation.

We assessed the distribution of HDQ item responses and domain scores, and health status summary scores. We classified floor and ceiling effects as >15 % of responses being at the bottom or top of the scales, respectively [37]. We assessed differences between Canadian and Irish participants for HDQ and health status measures using the Mann-Whitney test for continuous variables and chi-square test for categorical variables (statistical significance p < 0.05). Statistical analyses were conducted using SAS and SPSS statistical software [38, 39].

Internal consistency reliability

We calculated Cronbach’s alpha for the HDQ severity and episodic domain and total scores to assess internal consistency reliability, the degree to which the items within the instrument are correlated with each other. We considered Cronbach’s alpha >0.80 as acceptable [40].

Construct validity

We tested 40 a priori construct validity hypotheses about predicted relationships using correlations between HDQ severity scores and three health status measures (WHO-DAS 2.0; SF36 questionnaire; MOS-Social Support Scale) and two hypotheses comparing HDQ presence and severity scores based on age and comorbidity. Specifically, we assessed 15 convergent hypotheses with the WHO-DAS 2.0 questionnaire, 18 convergent hypotheses with the SF-36 questionnaire, and seven divergent hypotheses with the MOS-Social Support Scale. We also tested two known group hypotheses (greater presence and severity of disability with age and number of comorbidities) (Additional file 1). Correlation coefficients of | ≥ 0.30|, | ≥ 0.50| and | ≥ 0.70|, were defined as ‘weak’ , ‘moderate, ’ and ‘strong, ’ respectively [41]. Each construct validity hypothesis and its corresponding hypothesized level of correlation are provided in Additional file 1. Fisher transformation was used to normalize the distribution and stabilize the variance, and to compute the confidence intervals [42]. We considered acceptance of at least 75 % of hypotheses determined by the correlation coefficient as demonstrating support for construct validity (Additional file 1) [43].

Test-retest reliability

We assessed test-retest reliability by calculating an intra-class coefficient (ICC) with Canadian participants who completed the HDQ at time 1 and time 2, one week later. We included participants who did not report a major change in their health between time points and who reported no change in the ‘good day/bad day’ classification item. We considered an ICC > 0.70 as acceptable for test-retest reliability [40, 44].

Sample size estimation

Sample size estimation was based on the validity analysis and our desire to detect a weak correlation between the scores of the criterion reference measures and the HDQ scores. To detect a Pearson’s correlation coefficient of r = 0.30 with a power of 0.90, and an alpha of 0.05, we required at least 113 participants [45] for each study. To adjust for an estimated 15 % of questionnaires with missing responses and loss to follow-up at time 2 for the test-retest reliability analyses, we required at least 130 participants. Using a low correlation coefficient was a conservative approach to sample size estimation.

Results

We recruited 142 participants in Ontario between April-July 2012 and 101 participants in Dublin between June-July 2012, of whom 139 and 100 participants were eligible and agreed to participate, respectively. Four people from Dublin completed less than half of the questionnaires and were removed from the dataset, resulting in 139 Canadian, and 96 Irish participants in the study (Table 1). The majority of Irish participants (89; 93 %) were recruited from St. James’s Hospital GUIDE Clinic in Dublin, whereas the majority of Canadian participants (129; 91 %) were recruited from HIV community based service organizations in southern Ontario.

Table 1 Characteristics of participants (n = 235)
Full size table

Characteristics of participants

Of the 235 participants (139 Canada; 96 Ireland), the majority were men (74 % Ireland; 82 % Canada) taking antiretroviral therapy (88 % Ireland; 91 % Canada) with an undetectable viral load (85 % Ireland; 89 % Canada) (Table 1). Compared with Irish participants, Canadian participants were older (median age: 48 versus 41 years), living longer with their HIV diagnosis (median year of diagnosis 1999 in Canada; 2003 in Ireland), reported living with a higher number of comorbidities (4 in Canada versus 1 in Ireland), tended to live alone (66 % in Canada versus 29 % in Ireland), and were less frequently working for pay (21 % in Canada versus 54 % in Ireland). Canadian participants tended to rate their health status as either fair or good whereas Irish participants tended to rate their health status as very good or excellent (Table 1).

Health status questionnaires

Compared to Irish participants, Canadians had higher WHO-DAS 2.0 disability scores, lower mental and physical health summary scores on the SF-36 questionnaire (indicating lower quality of life), higher CES-D scores (indicating higher levels of anxiety and depression), higher HIV Symptom Index Scores (indicating more symptoms), higher maladaptive and adaptive coping scores on the Brief COPE, and lower scores on the MOS-Social Support Scale (indicating less social support) (all p < 0.05). Scores were similar between groups on the HIV Stigma Scale (Table 2).

Table 2 Characteristics of participants: health status measures
Full size table

HIV disability questionnaire

Canadian and Irish participants took a median of 10 min (1st-3rd quartile: 8–12 min) and 13 min (1st–3rd quartile: 10–15 min) to complete the HDQ, respectively. Missing item responses were <3 % across all HDQ disability and episodic items. We observed a ceiling effect (>15 % of responses indicating the highest severity of disability) for eight items with Canadian participants and four items with Irish participants. Ceiling effects were most common for items pertaining to uncertainty or worrying about the future. All 69 HDQ items demonstrated a floor effect (>15 % of responses indicating no disability) for Canadian and Irish participants. Of these, 28 and 54 HDQ items had >40 % of responses indicating no disability for the Canadian and Irish participants, respectively. Floor effects were most common for items that referred to symptoms and impairments or difficulties with day-to-day activities.

Median HDQ presence, severity and episodic scores are displayed in Table 3. Highest median disability presence scores were in the cognitive (Canada: 100) and uncertainty domains (Ireland: 71). Highest median disability severity scores were in the uncertainty domain for both Canada (score: 39) and Ireland (score: 30). The median episodic score, representing the number of challenges that fluctuated in the past week, was highest in the physical symptoms and impairments domain and similar between samples (20 Ireland and Canada). Median HDQ severity and presence scores were significantly higher among Canadian participants across all domain and total scores, except for uncertainty for which they were similar (Table 3).

Table 3 Median HIV Disability Questionnaire (HDQ)scores (n = 235)
Full size table

Episodic HDQ scores were similar between groups. The most common episodic health challenges included fatigue (52 % Canada; 38 % Ireland), followed by feeling sad, down or depressed (44 % Canada; 35 % Ireland), aches and pains (37 % Canada and Ireland), nausea (39 % Canada; 20 % Ireland), shortness of breath (36 % Canada; 28 % Ireland), and feeling anxious (35 % Canada; 32 % Ireland). Eighty-one percent (time 1) and 82 % (time 2) of Canadian participants and 88 % of Irish participants completed the HDQ on what they considered a ‘good day’ living with HIV.

Validity and reliability

Internal consistency reliability

All HDQ severity and episodic domain and total scores met pre-specified criteria for internal consistency reliability. Cronbach’s alpha for the HDQ severity domain scores ranged from 0.84 (95 % confidence interval (CI): 0.76, 0.92) in the cognitive domain to 0.93 (95 % CI: 0.91, 0.95) in the mental-emotional domains for the Irish and Canadian participants respectively (Table 4). Cronbach’s alpha for the HDQ episodic scores ranged from 0.81 in the cognitive domain to 0.95 in the uncertainty domain (Table 4).

Table 4 Internal consistency reliability of the HIV Disability Questionnaire (HDQ)
Full size table

Construct validity

Of the 40 construct validity correlation hypotheses, 80 and 55 % were supported by Canadian and Irish participants respectively and both known groups hypotheses (100 %) were confirmed (Table 5). Of the correlation HDQ hypotheses, 87 and 67 % that referred to the WHO-DAS scores, 78 and 67 % that referred to the SF36 scores, and 71 and 0 % that referred to the MOS-Social Support Scale scores were supported by Canadian and Irish participants respectively. For details on each of the 42 construct validity hypotheses and resulting correlation coefficients see Additional file 1. Bolded correlation coefficients and 95 % confidence intervals indicate a given hypothesis was confirmed (Additional file 1).

Table 5 Summary of construct validity analysis results
Full size table

Test-retest reliability

Ninety-nine Canadian participants reported having no major change in their health and reported no change in the good day/bad day item on the HDQ and were included in the test-retest reliability analysis. Results demonstrated that the HDQ domain and total scores were stable over time with ICC values ranging from 0.80 (95 % CI: 0.71, 0.86) in the cognitive domain to 0.89 (95 % CI: 0.83, 0.92) in the challenges to social inclusion domain (Table 6).

Table 6 Internal consistency reliability of the HIV Disability Questionnaire (HDQ) (n = 99)
Full size table

Discussion

The HDQ demonstrated internal consistency reliability and construct validity when administered to adults living with HIV in Canada and Ireland. This is the first known HIV-specific measure developed to assess the presence, severity and episodic nature of disability among adults with HIV developed from a conceptual model of disability [9]. This study builds on previous evidence that established the domain and scoring structure of the HDQ and confirmed the validity of the six domain structure with adults living with HIV in Ontario [26, 46].

Internal consistency reliability assessment resulted in Cronbach’s alpha >0.80 for all severity and episodic domain and total scores, suggesting the items in the HDQ are homogeneous within the six HDQ domains to collectively measure the broader construct of disability at one point in time. Test-retest reliability results among the Canadian participants indicated ICC ≥0.80 for all HDQ domain and total scores suggesting the HDQ is consistent at measuring disability over time.

Pre-specified construct validity criteria were confirmed for 80 % (Canada) and 55 % (Ireland) of hypothesized relationships between the HDQ and health reference measures and both known group hypotheses. Thus, our results indicate that the HDQ satisfied construct validity (>75 % of hypotheses) for Canadian, but not for Irish, participants. Differences in construct validity between Canada and Ireland may be attributed to the mechanism in which the HDQ and reference measures were administered. For example, Canadian participants completed the measures consecutively in one single sitting in a quiet location at an HIV service organization, whereas Irish participants completed the measures intermittently while seeing various health providers in a busy HIV clinic. This may have introduced inconsistencies in the way participants responded to items across the questionnaires resulting in lower correlations between measures. For instance, if a participant attended clinic with a level of disability, the HDQ scores (administered prior to the health provider encounter) may be higher or lower compared with SF36 questionnaire scores (administered post encounter) depending on whether the disability was influenced by the event of seeing the health provider. Differences in construct validity also may be attributed to differences in culture, and the willingness of participants to disclose their vulnerability to chronic illness and the health-related challenges of HIV [47]. Cross-cultural adaptation of the HDQ may be considered when administering the instrument in other countries with varying culture or perceptions related to chronic illness [48, 49]. Finally, lower levels of confirmed hypotheses were found among the seven a priori hypotheses for divergent construct validity with the MOS-Social Support Scale in both samples (71 % Canada; 0 % Ireland) suggesting we may have overestimated the extent to which we hypothesized items in the HDQ would correlate with items of social support.

Notably Canadian participants were older, living longer with their HIV diagnosis, reported a greater number of comorbidities, tended to live alone, and fewer were working for pay in some capacity (full time, part time, or under the table). These differences were reflected in the HDQ and health status measures with Canadian participants reporting higher disability, lower quality of life, higher depression, more symptoms, and lower levels of social support compared with Irish participants. Differences between the samples were likely attributed to the different mechanism of recruitment across countries. The majority of the Irish sample (93 %) was recruited from a hospital clinic that broadly served adults living with HIV receiving care in Dublin, whereas the majority of the Canadian sample was primarily recruited from HIV community-based organizations (91 %), or a specialty hospital (7 %) whereby participants likely were experiencing health-related challenges requiring them to seek out additional social support or health services. Nevertheless, floor effects (scores closer to the lower end of the scale) were evident across the HDQ in both samples and to a greater extent among Irish participants. This may have decreased the ability for Irish HDQ scores to correlate with the other measures that possessed more normalized distributions of scores. Floor and ceiling effects can pose challenges for responsiveness and diminish the ability for an instrument to detect a change if one occurs [50]. Removing items that demonstrate a large floor or ceiling effect, or adding items at the lower end of the scale, may be considered in future HDQ revision to increase the distribution of scores and enhance the ability to discriminate between participants.

Higher coping scores among the Canadian participants suggest that they engaged in both adaptive and maladaptive coping strategies more frequently compared with Irish participants. The higher frequency of coping may be attributed to the greater comorbidity and severity of disability experienced by Canadian participants which may result in their adopting both forms of coping (adaptive and/or maladaptive) more frequently. Alternatively, Irish participants who were living with less comorbidity and disability may not feel the need to adopt these strategies to the same extent.

Unlike all other HDQ domains whereby Canadian participants reported more disability, uncertainty was the highest HDQ domain severity score and it was similar among both Canadian and Irish participants. The concept of uncertainty is particularly relevant to aging with HIV, whereby older adults may worry about their source of health challenges; health providers’ knowledge and skills; financial uncertainty; transition to retirement; appropriate long-term housing and who will care for them as they age living with HIV [51].

Compared with other HDQ domains, physical symptoms and impairments tended to fluctuate more on a daily basis with median HDQ episodic scores greatest in the physical symptoms domain (20 challenges fluctuated within the week among Irish and Canadian samples) followed by mental emotional symptoms and impairments (nine challenges), demonstrating the potential episodic nature of disability. Items related to symptoms and impairments such as fatigue, weakness and trouble concentrating may fluctuate more readily than those associated with social inclusion such as the ability to engage or re-engage in the workforce [52]. Specific symptoms and impairments that fluctuated the most included fatigue, feeling sad, down or depressed, nausea, aches and pains, shortness of breath, and feeling anxious. The episodic scale of the HDQ represents the daily episodic nature of disability rather than the larger fluctuations in health that may occur over a longer period of time. Further work is needed to assess the properties of the HDQ episodic scale.

The majority of participants (81–88 %) reported completing the HDQ on a ‘good day’ living with HIV despite the presence and severity of disability reported in the HDQ. This may be a reflection of resiliency, adaptation and hardiness among adults aging with HIV as a chronic disease [53, 54]. Nevertheless, it is unclear how participants defined a ‘good day’ versus a ‘bad day’ living with HIV. Further work exploring the interpretation of this item as it relates to the HDQ disability scores is needed.

Implications for practice, research and policy

The HDQ may be used by clinicians and HIV community organizations to assess disability experienced by their clients as a result of HIV and other concurrent health conditions. This may help to identify areas of need where programs, services and interventions that reduce disability experienced by clients with HIV can be effective. Further, given the findings suggest insufficient validity in Ireland, further refinement and validity assessment of the HDQ beyond Canada is needed. Future revision of the HDQ also may include a short-form version to enhance the feasibility of the HDQ for use in the clinical setting. Future assessment of interpretability will be critical to understand the meaning of HDQ scores for adults living with HIV and clinicians. Additional psychometric assessment of responsiveness will enable researchers to use the HDQ to document changes in disability over time and determine the effectiveness of interventions.

Our study has limitations. First, the HDQ was developed with men living with HIV living in a large metropolitan city who were taking antiretroviral therapy, living with concurrent health conditions, and not currently working. Accordingly, the HDQ demonstrated higher construct validity among Canadian participants who resembled the sample from which the HDQ was originally derived, validated and refined. While this study was the first to assess the validity of the HDQ internationally, the properties of the HDQ among people living with HIV in the developing country context are unknown. Second, because our goal was to assess the measurement properties of the HDQ rather than to measure disability per se, the HDQ scores should be considered cautiously since the interpretability of the HDQ scores are unknown. Lastly, our analysis focused on assessing the construct validity, internal consistency reliability and test-retest reliability of the domains of the disability severity scale of the HDQ. Test–retest reliability assessment was limited to Canadian participants; hence further investigation of this property outside of Canada is needed.

Conclusions

The HDQ demonstrates internal consistency reliability and a variable degree of construct validity when administered to adults living with HIV in Canada and Ireland. The HDQ demonstrates test-retest reliability when administered to adults with HIV in Canada. Differences in construct validity between countries may be due to lower HDQ scores among Irish participants who were younger and reported less comorbidity, and differences in the mechanism of questionnaire administration. Further work to explore HDQ applications outside of Canada is needed.

Abbreviations

CES-D:

Centers for Epidemiologic Studies Depression Scale (CES-D)

CI:

Confidence Interval

HIV:

Human Immunodeficiency Virus

HDQ:

HIV Disability Questionnaire

HRQL:

Health-related quality of life

MOS-SS:

Medical Outcomes Study Social Support Survey

WHODAS-2.0:

World Health Organization Disability Assessment Schedule 2.0

References

  1. Public Health Agency of Canada: Summary: Estimates of HIV Prevalence and Incidence in Canada. 2011. Available at: http://webqa.phac-aspc.gc.ca/aids-sida/publication/survreport/estimat2011-eng.php

  2. AVERT. AVERTing HIV and AIDS: European HIV and AIDS Statistics. 2011. Available at: http://www.avert.org/european-hiv-aids-statistics.htm.

  3. Health Protection Surveillance Centre (HPSC). HIV in Ireland, 2013. Dublin: Health Protection Surveillance Centre; 2014.

    Google Scholar 

  4. Goodroad BK. HIV and AIDS in people older than 50. A continuing concern. J Gerontol Nurs. 2003;29:18–24.

    Article  PubMed  Google Scholar 

  5. Justice AC. HIV and aging: time for a new paradigm. Curr HIV/AIDS Rep. 2010;7:69–76.

    Article  PubMed  Google Scholar 

  6. Guaraldi G, Orlando G, Zona S, Menozzi M, Carli F, Garlassi E, et al. Premature age-related comorbidities among HIV-infected persons compared with the general population. Clin Infect Dis. 2011;53:1120–6.

  7. Mallon PW. Aging with HIV: osteoporosis and fractures. Curr Opin HIV AIDS. 2014;9:428–35.

    Article  PubMed  Google Scholar 

  8. Kendall CE, Wong J, Taljaard M, Glazier RH, Hogg W, Younger J, et al. A cross-sectional, population-based study measuring comorbidity among people living with HIV in Ontario. BMC Public Health. 2014;14:161.

    Article  PubMed Central  PubMed  Google Scholar 

  9. O’Brien KK, Bayoumi AM, Strike C, Young NL, Davis AM. Exploring disability from the perspective of adults living with HIV/AIDS: development of a conceptual framework. Health Qual Life Outcomes. 2008;6:76.

    Article  PubMed Central  PubMed  Google Scholar 

  10. O’Brien KK, Bayoumi AM, Strike C, Young NL, King K, Davis AM. How do existing HIV-specific instruments measure up? Evaluating the ability of instruments to describe disability experienced by adults living with HIV. Health Qual Life Outcomes. 2010;8:88.

    Article  PubMed Central  PubMed  Google Scholar 

  11. Palella Jr FJ, Baker RK, Moorman AC, Chmiel JS, Wood KC, Brooks JT, et al. Mortality in the highly active antiretroviral therapy era: changing causes of death and disease in the HIV outpatient study. J Acquir Immune Defic Syndr. 2006;43:27–34.

    Article  CAS  PubMed  Google Scholar 

  12. Duracinsky M, Lalanne C, Le Coeur S, Herrmann S, Berzins B, Armstrong AR, et al. Psychometric validation of the PROQOL-HIV questionnaire, a new health-related quality of life instrument-specific to HIV disease. J Acquir Immune Defic Syndr. 2012;59:506–15.

    Article  PubMed  Google Scholar 

  13. Duracinsky M, Herrmann S, Berzins B, Armstrong AR, Kohli R, Le Coeur S, et al. The development of PROQOL-HIV: an international instrument to assess the health-related quality of life of persons living with HIV/AIDS. J Acquir Immune Defic Syndr. 2012;59:498–505.

    Article  PubMed  Google Scholar 

  14. Mayo NE, Wood-Dauphinee S, Ahmed S, Gordon C, Higgins J, McEwen S, et al. Disablement following stroke. Disabil Rehabil. 1999;21:258–68.

    Article  CAS  PubMed  Google Scholar 

  15. Wilson IB, Cleary PD. Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA. 1995;273:59–65.

    Article  CAS  PubMed  Google Scholar 

  16. O’Brien KK, Bayoumi AM, King K, Alexander R, Solomon P. Community engagement in health status instrument development: experience with the HIV disability questionnaire. Prog Community Health Partnersh. 2014;8:549–59.

    Article  PubMed  Google Scholar 

  17. O’Brien KK, Davis AM, Strike C, Young NL, Bayoumi AM. Putting episodic disability into context: a qualitative study exploring factors that influence disability experienced by adults living with HIV/AIDS. J Int AIDS Soc. 2009;12:5.

    Article  PubMed  Google Scholar 

  18. O’Brien KK, Bayoumi AM, Bereket T, Swinton M, Alexander R, King K, et al. Sensibility assessment of the HIV disability questionnaire. Disabil Rehabil. 2013;35(7):566–77.

    Article  PubMed  Google Scholar 

  19. World Health Organization. World Health Organizations Disability Assessment Schedule II (WHO-DAS-II). Geneva: World Health Organization; 2001.

    Google Scholar 

  20. Berger BE, Ferrans CE, Lashley FR. Measuring stigma in people with HIV: Psychometric assessment of the HIV stigma scale. Res Nurs Health. 2001;24:518–29.

    Article  CAS  PubMed  Google Scholar 

  21. Carver CS. You want to measure coping but your protocol’s too long: consider the brief COPE. Int J Behav Med. 1997;4:92–100.

    Article  CAS  PubMed  Google Scholar 

  22. Justice AC, Holmes W, Gifford AL, Rabeneck L, Zackin R, Sinclair G, et al. Development and validation of a self-completed HIV symptom Index. J Clin Epidemiol. 2001;54:S77–90.

  23. Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 1977;1:385–401.

    Article  Google Scholar 

  24. Sherbourne CD, Stewart AL. The MOS social support survey. Soc Sci Med. 1991;32:705–14.

    Article  CAS  PubMed  Google Scholar 

  25. Ware Jr JE, Gandek B. Overview of the SF-36 health survey and the International Quality of Life Assessment (IQOLA) project. J Clin Epidemiol. 1998;51:903–12.

    Article  PubMed  Google Scholar 

  26. O’Brien KK, Bayoumi AM, Stratford P, Solomon P. Which dimensions of disability does the HIV Disability Questionnaire (HDQ) measure? A factor analysis. Disabil Rehabil. 2014;37:1–9.

    Google Scholar 

  27. Kemmler G, Schmied B, Shetty-Lee A, Zangerle R, Hinterhuber H, Schussler G, et al. Quality of life of HIV-infected patients: psychometric properties and validation of the German version of the MQOL-HIV. Qual Life Res. 2003;12:1037–50.

    Article  CAS  PubMed  Google Scholar 

  28. Meesters JJ, Verhoef J, Liem IS, Putter H, Vliet Vlieland TP. Validity and responsiveness of the World Health Organization Disability Assessment Schedule II to assess disability in rheumatoid arthritis patients. Rheumatology (Oxford). 2010;49:326–33.

    Article  Google Scholar 

  29. Lyon DE, Younger JB. Purpose in life and depressive symptoms in persons living with HIV disease. J Nurs Scholarsh. 2001;33:129–33.

    Article  CAS  PubMed  Google Scholar 

  30. Burack JH, Barrett DC, Stall RD, Chesney MA, Ekstrand ML, Coates TJ. Depressive symptoms and CD4 lymphocyte decline among HIV-infected men. JAMA. 1993;270:2568–73.

    Article  CAS  PubMed  Google Scholar 

  31. Chishinga N, Kinyanda E, Weiss HA, Patel V, Ayles H, Seedat S. Validation of brief screening tools for depressive and alcohol use disorders among TB and HIV patients in primary care in Zambia. BMC Psychiatry. 2011;11:75.

    Article  PubMed Central  PubMed  Google Scholar 

  32. Lyketsos CG, Hoover DR, Guccione M, Senterfitt W, Dew MA, Wesch J, et al. Depressive symptoms as predictors of medical outcomes in HIV infection. Multicenter AIDS Cohort Study. JAMA. 1993;270:2563–7.

    Article  CAS  PubMed  Google Scholar 

  33. Cook JA, Grey D, Burke J, Cohen MH, Gurtman AC, Richardson JL, et al. Depressive symptoms and AIDS-related mortality among a multisite cohort of HIV-positive women. Am J Public Health. 2004;94:1133–40.

    Article  PubMed Central  PubMed  Google Scholar 

  34. Ware Jr JE. SF-36 health survey update. Spine (Phila Pa 1976). 2000;25:3130–9.

    Article  Google Scholar 

  35. McHorney CA, Ware Jr JE, Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care. 1993;31:247–63.

    Article  CAS  PubMed  Google Scholar 

  36. McHorney CA, Ware Jr JE, Lu JF, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 1994;32:40–66.

    Article  CAS  PubMed  Google Scholar 

  37. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4:293–307.

    Article  CAS  PubMed  Google Scholar 

  38. Corp IBM. SPSS Statistics for Windows, Version 19.0. Armonk, NY: IBM Corp; 2010.

    Google Scholar 

  39. SAS Institute Inc.: SAS Computer Software 9.2. 2008.

  40. Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. New York: McGraw-Hill; 1978.

    Google Scholar 

  41. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

    Article  CAS  PubMed  Google Scholar 

  42. Shen S, Lu Z: Computation of Correlation Coefficient and Its Confidence Interval in SAS®. AstraZeneca Pharmaceuticals. Wilmington, Delaware. Avaiable at: http://www2.sas.com/proceedings/sugi31/170-31.pdf

  43. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42.

    Article  PubMed  Google Scholar 

  44. Aaronson N, Alonso J, Burnam A, Lohr KN, Patrick DL, Perrin E, et al. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11:193–205.

    Article  PubMed  Google Scholar 

  45. Browner WS, Newman TB, Cummings SR, Hulley SB.  Chapter 6: Estimating sample size and power: The nitty-gritty. In: Designing Clinical Research. 2nd ed. Editors: Hulley SB, Cummings SR, Browner WS, Grady D, Hearst N, Newman TB. Philadelphia, PA: Lippincott Williams & Wilkins; 2001.

  46. O’Brien KK, Solomon P, Bayoumi AM. Measuring disability experienced by adults living with HIV: assessing construct validity of the HIV Disability Questionnaire using confirmatory factor analysis. BMJ Open. 2014;4, e005456.

    Article  PubMed Central  PubMed  Google Scholar 

  47. Shaw SJ, Huebner C, Armin J, Orzech K, Vivian J. The role of culture in health literacy and chronic disease screening and management. J Immigr Minor Health. 2009;11:460–7.

    Article  PubMed  Google Scholar 

  48. Epstein J, Santo RM, Guillemin F. A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. J Clin Epidemiol. 2015;68:435–41.

    Article  PubMed  Google Scholar 

  49. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25:3186–91.

    Article  CAS  Google Scholar 

  50. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19:539–49.

    Article  PubMed Central  PubMed  Google Scholar 

  51. Solomon P, O’Brien K, Wilkins S, Gervais N. Aging with HIV and disability: the role of uncertainty. AIDS Care. 2014;26:240–5.

    Article  PubMed  Google Scholar 

  52. National AIDS Trust. Fluctuating symptoms of HIV. United Kingdom: National AIDS Trust; 2011.

    Google Scholar 

  53. Farber EW, Schwartz JA, Schaper PE, Moonen DJ, McDaniel JS. Resilience factors associated with adaptation to HIV disease. Psychosomatics. 2000;41:140–6.

    Article  CAS  PubMed  Google Scholar 

  54. Emlet CA, Tozay S, Raveis VH. “I’m not going to die from the AIDS”: resilience in aging with HIV disease. Gerontologist. 2011;51:101–11.

    Article  PubMed Central  PubMed  Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge the members of the Community Advisory Committee including Ken King (Canadian Working Group on HIV and Rehabilitation), James Murray (Ontario Ministry of Health and Long-Term Care) and Shane Patey (Toronto People With AIDS Foundation) for their contributions to this work.

We gratefully acknowledge Community Collaborators in this work including Casey House, Toronto People With AIDS Foundation, Canadian Working Group on HIV and Rehabilitation, McMaster Special Immunology Services (SIS) Clinic, the AIDS Network (Hamilton), AIDS Niagara, AIDS Committee of Durham Region, AIDS Committee of Toronto, GUIDE Clinic at St. James’s Hospital, and Open Heart House (Dublin, Ireland).

This research was supported by an operating grant from the Canadian Institutes of Health Research (CIHR) HIV/AIDS Community Based Research Program (CBR#104072) and a Dobbin Scholarship from the Ireland Canada University Foundation (ICUF). Dr. Kelly O’Brien is supported by a CIHR New Investigator Award. Dr. Ahmed Bayoumi was supported by a Canadian Institutes of Health Research / Ontario Ministry of Health and Long-Term Care Applied Chair in Health Services and Policy Research. The views expressed in this article are those of the authors, and no official endorsement by supporting agencies is intended or should be inferred.

Author information

Author notes

    Authors and Affiliations

    1. Department of Physical Therapy, University of Toronto, 500 University Avenue, Room 160, Toronto, ON, M5G 1V7, Canada

      Kelly K. O’Brien & Nkem Iku

    2. Institute of Health Policy, Management and Evaluation (IHPME), University of Toronto, 4th Floor, 155 College Street, Toronto, ON, M5T 3M6, Canada

      Kelly K. O’Brien & Ahmed M. Bayoumi

    3. School of Rehabilitation Science, McMaster University, 1400 Main Street West, Room 408, Hamilton, ON, L8S 1C7, Canada

      Kelly K. O’Brien, Patricia Solomon & Paul Stratford

    4. Department of Genito Urinary Medicine and Infectious Diseases, Hospital 5, St. James’s Hospital, James’s Street, Dublin 8, Ireland

      Colm Bergin & Siobhán O’Dea

    5. Department of Clinical Medicine, Trinity College Dublin, Dublin, Ireland

      Colm Bergin

    6. Centre for Research on Inner City Health, Keenan Research Centre of the Li Ka Shing Knowledge Institute, St. Michael’s Hospital, 30 Bond Street, Toronto, ON, M5B 1W8, Canada

      Ahmed M. Bayoumi

    7. Department of Medicine, University of Toronto, Toronto, ON, Canada

      Ahmed M. Bayoumi

    Authors
    1. Kelly K. O’Brien
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Patricia Solomon
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Colm Bergin
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Siobhán O’Dea
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Paul Stratford
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Nkem Iku
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. Ahmed M. Bayoumi
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Kelly K. O’Brien.

    Additional information

    Competing interests

    The authors declare that they have no competing interests.

    Authors’ contributions

    KKO led the conceptual design of the study, acquisition of funding, recruitment and data collection, conducted the analysis, and drafted the manuscript. AMB and PS participated in the conceptual design of the study, acquisition of funding, analytical interpretations and drafting the manuscript. CB and SOD participated in recruitment and data collection, analytical interpretations and drafting the manuscript. NI participated in the data collection, analytical interpretations and drafting the manuscript. PSt participated in the analytical interpretations, and drafting the manuscript. All authors read and approved the final manuscript.

    Patricia Solomon, Colm Bergin, Siobhán O’Dea, Paul Stratford, Nkem Iku and Ahmed M. Bayoumi contributed equally to this work.

    Additional file

    Additional file 1:

    Details of construct validity analysis. (DOCX 229 kb)

    Rights and permissions

    Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    O’Brien, K.K., Solomon, P., Bergin, C. et al. Reliability and validity of a new HIV-specific questionnaire with adults living with HIV in Canada and Ireland: the HIV Disability Questionnaire (HDQ). Health Qual Life Outcomes 13, 124 (2015). https://doi.org/10.1186/s12955-015-0310-9

    Download citation

    • Received:

    • Accepted:

    • Published:

    • DOI: https://doi.org/10.1186/s12955-015-0310-9

    Keywords

    Health and Quality of Life Outcomes

    ISSN: 1477-7525

    Contact us