Injection Drug User Quality of Life Scale (IDUQOL): Findings from a content validation study

Background Quality of life studies among injection drug users have primarily focused on health-related measures. The chaotic life-style of many injection drug users (IDUs), however, extends far beyond their health, and impacts upon social relationships, employment opportunities, housing, and day to day survival. Most current quality of life instruments do not capture the realities of people living with addictions. The Injection Drug Users' Quality of Life Scale (IDUQOL) was developed to reflect the life areas of relevance to IDUs. The present study examined the content validity of the IDUQOL using judgmental methods based on subject matter experts' (SMEs) ratings of various elements of this measure (e.g., appropriateness of life areas or items, names and descriptions of life areas, instructions for administration and scoring). Methods Six SMEs were provided with a copy of the IDUQOL and its administration and scoring manual and a detailed content validation questionnaire. Two commonly used judgmental measures of inter-rater agreement, the Content Validity Index (CVI) and the Average Deviation Mean Index (ADM), were used to evaluate SMEs' agreement on ratings of IDUQOL elements. Results A total of 75 elements of the IDUQOL were examined. The CVI results showed that all elements were endorsed by the required number of SMEs or more. The ADM results showed that acceptable agreement (i.e., practical significance) was obtained for all elements but statistically significant agreement was missed for nine elements. For these elements, SMEs' feedback was examined for ways to improve the elements. Open-ended feedback also provided suggestions for other revisions to the IDUQOL. Conclusion The results of the study provided strong evidence in support of the content validity of the IDUQOL and direction for the revision of some IDUQOL elements.


Background
In the health and medical fields, quality of life (QoL) is widely used to evaluate social and clinical interventions, treatment side effects, and disease impact over time [1,2]. Most of these QoL instruments tend to focus on health-related quality of life (HRQOL) or the functional effects of respondents' perceived mental and physical health [3]. Gill and Feinstein [3], however, defined QoL as a reflection of respondents' perceptions and reactions to not only their mental and physical health, but also to non-health related aspects of their lives (e.g., family, friends, work). Thus, measurement of QoL needs to encompass more than just the health-related aspects of respondents' lives.
Although previous research with IDUs and related populations (e.g., illicit drug users, HIV/AIDS) has considered the effects of non-health related aspects of respondents' lives on their HRQOL [4,7,15,[24][25][26][27][28] or even on the initiation or maintenance of drug use [29][30][31][32][33][34][35], rarely has published research with IDUs used a broadly-defined QoL measure (i.e., one that captures various social, psychological, physical, geographic, and occupational domains of QoL). Two exceptions would be Wasserman and colleagues [36], who examined the psychometric properties of Lehman's [37] Quality of Life Interview -Brief Version with IDUs, and Dunaj and Kovác [38], who compared convicted drug addicts and controls on broadly-defined QoL using the WHOQOL-BREF [39] and ComQol-A5 [40]. Dunaj and Kovác reported that addicts scored significantly lower than controls in their subjective ratings of areas such as health, emotional well-being, safety, and social standing.
Although broader measures of QoL are beginning to be used with IDUs [36,38] and are certainly an improvement over the use of strictly HRQOL measures, measures developed specifically for the IDU population and using a context sensitive approach that considers the many life areas deemed by IDUs as critical to their QoL, are still needed. QoL, as defined by the World Health Organization Quality of Life (WHO-QOL) group, refers to "an individual's perceptions of their position in life in the context of the culture and value systems in which they live, and in relation to their goals, expectations, standards, and concerns" (pp. 1-2) [41]. The item content and methods of administration for most available QoL instruments do not measure the QoL of drug users in a culturally-sensitive fashion [42]. IDUs live in a distinct environment characterized by a high prevalence of infectious disease, crime, violence, and lack of stable housing. Many IDUs cannot depend on basic necessities and experience considerable instability in many aspects of their lives.
A recently developed broadly-defined QoL measure, the Injection Drug User Quality of Life (IDUQOL) scale, was designed to capture the health and non-health related aspects of IDUs' lives that would be important components of their quality of life, particularly given their individual circumstances and environment [43,44]. This measure has also been adapted for use in Spanish with injection and non-injection drug users [45]. To use an instrument with confidence, it is important that there be evidence of validity -that is, the meaningfulness, usefulness, and appropriateness of an instrument for a given population in a given context [46][47][48]. Previous research has examined the factor structure, internal consistency, and test-retest reliability of scores from the IDUQOL as well as the criterion-related, convergent, and discriminant validity of inferences made from the measure [44]. Content validity, a critical step in the test development and validation process [49][50][51], refers to the degree to which elements of an assessment tool are representative of the construct of interest and appropriate for a given population [52]. Importantly, the elements of interest in a content validation study are not just the content or items of the measure, but all elements of the instrument including the instructions, response format, and scoring procedures [53].
The purpose of the present study was to examine the content validity of the IDUQOL using judgmental methods based on subject matter experts' (SMEs) ratings of the IDUQOL title, items, instructions, response format, scoring procedures, and record form.

Participants
The sample consisted of a panel of six subject matter experts (SMEs; 50% male), all of whom were researchers working in the area of drug use in the United States or Canada with an average of 10 years experience in the field. Two SMEs were epidemiologists and four SMEs were physicians who also provided addiction and medical care to drug users in their clinical practice. As noted by others [54], there is no set number of SMEs required for content validation studies. Typically, somewhere between three to ten experts is recommended, although a minimum of five SMEs is recommended to control for chance agreement; furthermore, the larger the number of experts, the greater the confidence in the ratings and the easier it is to detect rater outliers [53,55].

Measures
The subject matter experts were provided with a copy of the IDUQOL and its administration and scoring manual and a detailed content validation questionnaire. Ethics approval for this study was obtained from the University of British Columbia and Providence Health Care Research Ethics Boards.

Injection Drug User Quality of Life Scale (IDUQOL)
The original IDUQOL, which includes both health and non-health related aspects of QoL [3] and is based on the WHO-QOL group definition of QoL [41], consisted of 20 life areas. Several of these areas (e.g., Drugs, Drug Treatment, Harm Reduction and Neighbourhood Safety) were included in the measure precisely because of their particular relevance to the social and physical reality of IDUs as confirmed by focus groups during the development phase [43]. Each IDUQOL life area is represented on a 4 by 4 inch card, with the name of the area printed on the front along with a simple picture. A description of the life area is presented on the back (see Table 1 for a list of all 20 life areas and descriptions). Graphic representation of the life areas is intended to make the instrument more accessible to individuals who have low literacy skills or do not speak English as a first language. When administering the IDU-QOL, the interviewer starts by showing the respondent each of the 20 life area cards and describes the area. The participant selects those areas that he/she deems important to his/her quality of life and any remaining cards are set aside. The cards representing important areas are laid out and the participant is given three poker chips for each card. The total number of chips can, therefore, range from 0 (no life areas are important) to 60 (all 20 life areas are important). The participant then distributes the chips across the cards to indicate the level of importance of each life area, with more chips indicating greater importance. Next, the participant provides a satisfaction rating for each area, using a 6-point Likert-type scale anchored by 1 (very dissatisfied) and 6 (very satisfied) and illustrated with six stylised frowning and smiling faces.
When scoring the IDUQOL, the importance rating (number of chips) of each area is divided by the total number of chips used by that participant and then multiplied by the satisfaction rating for that area. This produces an area score. Finally, all area scores are summed to obtain an overall quality of life score ranging from 1 (very dissatisfied) to 6 (very satisfied).

IDUQOL content validation questionnaire
The questionnaire was divided into seven sections covering the clarity of the instrument title, ease of administration procedure instructions, clarity of the names and descriptions of the 20 IDUQOL life areas, whether each of the 20 IDUQOL life areas should be included in the measure (including whether any life areas need to be added, revised, or deleted), ease of the response formats used for each of importance and satisfaction ratings, clarity of scoring procedure instructions, and the ease of use of the record form. Experts were also given the opportunity to provide open-ended commentary in each section. As recommended by Lynn [55], a four-point Likert type scale was used in most cases. For questions involving clarity, the following four response options were used: 0 = not at all clear, 1 = somewhat clear, 2 = mostly clear, 3 = very clear. For questions involving ease, the following four response options were used: 0 = not at all easy to follow/ use, 1 = somewhat easy to follow/use, 2 = mostly easy to follow/use, 3 = very easy to follow/use. For questions involving inclusion of items, the following three response options were used: 0 = no, 1 = unsure, 2 = yes. Two questions asked about how helpful the provided examples were in the manual; for these, the following four response options were used: 0 = not at all helpful, 1 = somewhat helpful, 2 = mostly helpful, 3 = very helpful.

Procedures
The six SMEs were identified through the second author's professional contacts with nationally and internationally recognized experts in the area of substance abuse epidemiology and treatment. They were sent a letter of invitation and agreed to take part in the study. None of the SMEs were associated with the development of the IDUQOL. The SMEs were mailed a copy of the IDUQOL (which included the 20 life area cards, poker chips, satisfaction rating card, and record form), the administration and scoring manual, and the IDUQOL content validation questionnaire. As suggested by Grant and Davis [56], SMEs were provided with the conceptual basis for the IDUQOL via the brief introduction in the manual in which the definition of QoL underlying this measure, the target population, and how the measure is intended to be used was provided. The SMEs completed the content validation questionnaire at their leisure and independently of one another. All SMEs returned usable questionnaires.
Two commonly used judgmental measures of inter-rater agreement, the Content Validity Index (CVI) [55,57] and the Average Deviation Mean Index (AD M ) [58][59][60], were used to evaluate SMEs' agreement on ratings of the various IDUQOL elements. The two measures provide very different types of information, however, and should be viewed as complementary. Generally, the CVI indicates the proportion of SMEs that endorse an element as content valid whereas the AD M indicates the degree of disagreement among SMEs in the response option selected regardless of whether they, as a group, endorsed an element or not. Thus, one should first examine the CVI values to determine whether the SMEs endorsed an item or not and then consider the level of agreement among the SMEs by examining the AD M .
The CVI can be computed at the individual item level (I-CVI) and at the level of the overall scale or subscale (S-CVI). I-CVI is computed as the proportion of SMEs that endorse an item. Following standard procedures for four response options [49,55], ratings of 2 or 3 were combined and treated as endorsements by SMEs whereas ratings of 0 or 1 were combined and treated as non-endorsements in the present study. When three response options were used, a rating of 2 was treated as an endorsement by SMEs whereas ratings of 0 or 1 were combined and treated as non-endorsements. A minimum of five out of the six SMEs (I-CVI ≥ .83) had to endorse an item to achieve significant evidence (α = .05) of content validity for any given item on the IDUQOL content validity questionnaire and to provide confidence that agreement was not occurring by chance alone [55]. Elements that were not endorsed by a minimum of five SMEs were examined further to determine if appropriate revisions could be made.
The S-CVI may be defined and computed a number of different ways, but Polit and Beck [49] recommend using the average proportion of items endorsed by the SMEs (what they refer to as S-CVI/Ave) and computing this as the average of the I-CVI values. This is the approach that will be used in the present study in conjunction with Lynn's [55] description of S-CVI as "the proportion of total items judged content valid" (p. 384). For S-CVI/Ave, the minimum acceptable value is recommended to be .90 [49].
The AD M Index measures dispersion of ratings about the mean rating; thus, it is actually a measure of disagreement so lower values indicate higher levels of agreement among SMEs. An advantage of the AD M Index is that it provides a measure of dispersion that is directly interpretable in terms of the original rating scale units. The general cut-off for determining acceptable AD M values is based on c/6, with c referring to the number of response options [59]. Thus, using this guideline for practical significance, acceptable AD M values are .50 or less for ratings with three response options and .69 or less for ratings with four response options. Critical values that can be used to evaluate whether an obtained AD M could have been achieved by chance can also be computed. Critical values for AD M at the 5% level of significance, taking into account the number of SMEs and the number of response options in the present study, would be .28 or less for three response options and .44 or less for four response options [58]. AD M values that are equal to or below these critical values are unlikely to have been obtained by chance. Elements  Most SME comments were focused on the visual depiction provided on the card and no suggestions for changes or additions to the description were offered. The description for Drugs (i.e., "drug use -e.g., alcohol, heroin, cocaine, crack") was rated as 'somewhat clear' by one SME, 'mostly clear' by two SMEs and 'very clear' by three SMEs. There were two concerns raised by SMEs. The first and most prominent concern was that we limited this life area to use of drugs; the second concern was that we only listed four drugs. The description for Feeling Good about Yourself (i.e., "e.g., self-esteem, selfworth") was rated as 'mostly clear' by half of the SMEs and 'very clear' by the other half of the SMEs. No suggestions were made for changes to the description.

Content validity evidence for IDUQOL life areas
SMEs were also asked if there were any life areas that they would recommend deleting or adding to the IDUQOL. No life areas were recommended for deletion from the IDUQOL. The following additional life areas were suggested: food, pets, personal safety, sense of future (e.g., hopefulness, aspirations), employment (as its own life area separate from Being Useful), and pain. Table 3 shows that the S-CVI/Ave for the element groupings of Appropriateness, Name clarity, and Description clarity of the IDUQOL life areas ranged from .97 to .99, which exceeded the minimum value of .90 and is also strong evidence of content validity. The name or title of the IDUQOL was rated as 'somewhat clear' by one SME, 'mostly clear' by one SME and 'very clear' by four SMEs, with no suggestions made for how to make the title clearer. The Response Format -Chips was rated as 'somewhat easy to use' by one SME, 'mostly easy to use' by one SME, and 'very easy to use' by four SMEs. Suggestions were made for simplifying the instructions given to IDUs about how to use the poker chips to indicate the importance of the different life areas. In addition, it was suggested that poker chips might act as a trigger for IDUs with gambling issues. The Response Format -Smiley Faces was rated as 'somewhat easy to use' by one SME and 'very easy to use' by five SMEs. It was suggested that we consider using an odd-numbered Likert-type scale (rather than our 6-point Likert-type scale) for the smiley faces that would permit a neutral response. The Scoring Procedure -Summed Score was rated as 'somewhat easy to follow' by one SME, 'mostly easy to follow' by one SME, and 'very easy to follow' by four SMEs. It was suggested that we clarify the headings on the record form so they would better match the terms used in the manual.

Suggested revisions to the IDUQOL based on SME feedback
Open-ended feedback and comments from the SMEs resulted in several other suggestions for changes to the IDUQOL measure, materials, and manual. These may be grouped into three points. First, suggestions were made to expand the descriptions for (a) Education, (b) Family, and (c) Sex. Second, it was pointed out that we needed to show greater diversity in our cards involving people -specifically Family and Friends. Third, we were advised to revise the cards depicting (a) Being Useful, and (b) Independence and Free Choice to make them clearer.

Discussion
Test development and validation are ongoing processes designed to ensure measures and the inferences made from them remain appropriate, relevant, and useful for  the target population and context of use [47]. The IDU-QOL was developed as a measure of broadly defined subjective QoL that incorporates both health and non-health related aspects of IDUs' lives. Administration of the IDU-QOL was designed to be sensitive to the diversity of literacy levels, English language skills, attention levels, and cognitive abilities of the target population. An important step in test development and validation is the evaluation of content validity. The purpose of the present study was to examine the content validity of various elements of the IDUQOL measure and manual using SMEs and two commonly used judgmental methods (i.e., CVI and AD M ). The CVI indicates the proportion of SMEs that endorse an element as content valid whereas the AD M indicates the degree of disagreement among SMEs in the response option selected. Overall, the results of this study provide strong evidence for the content validity of the elements of the IDUQOL measure and manual. Specifically, the I-CVI results supported the content validity of each of the individual elements. These elements include the appropriateness, name, and description of each of the 20 life areas, clarity of the name of the measure, clarity of the target population, ease of each step of the administration procedure, ease of each response format (i.e., chips and smiley face scale), ease of each step of the scoring procedure, helpfulness of each of the provided example boxes in the manual, and ease of use of the IDUQOL record form.
The S-CVI/Ave results also supported the content validity of all of the grouped elements of the IDUQOL measure and manual (e.g., Appropriateness of Life Areas, Ease of Administration Procedure), with the exception of Ease of Response Formats. Two points are worth noting about the Ease of Response Formats case. First, when examined individually using I-CVI, each of the two items under Ease of Response Formats was endorsed by five of the six SMEs, supporting their content validity. In fact, the one SME who supposedly did not endorse either of these items actually circled both 'somewhat easy to use' and 'mostly easy to use' (responses that fell into the 'not endorsed' and 'endorsed' categories, respectively) in each case and indicated that the ease of each response format for the target population was an empirical question that should be piloted instead. Using a conservative approach, we treated this SME's response as a non-endorsement, although it could be argued to be more ambiguous. Second, it should be noted that when there are only two or three items making up a grouping of elements (as is the case for the twoitem Ease of Response Formats grouping), the minimum acceptable level of .90 for S-CVI/Ave cannot be reached unless all but one item in the grouping achieves endorsement by all six of the SMEs. Taking each of these points into account, we would argue that the content validity of the response formats should not be discounted, but that care should be taken to ensure, through further study or pilot testing, that these response formats are appropriate to the group of IDUs with whom a researcher or practitioner wishes to use the IDUQOL. Based on interviews we conducted with the experienced staff who administered the IDUQOL to participants in another study [44,61], that sample of IDUs did not have difficulty using either response format, although some respondents expressed the desire for a neutral response option on the Likert-type smiley face satisfaction scale.  Figure 2). For the Being Useful and Free Choice life areas, SMEs recommended using different images to make these concepts clearer to respondents. The old and new cards are shown in Figure 3.
Fifth, we were particularly struck by one SME's suggestion that the poker chips used in the response format for importance ratings might act as a trigger for IDUs with gambling issues and so we changed these to unmarked chips. We also incorporated suggestions made by SMEs for simplifying the instructions given to IDUs in the manual about how to use the chips to indicate the importance of the different life areas.
Sixth, we changed the Likert-type smiley face scale used to rate satisfaction from a 6-point scale to a 7-point scale that would permit a neutral response. This was based not only on SMEs' suggestions but also on our own concurrent experiences in administering the IDUQOL to IDUs [44,61]. We found that it was particularly appropriate to have a neutral option available when the respondent was rating satisfaction with a life area that had not been rated as particularly important.
Seventh, we revised the headings used in the IDUQOL record form so they would better match the terms used in the manual and would make obtaining the summed score easier (see Figure 4.) Given the strong support provided by this study for the content validity of the IDUQOL and its manual, the revisions made to improve them further, and previous research supporting the validity of inferences made from the IDUQOL [44,61], the IDUQOL is a viable instrument for assessing broad-based QoL in IDUs and potentially in non-injection drug users [45]. The majority of published research in which QoL is examined with IDUs focuses on HRQOL. Future research is needed that examines the impact of drug use and various treatment options on QoL using broadly defined subjective QoL measures such as the IDUQOL. Future research on the IDUQOL needs to further examine its appropriateness and usefulness with non-injection drug users, its sensitivity to change, and its relationship with other broad-based QoL measures such as the Quality of Life Interview -Brief Version, which was originally developed for use with the mentally ill, or the Personal Wellbeing Index, which is the successor to the ComQol-A5 and was developed for use with the general population [62].

Conclusion
The results from the present study provided strong support for the content validity of the elements of the IDU-QOL measure and manual.

Original card descriptions
DRUGS -drug use -e.g., alcohol, heroin, cocaine, crack EDUCATION -e.g., formal schooling, literacy programs FAMILY -e.g., parents, children, siblings, foster families (not friends) SEX -e.g., sexual intimacy, quantity or quality of sex, sex in exchange for money or drugs, sexual abuse

Revised card descriptions
DRUGS & ALCOHOL -e.g., marijuana, speed, alcohol, heroin, cocaine, crack, etc. and includes selling, buying, and using EDUCATION & TRAINING -e.g., formal schooling, literacy programs, high school equivalency, life skills training, job training, certification, pre-employment programs, language courses (e.g., ESL) FAMILY -e.g., parents, children, siblings, foster families, grandparents, cousins, aunts and uncles (not friends) SEX -e.g., sexual intimacy, sex in exchange for money or drugs, being safe when having sex (use of condoms), birth control, sexual abuse Original and revised cards to improve diversity of people Figure 2 Original and revised cards to improve diversity of people.

Original cards Revised cards
Original and revised cards to improve poor graphics on cards Figure 3 Original and revised cards to improve poor graphics on cards.

Original cards Revised cards
researchers, practitioners, and program evaluators to use as a way of assessing and tracking changes in QoL over time or as a result of interventions in IDUs.

Competing interests
The author(s) declare that they have no competing interests.
Original and revised IDUQOL record form headings Figure 4 Original and revised IDUQOL record form headings.