Evaluation of the Content Validity Index of the Australian Canadian hand OA Index, the Patient-Rated Wrist/Hand Evaluation and the Thumb Disability Exam in people with hand arthritis


 BackgroundThe Australian Canadian hand OA Index (AUSCAN), the Patient-Rated Wrist/Hand Evaluation (PRWHE) and the Thumb Disability Exam (TDX) are patient-reported outcome measures (PROM) used to assess pain and hand function in patients with hand arthritis. This study evaluated the content validity of PRWHE, AUSCAN and TDX. MethodsThis study enrolled participants with hand arthritis to rate the items of all 3 PROM in terms of relevance and clarity. The Content Validity Index (CVI) was computed for each item in each scale (I-CVI) as well as for the overall scale (S-CVI). Kappa was used to determine the inter-rater agreement among the experts. ResultsOverall, 64 individuals with hand arthritis (27% with OA, 67% with rheumatoid arthritis and 6% with psoriatic arthritis) participated in the study. The I-CVI for all items and all scales was very high (I-CVI > 0.76) and the modified Kappa agreement among the raters demonstrated excellent agreement (k>0.76).The S-CVI for all PROMs was found very high for relevance (PRWHE = 0.85, 95% CI 0.82 to 0.88; TDX = 0.87, 95% CI 0.85 to 0.89 and AUSCAN = 0.92, 95% CI 0.90 to 0.94) and for clarity (PRWHE = 0.95, 95% CI 0.93 to 0.97; TDX = 0.91, 95% CI 0.89 to 0.94 and AUSCAN = 0.99, 95% CI 0.98 to 1.00) respectively. ConclusionsThis study demonstrated very high content validity indices for the PRWHE, AUSCAN, and TDX; with strong consensus across reviewers. This augments prior statistical evidence supporting statistical measurement properties, to provide support for the content validity.


Introduction
Hand osteoarthritis (OA) is one of the most common musculoskeletal diseases and a leading cause of disability with an increasing prevalence mainly attributed to increased life expectancy. [1,2] Clinical characteristics of hand OA typically involve pain, reduced hand function, decreased hand grip strength, poor quality of life [3,4] joint degeneration, bony enlargements and joint swelling. [5] Rheumatoid arthritis, although leading to bone tissue abnormalities, loss of joint function and impact on quality of life similarly to OA, is a distinct pathology that mainly targets synovial and soft tissue structures. [6] Patient-reported outcome measures (PROMs) are often administered to assess any health-related changes that may have occurred as a consequence of health-management interventions. [7,8] Many properties are important [9][10][11][12] during an instrument development but a key property is considered to be content validity. [13] Content validity can be de ned as the degree of which the instrument or the questionnaire is an adequate re ection of the construct being measured. [14] Based on the Consensusbased Standards of the selection of health Measurement Instruments (COSMIN) initiative content validity is considered as one of the most important measurement properties. [13] While reliability, responsiveness and other types of validity can be pivotal for an outcome assessment they may be insu cient to establish the validity of a PROM. [15] When PROMs include irrelevant items and lack of clarity they are ine cient, and may have weaker measurement properties. [13] Most importantly, if key aspects are missing or the questions are not relevant responses, they may not re ect patient status or concerns, and may be biased because patients may get frustrated. [16] The Australian Canadian hand OA Index (AUSCAN) [17], the Patient-Rated Wrist/Hand Evaluation (PRHWE) [17]and the Thumb Disability Exam (TDX) [18] are clinical tools designed to assess pain and hand function in hand arthritis. Their psychometric properties such as reliability, validity and responsiveness have been adequately studied however, the evaluation of relevance and clarity of all the individual items by including patient input as experts has not been quanti ed. Therefore, we aimed to investigate the quanti cation of content validity index by asking patients with hand arthritis to rate each of the instruments items in terms of relevance and clarity. information about the survey. separate Both electronic and paper versions of the survey were available for participants. An email with the link of the online survey was sent out to the participants that were interested to complete the electronic version. The electronic version was hosted on Qualtrics which is a secure data collection platform. [19] Participants were asked to provide consent to proceed into the survey questions.

Patient-reported Outcome Measures
The Australian and Canadian Osteoarthritis Index (AUSCAN) is a 15-item self-reported disease speci c questionnaire measuring pain (5-items), function (9-items) and stiffness (1-item) in the hand on a scale from 0 -none to 4 -extreme for all items. [17,20] The Patient-Rated Wrist/Hand Evaluation (PRWE)is a self-administered questionnaire which has 2 subscales of pain (5-items) and function (10-items). The PRWE was originally developed and tested for people with distal radius fracture (DRF) [21][22][23] and later validated as applicable to the wrist/hand for multiple conditions including arthritis as the PRWHE. [17,24] Each item is scored from 0 to 10 scale which 10 indicates the worst possible pain or disability. The Thumb Disability Exam (TDX) is composed of 20 questions divided into 3 sections: hand function (11items), pain (5-items) and satisfaction (4-items). Each item for hand function is scored from 1 -not di cult to 5 -unable, for level of pain 1 -never to 5 -always and for satisfaction from 1 -very satis ed to 5 -very dissatis ed. [18] Data Analysis Descriptive statistics were used to capture the demographics characteristics of the included sample. A Content Validity Index (CVI) value was computed for each item on the AUSCAN, PRWE and TDX (I-CVI) as well as for the overall scale (S-CVI). To calculate an item-level CVI (I-CVI), patients with hand arthritis were asked to rate the relevance of each item, on a 4-point scale. Four ordinal points were used foreach scale which was 1=not relevant, 2=somewhat relevant, 3=quite relevant, 4=highly relevant. Then, for each item, the I-CVI was computed as the number of patients giving a rating of either 3 or 4, divided by the number of experts-that is, the proportion in agreement about relevance and clarity which is between 0 and 1. To calculate the modi ed kappa statistic, the probability of chance agreement (Pc) was rst calculated for each item by the following formula: Pc= [N! /A! (N -A)!] *0.5 N with N being the number of experts (patients with arthritis) and A is the number of patients that agree that the item was clear or relevant. Then Kappa was calculated of entering the probability of chance agreement (Pc) and content validity index of each item (I-CVI) in the following formula: K= (I-CVI -PC) / (1-PC). Kappa values of 0.74 and above were considered as excellent, 0.60 to 0.74 as good and 0.54 to 0.59 as fair. [25] We performed a Shapiro-Wilk as the omnibus test for assessing univariate normality of each S-CVI distribution, in both relevance and clarity subscales of PROMs. Then, the S-CVI scores were compared with a paired student's t-Test if normality assumption was met or with Wilcoxon paired signed-ranks test, if assumptions of normality were violated. [26] We conducted all the analyses with STATA (StataCorp. 2019. Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC)

Results
Overall, 64 individuals with hand arthritis (27% with hand OA, 67% with rheumatoid arthritis in the hand and 6% with psoriatic arthritis) participated in the study. Four individuals were excluded from the analysis because their arthritis was not affecting their hand. The majority of the participants (66%) were taking pain medication on a daily basis (Table 1).

Content Validity Index and Modi ed Kappa agreement of the PRWHE
The I-CVI and the S-CVI of the PRWHE for pain subscale and function subscales all supported the content validity of the PRWHE (Table 2). Five items of pain subscale were rated for relevancy and clarity with I-CVI values ranging from 0.79 to 0.89 and from 0.87 to 0.94, respectively. For function subscales, 10 items were rated for relevancy and clarity with I-CVI values ranging from 0.79 to 0.95 and from 0.92 to 1.00 respectively. The S-CVI for PRWHE was 0.85, 95% con dence intervals (CI): 0.82 to 0.88 for relevance and 0.95,95%CI: 0.93 to 0.97 for clarity. The modi ed Kappa agreement for every item of PRWHE demonstrated excellent agreement (K ranging from 0.79 to 1.00).

Content Validity Index and Modi ed Kappa agreement of the TDX
The I-CVI and the S-CVI supported the content validity of the TDX for hand function, pain and satisfaction subscales (Table 3) The I-CVI and the S-CVI supported the content validity of the hand pain, stiffness and function items and subscales of the AUSCANs (Table 4). Five items of pain subscale were rated for relevancy and clarity with I-CVI scores ranging from 0.86 to 0.96 and from 0.92 to 1.00 respectively. For 1-item in stiffness subscale the I-CVI was found 0.93 for relevancy and 1.00 for clarity. For function subscale, 9-items were rated for relevancy and clarity with an I-CVI ranging from 0.88 to 0.97 and from 0.98 to 1.00 respectively. The S-CVI for AUSCAN was found 0.92,95% CI: 0.90 to 0.94 for relevance and 0.99, 95% CI: 0.98 to 1.00 for clarity. Content validity of PRWHE was established during the development of the PRWE by using semistructured interviews in patients with distal radius fracture and expert opinion. [27] However, neither were quanti ed. Thus, this study provides novel information on the content validity of the items of the PRWE/PRWHE, with speci c reference to those with hand arthritis. All items of PRWHE were found with very high content validity index in terms of relevance (I-CVI > 0.79) and clarity (I-CVI > 0.87).
For the AUSCAN the content validity was established during development using a formal clinimetric process where patients in a tertiary care centre rated items by importance and frequency to establish relevance. [20] This study provides additional support for the content validity in a community sample of people living with hand arthritis, and by adding new data on the clarity of the items.
It might have been expected that the AUSCAN would have more relevance to our sample, than the PRWHE since it a disease-speci c PROM. Both point estimate and CI comparisons indicate that AUSCAN had slightly higher overall scores in terms of relevancy (S-CVI = 0.92, 95% CI: 0.90 to 0.94) and clarity (S-CVI = 0.99, 95% CI: 0.98 to 1.00) than the PRWHE (S-CVI=0.85, 95% CI :0.82 to 0.88 for relevancy and S-CVI=0.95, 95% CI: 0.93 to 0.97 for clarity). Although the CIs of the respective S-CVIs indicate that there was a small statistically signi cant difference (Table 5) between compared S-CVI values (AUSCAN vs TDX and AUSCAN vs PRWHE), all PROMs met standards of very high content validity. Finally, since 6 to 8 additional raters assessed the PRWHE than the AUSCAN, the small differences may re ect differences in rater pools rather than an actual difference in perceptions. . Then, the development process included item reduction and pilot testing and then nal item reduction. [18] Our kappa statistics indicated excellent agreement between patient raters after correcting for chance agreement. (K> 0.77). The assessment from a large pool of experts (n> 60) generated similar scores between the I-CVI and K scores. This has been previously described in the literature when the number of experts increasing and the probability of chance (Pc) decreases the K agreement and I-CVI values tend to converge. [28] This study provided novel data on the content validity index in 3 different PROMs in patients with hand arthritis. Since few studies address content validity, this is important to support the conceptual foundations of these measures. While the computation of CVI is relatively easy, its major weakness is the failure to adjust for chance agreement. However, the authors tried to mitigate this problem by calculating a modi ed kappa agreement. A potential limitation is that the items of the PROMs were not randomized but the items were rated for relevance and clarity in an order (PRWE, AUSCAN, TDX). We deem that it is highly unlikely to have an order effect in the CVI values. First, higher scores were found in AUSCAN and not in PRWE and second, all CVI scores were very high and this indicates that the conclusion is not affected by order.

Conclusions
This study demonstrated evidence of very high content validity index for all the individual items and for the overall scale of PRWHE, AUSCAN and TDX for patients with hand arthritis, with high agreement across raters.

Disclosure of Funding
This study is supported by the Canadian Institutes of Health Research (CIHR) with funding reference number (FRN: 201710GSD-402354-282879) and by an operating grant from the Arthritis Society of Canada.

Con ict of Interest
The authors declare that they have no con ict of interest.

Data sharing it not permitted by our institutional research ethics board
Code availability Code will be made available upon request Authors' Contribution PB conceived and designed the study, collected, analyzed and interpreted data, wrote the rst draft of the article, and contributed to all revisions. JCM and ECB analyzed and interpreted data and contributed to all revisions. EL, RG and LF contributed to designing the study, collected and interpreted data, and contributed to all revisions. All authors read and approved the nal manuscript.