Assessment of psychometric properties of the Brazilian version of the oral anticoagulation knowledge test

Background The aim of this study was to evaluate the psychometric properties of the Brazilian version of the Oral Anticoagulation Knowledge (OAK) Test. Methods This study, conducted in an anticoagulation clinic, included 201 Brazilian participants aged over 18 years, who had been using warfarin for more than two months. The reliability of the instrument was evaluated by assessing internal consistency (Kuder-Richardson coefficient) and reproducibility (test-retest reliability). The validity was evaluated by hypothesizing that there would be a positive correlation of moderate to strong intensity between the correctness levels of the OAK Test and time within therapeutic range (TTR) values, which is a measure used to evaluate the quality of oral anticoagulation. Results The instrument exhibited good psychometric properties. The total a Kuder-Richardson coefficient value was 0.818 and intraclass correlation coefficient was 0.967. The validity revealed a strong positive correlation between the values of the level of knowledge, as measured by the OAK Test and the TTR values (rs = 0.780). Conclusion The instrument proved to be a reliable and valid tool for evaluating the knowledge of Brazilian patients on oral anticoagulation therapy with warfarin. This instrument may be incorporated into the practice of health care for substantiating the structuring of educational activities to ensure the improvement of knowledge about the use of warfarin, thereby increasing the effectiveness and safety of treatment.


Background
Warfarin is an oral anticoagulant, coumarinic derivative used worldwide for primary and secondary prophylaxis of thromboembolism. Its efficacy and safety have been widely studied for several indications through wellconducted clinical trials. The indication of oral anticoagulation is increasing due to population aging and the consequent increase in the incidence of thromboembolic diseases [1][2][3][4][5].
The management of warfarin is quite complex due to its narrow therapeutic range and wide dose-response variability. Among the interfering factors in the anticoagulant effect, we can cite the presence of genetic polymorphisms, drug and dietary interactions of potential clinical significance, as well as changes in the patient's condition as some of the common ones [1,6].
For proper dose management, laboratory monitoring is conducted by means of the International Normalized Ratio (INR) calculated from the prothrombin activity. The target INR is considered between 2.00 and 3.00 for most of warfarin indications. Subtherapeutic results were related to the occurrence of venous thromboembolism and higher values, to increased risk of bleeding [1]. To evaluate the quality of anticoagulant therapy, the percentage of time of INR in the therapeutic range is calculated (time within therapeutic range; TTR), the bleeding and mortality rates being significantly higher in patients with TTR values <60 %, in comparison with those presenting TTR values >75 % [7].
Studies have indicated that higher levels of knowledge regarding warfarin therapy are related to an improved stability of INR and increased TTR values [6,8]. It is therefore essential that patients using warfarin have adequate knowledge about the therapeutic objective (indication and effectiveness), the process of use (dosage, therapeutic regimen, route of administration, and duration of treatment), safety (adverse events, precautions, contraindications, and interactions), as well as the conservation of the drug [9]. In this sense, the evaluation of the patient's level of knowledge using validated instruments becomes necessary for identifying deficiencies and guide educational activities to create a continuous system of quality improvement of patient monitoring and safety [10].
Among the few validated tools to evaluate patients' knowledge regarding the use of warfarin, we can highlight the Oral Anticoagulation Knowledge (OAK) Test, which was validated for use in the United States of America [8] and Malaysia [11], and showed good psychometric properties. The OAK Test was crossculturally adapted into Brazilian Portuguese, presenting a semantic and conceptual equivalence of items in relation to the original version [12]. It is recommended that the psychometric properties of this adapted version be evaluated to ensure its validity and reliability to achieve the proposed objectives [13]. Therefore, the present study aimed at evaluating the psychometric properties of the Brazilian version of the OAK Test.

Study design and setting
This is a methodological study designed according to the recommendations outlined in the literature for the evaluation and validation of the instrument in question [14]. This study was conducted in an anticoagulation clinic of a university hospital located in Belo Horizonte, southeastern Brazil, which is a regional reference center that offers medium and high complexity health assistance.

Sample
The number of participants in the phases of test and retest of the questionnaire was according to the recommendation to include at least 10 respondents for each question of the instrument under validation and 30 % of the initial sample for retest [15]. Thus, the sample comprised 201 consecutive patients who were recruited during the service office hours, were aged 18 years or older, were Brazilians, and were on warfarin for at least one year. For the retest, 60 of these patients were randomly selected for a new interview that was conducted within two to three months after the first interview, a period equal to that used in the study in which the original version of the OAK Test was developed [8]. It is noteworthy that the correct answers to the test were not revealed to the participants during any of the steps of the study.

The instrument
The psychometric properties of the Brazilian version of the OAK Test [12] were evaluated in this study by employing internationally recognized methodology [13,16]. The instrument consists of 20 questions with four possible answers, and only one correct option. Each correct answer equals one point, leading to a total score ranging from zero to 20 points. A higher score indicates better level of knowledge about oral anticoagulant therapy. Content and face validity were evaluated by an expert committee that examined the relevance of each item of the instrument and their effectiveness in measuring what they propose to measure, respectively [12].

Evaluation of psychometric properties
The evaluation of the psychometric properties of the Brazilian version of the OAK Test included the analysis of reliability and validity. The instrument's reliability was evaluated by assessing the internal consistency (Kuder-Richardson coefficient) and reproducibility (test-retest reliability).
For the analysis of the validity, it was hypothesized that there would be a positive correlation, of moderate to strong intensity, between the correctness levels of the OAK Test and the TTR values.

Data collection
Data were collected between January to April 2015 by means of individual interviews and consultation of the hospital records of the participants. Instruments were applied for sociodemographic (age, sex, education, and income) and clinical characterization (TTR), and the Brazilian version of the OAK Test was administered. Interviews were conducted in a standardized manner by a single interviewer to avoid interference with the respondents' answers. For TTR calculation, all patients' INR values obtained up to one year prior to interview were interpolated by using the method proposed by Rosendaal. This method assumes that there is a linear variation between values over time and the result correspond to the percentage of time that anticoagulation has been within therapeutic range [17].

Statistical analysis
The database was validated by double entry in the EpiData program (version 3.1, EpiData Assoc, Denmark) and analyzed using the Statistical Package for Social Sciences (SPSS for Windows, version 20.0, SPSS Inc., Chicago, IL). Descriptive analyses were performed for all variables using measures of dispersion, frequency, and central tendency.
Statistical analysis involved the evaluation of internal consistency by the Kuder-Richardson coefficient, for which values above 0.70 reflect a high degree of internal consistency [15,18]. Temporal stability was verified by the values of the intraclass correlation coefficients (ICC) of the OAK final score (quantitative variable with scores varying from zero to 20) and by calculating the Cohen's Kappa coefficients for each dichotomized question of the OAK Test (correctness and error).
We analyzed the Spearman correlation between the OAK Test correctness levels and TTR to evaluate validity. The results were analyzed according to the following classification: values below 0.30, even when statistically significant, are of little clinical applicability, those between 0.30 and 0.50 are moderate, and those above 0.5 are of strong magnitude [19].
We also evaluated the correlations between the OAK Test and age (years), educational level (years) and income (Brazilian Reais). The gender-based comparison of the OAK Test values was conducted using the Mann-Whitney test, considering that the instrument did not show normal distribution (Kolmogorov-Smirnov test, p < 0.001). The statistical significance was declared at p = 0.05.
The description of knowledge was categorized according to previous studies that used the same instrument validated in other countries. The categories have included scores <50 % as low, 50-75 % as average, and >75 % as high level of knowledge regarding oral anticoagulation [11,20]. The quality of oral anticoagulation was evaluated by stratified TTR, according to a previous study, classified into three groups: <60 %, 60 %-75 % and >75 % [7].

Results
The final analysis included 201 participants under treatment with warfarin. The characteristics of the patients have been presented in Table 1. The average number of correct answers to the OAK Test was 63 % (±22) ( Table 1). The analysis of the instrument's internal consistency revealed a Kuder-Richardson coefficient of 0.818 for the total score. The exclusion of any instrument items showed small changes in the Kuder-Richardson coefficient, which ranged from 0.802 to 0.821 ( Table 2).
The intraclass correlation coefficient of the total score of the OAK Test was 0.967. Table 3 shows the Cohen's Kappa coefficients of the test and retest performed, revealing temporal stability ranging between good and perfect. By using a specific evaluation form, most items of the instrument, 19 (95 %), were considered relevant for the study population. Only item 2 was regarded irrelevant to the study in question, but it was maintained in the instrument.
The validity revealed a strong positive correlation between the values of the level of knowledge measured by the OAK Test and the TTR (r s = 0.780; p = 0.001). There was a positive and weak correlation between the OAK Test score and education (r s = 0.191; p = 0.007) and income (r s = 0.170; p = 0.016). Other analyses were performed and showed a negative but weak and nonsignificant correlation between the OAK Test score and age (r s = −0.116; p = 0.100). There was no association between the OAK Test score and the gender of the participants (Mann-Whitney Test, p = 0.182).

Discussion
This study systematically evaluated the psychometric properties of the Brazilian version of the OAK Test [12].
The results of this evaluation demonstrated adequate   performance of the instrument, supporting its validation for use in Brazil. The reliability of an instrument is related to the ability to measure an attribute accurately, over time, and the consistency and stability of the attribute measured [19]. In this sense, the Brazilian version of the OAK Test retained the reliability and validity presented by the original English instrument. Reliability was confirmed by internal consistency and test-retest measures and the content, face [12], and validities were also confirmed.
In terms of the application of the OAK Test, good internal consistency was observed as the temporal stability ranged between good and perfect. These results demonstrate that the instrument in question is reliable and has adequate temporal stability, corroborating the findings of the original study in which the instrument was developed [8], and another study that obtained similar results [11].
Through the initial hypothesis that there would be a positive correlation, of moderate to strong intensity, between the correctness levels of the OAK Test and the TTR values, the validity of the instrument was proven, since there was a strong positive correlation (r s = 0.780). This hypothesis was also confirmed by other investigations, demonstrating that the greater the patient's knowledge about treatment with warfarin, the better was the control of oral anticoagulation, as expressed by higher TTR values [11,20]. We observed that formal education and income showed a weak association with the OAK Test, which is similar to the results of other studies that have not demonstrated significant associations between knowledge and these variables as well [8,21].
By applying the OAK Test, it was possible to verify the level of knowledge of the patients treated with warfarin in a public hospital in Brazil. The average correctness of the test answers was 63 %, which is in line with that found in other studies [8,22]; besides 71 % of patients had average or high level of knowledge. However, a similar study conducted in Malaysia identified an average of only 11.2 % of correctness [11]. Studies that used other tests also revealed differences in the level of knowledge of participants. Both low levels of knowledge [23,24], and good levels of knowledge were observed [10,25]. Such differences may be related to sociodemographic and clinical particularities of the patients included, the quality of anticoagulation services, or methodological differences employed in the application of knowledge tests.
In the present study, it was observed that a deficit in knowledge about the drug and dietary interactions with warfarin correlated with a correctness rate of less than 50 %. These data are in line with those of another study carried out in an anticoagulation clinic, which showed meaningful gaps in the knowledge regarding the drug and dietary interactions [22]. These findings signal the importance of implementing educational strategies to improve the understanding of these issues using approaches that can contribute to modifying patients' behavior regarding self-care and appropriate lifestyle choices.
Most patients had TTR lower than 60 %, which is a worrisome result since the rates of bleeding and mortality are significantly higher in patients with TTR < 60 % when compared to those with TTR > 75 % [7]. Laboratorial monitoring is necessary to guide dose adjustments in order to achieve INR target. Interventions focused on INR control, and consequent TTR elevation, contribute to the increase of effectiveness of drug therapy and the minimization of adverse drug events. In this sense, the use of valid and reliable instruments allow health teams to evaluate the level of knowledge of patients and identify knowledge deficits on important aspects related to the treatment with warfarin substantiating appropriate decision for patient management [1].
Regarding the limitations of this study, it is noteworthy that, as there are no instruments aimed at evaluating patients' knowledge about therapy with warfarin already validated for the Brazilian population, the criterion validity could not be established, as this would have required using a gold standard instrument. Added to this, the fact that although the sample size was sufficient to demonstrate the performance of the instrument, this relatively small number of individuals can also limit the ability to generalize the results for different groups of Brazilian patients on warfarin.
As another limitation of the study, OAK Test was originally designed to be self-administered by individuals with at least seven years of education. However, due to the low level of education presented by several individuals enrolled in this study, we preferred to administer the instrument by patient interview, which has extended the time of application.
However, the use of a rigorous methodological process to evaluate the psychometric properties of the Brazilian version of the OAK Test tried to maintain the reliability and validity of the original instrument in English. In addition, this instrument is useful, quick, and easy-toapply in clinical practice, by means of individual interviews with patients with different levels of education. Based on a previous publication [12] we identified good content and face validity of the instrument. The instrument was proved to be effective in assessing what it proposes to measure, and that most of the items analyzed are relevant. This finding is in agreement with that of other studies with similar goals [8,11]. Item 2 of the instrument was a reason of debate among the authors. Most of the population studied uses only the 5 mg warfarin tablet that belongs to the list of drugs distributed by the National Health System in Brazil, and that is not distinguishable by the color that identifies the drug concentration as it happens in the US. Thus, participants may present difficulties in understanding this question. However, we made a decision to retain Item 2 in the final version to provide a broader scope of the instrument to be used not only by public health services but also by private institutions where warfarin brands that allow patients to distinguish different drug concentrations by color may be preferably prescribed.
The results of the test may assist the stratification of patients and the improvement of planning and individualization of educational practices according to the needs of the patients. The need for further studies to check the performance of the instrument in individuals with characteristics different from the study group is indicated.