- Research
- Open access
- Published:
Assessment of psychometric properties of the Brazilian version of the oral anticoagulation knowledge test
Health and Quality of Life Outcomes volume 14, Article number: 96 (2016)
Abstract
Background
The aim of this study was to evaluate the psychometric properties of the Brazilian version of the Oral Anticoagulation Knowledge (OAK) Test.
Methods
This study, conducted in an anticoagulation clinic, included 201 Brazilian participants aged over 18 years, who had been using warfarin for more than two months. The reliability of the instrument was evaluated by assessing internal consistency (Kuder-Richardson coefficient) and reproducibility (test-retest reliability). The validity was evaluated by hypothesizing that there would be a positive correlation of moderate to strong intensity between the correctness levels of the OAK Test and time within therapeutic range (TTR) values, which is a measure used to evaluate the quality of oral anticoagulation.
Results
The instrument exhibited good psychometric properties. The total a Kuder-Richardson coefficient value was 0.818 and intraclass correlation coefficient was 0.967. The validity revealed a strong positive correlation between the values of the level of knowledge, as measured by the OAK Test and the TTR values (rs = 0.780).
Conclusion
The instrument proved to be a reliable and valid tool for evaluating the knowledge of Brazilian patients on oral anticoagulation therapy with warfarin. This instrument may be incorporated into the practice of health care for substantiating the structuring of educational activities to ensure the improvement of knowledge about the use of warfarin, thereby increasing the effectiveness and safety of treatment.
Background
Warfarin is an oral anticoagulant, coumarinic derivative used worldwide for primary and secondary prophylaxis of thromboembolism. Its efficacy and safety have been widely studied for several indications through well-conducted clinical trials. The indication of oral anticoagulation is increasing due to population aging and the consequent increase in the incidence of thromboembolic diseases [1–5].
The management of warfarin is quite complex due to its narrow therapeutic range and wide dose–response variability. Among the interfering factors in the anticoagulant effect, we can cite the presence of genetic polymorphisms, drug and dietary interactions of potential clinical significance, as well as changes in the patient’s condition as some of the common ones [1, 6].
For proper dose management, laboratory monitoring is conducted by means of the International Normalized Ratio (INR) calculated from the prothrombin activity. The target INR is considered between 2.00 and 3.00 for most of warfarin indications. Subtherapeutic results were related to the occurrence of venous thromboembolism and higher values, to increased risk of bleeding [1]. To evaluate the quality of anticoagulant therapy, the percentage of time of INR in the therapeutic range is calculated (time within therapeutic range; TTR), the bleeding and mortality rates being significantly higher in patients with TTR values <60 %, in comparison with those presenting TTR values >75 % [7].
Studies have indicated that higher levels of knowledge regarding warfarin therapy are related to an improved stability of INR and increased TTR values [6, 8]. It is therefore essential that patients using warfarin have adequate knowledge about the therapeutic objective (indication and effectiveness), the process of use (dosage, therapeutic regimen, route of administration, and duration of treatment), safety (adverse events, precautions, contraindications, and interactions), as well as the conservation of the drug [9]. In this sense, the evaluation of the patient’s level of knowledge using validated instruments becomes necessary for identifying deficiencies and guide educational activities to create a continuous system of quality improvement of patient monitoring and safety [10].
Among the few validated tools to evaluate patients’ knowledge regarding the use of warfarin, we can highlight the Oral Anticoagulation Knowledge (OAK) Test, which was validated for use in the United States of America [8] and Malaysia [11], and showed good psychometric properties. The OAK Test was cross-culturally adapted into Brazilian Portuguese, presenting a semantic and conceptual equivalence of items in relation to the original version [12]. It is recommended that the psychometric properties of this adapted version be evaluated to ensure its validity and reliability to achieve the proposed objectives [13]. Therefore, the present study aimed at evaluating the psychometric properties of the Brazilian version of the OAK Test.
Methods
Study design and setting
This is a methodological study designed according to the recommendations outlined in the literature for the evaluation and validation of the instrument in question [14]. This study was conducted in an anticoagulation clinic of a university hospital located in Belo Horizonte, southeastern Brazil, which is a regional reference center that offers medium and high complexity health assistance.
Sample
The number of participants in the phases of test and retest of the questionnaire was according to the recommendation to include at least 10 respondents for each question of the instrument under validation and 30 % of the initial sample for retest [15]. Thus, the sample comprised 201 consecutive patients who were recruited during the service office hours, were aged 18 years or older, were Brazilians, and were on warfarin for at least one year. For the retest, 60 of these patients were randomly selected for a new interview that was conducted within two to three months after the first interview, a period equal to that used in the study in which the original version of the OAK Test was developed [8]. It is noteworthy that the correct answers to the test were not revealed to the participants during any of the steps of the study.
The instrument
The psychometric properties of the Brazilian version of the OAK Test [12] were evaluated in this study by employing internationally recognized methodology [13, 16]. The instrument consists of 20 questions with four possible answers, and only one correct option. Each correct answer equals one point, leading to a total score ranging from zero to 20 points. A higher score indicates better level of knowledge about oral anticoagulant therapy. Content and face validity were evaluated by an expert committee that examined the relevance of each item of the instrument and their effectiveness in measuring what they propose to measure, respectively [12].
Evaluation of psychometric properties
The evaluation of the psychometric properties of the Brazilian version of the OAK Test included the analysis of reliability and validity. The instrument’s reliability was evaluated by assessing the internal consistency (Kuder-Richardson coefficient) and reproducibility (test-retest reliability).
For the analysis of the validity, it was hypothesized that there would be a positive correlation, of moderate to strong intensity, between the correctness levels of the OAK Test and the TTR values.
Data collection
Data were collected between January to April 2015 by means of individual interviews and consultation of the hospital records of the participants. Instruments were applied for sociodemographic (age, sex, education, and income) and clinical characterization (TTR), and the Brazilian version of the OAK Test was administered. Interviews were conducted in a standardized manner by a single interviewer to avoid interference with the respondents’ answers. For TTR calculation, all patients’ INR values obtained up to one year prior to interview were interpolated by using the method proposed by Rosendaal. This method assumes that there is a linear variation between values over time and the result correspond to the percentage of time that anticoagulation has been within therapeutic range [17].
Statistical analysis
The database was validated by double entry in the EpiData program (version 3.1, EpiData Assoc, Denmark) and analyzed using the Statistical Package for Social Sciences (SPSS for Windows, version 20.0, SPSS Inc., Chicago, IL). Descriptive analyses were performed for all variables using measures of dispersion, frequency, and central tendency.
Statistical analysis involved the evaluation of internal consistency by the Kuder-Richardson coefficient, for which values above 0.70 reflect a high degree of internal consistency [15, 18]. Temporal stability was verified by the values of the intraclass correlation coefficients (ICC) of the OAK final score (quantitative variable with scores varying from zero to 20) and by calculating the Cohen’s Kappa coefficients for each dichotomized question of the OAK Test (correctness and error).
We analyzed the Spearman correlation between the OAK Test correctness levels and TTR to evaluate validity. The results were analyzed according to the following classification: values below 0.30, even when statistically significant, are of little clinical applicability, those between 0.30 and 0.50 are moderate, and those above 0.5 are of strong magnitude [19].
We also evaluated the correlations between the OAK Test and age (years), educational level (years) and income (Brazilian Reais). The gender-based comparison of the OAK Test values was conducted using the Mann–Whitney test, considering that the instrument did not show normal distribution (Kolmogorov-Smirnov test, p < 0.001). The statistical significance was declared at p = 0.05.
The description of knowledge was categorized according to previous studies that used the same instrument validated in other countries. The categories have included scores <50 % as low, 50–75 % as average, and >75 % as high level of knowledge regarding oral anticoagulation [11, 20]. The quality of oral anticoagulation was evaluated by stratified TTR, according to a previous study, classified into three groups: <60 %, 60 %–75 % and >75 % [7].
Results
The final analysis included 201 participants under treatment with warfarin. The characteristics of the patients have been presented in Table 1. The average number of correct answers to the OAK Test was 63 % (±22) (Table 1). The analysis of the instrument’s internal consistency revealed a Kuder-Richardson coefficient of 0.818 for the total score. The exclusion of any instrument items showed small changes in the Kuder-Richardson coefficient, which ranged from 0.802 to 0.821 (Table 2).
The intraclass correlation coefficient of the total score of the OAK Test was 0.967. Table 3 shows the Cohen's Kappa coefficients of the test and retest performed, revealing temporal stability ranging between good and perfect. By using a specific evaluation form, most items of the instrument, 19 (95 %), were considered relevant for the study population. Only item 2 was regarded irrelevant to the study in question, but it was maintained in the instrument.
The validity revealed a strong positive correlation between the values of the level of knowledge measured by the OAK Test and the TTR (rs = 0.780; p = 0.001). There was a positive and weak correlation between the OAK Test score and education (rs = 0.191; p = 0.007) and income (rs = 0.170; p = 0.016). Other analyses were performed and showed a negative but weak and non-significant correlation between the OAK Test score and age (rs = −0.116; p = 0.100). There was no association between the OAK Test score and the gender of the participants (Mann–Whitney Test, p = 0.182).
Discussion
This study systematically evaluated the psychometric properties of the Brazilian version of the OAK Test [12]. The results of this evaluation demonstrated adequate performance of the instrument, supporting its validation for use in Brazil.
The reliability of an instrument is related to the ability to measure an attribute accurately, over time, and the consistency and stability of the attribute measured [19]. In this sense, the Brazilian version of the OAK Test retained the reliability and validity presented by the original English instrument. Reliability was confirmed by internal consistency and test-retest measures and the content, face [12], and validities were also confirmed.
In terms of the application of the OAK Test, good internal consistency was observed as the temporal stability ranged between good and perfect. These results demonstrate that the instrument in question is reliable and has adequate temporal stability, corroborating the findings of the original study in which the instrument was developed [8], and another study that obtained similar results [11].
Through the initial hypothesis that there would be a positive correlation, of moderate to strong intensity, between the correctness levels of the OAK Test and the TTR values, the validity of the instrument was proven, since there was a strong positive correlation (rs = 0.780). This hypothesis was also confirmed by other investigations, demonstrating that the greater the patient’s knowledge about treatment with warfarin, the better was the control of oral anticoagulation, as expressed by higher TTR values [11, 20]. We observed that formal education and income showed a weak association with the OAK Test, which is similar to the results of other studies that have not demonstrated significant associations between knowledge and these variables as well [8, 21].
By applying the OAK Test, it was possible to verify the level of knowledge of the patients treated with warfarin in a public hospital in Brazil. The average correctness of the test answers was 63 %, which is in line with that found in other studies [8, 22]; besides 71 % of patients had average or high level of knowledge. However, a similar study conducted in Malaysia identified an average of only 11.2 % of correctness [11]. Studies that used other tests also revealed differences in the level of knowledge of participants. Both low levels of knowledge [23, 24], and good levels of knowledge were observed [10, 25]. Such differences may be related to sociodemographic and clinical particularities of the patients included, the quality of anticoagulation services, or methodological differences employed in the application of knowledge tests.
In the present study, it was observed that a deficit in knowledge about the drug and dietary interactions with warfarin correlated with a correctness rate of less than 50 %. These data are in line with those of another study carried out in an anticoagulation clinic, which showed meaningful gaps in the knowledge regarding the drug and dietary interactions [22]. These findings signal the importance of implementing educational strategies to improve the understanding of these issues using approaches that can contribute to modifying patients’ behavior regarding self-care and appropriate lifestyle choices.
Most patients had TTR lower than 60 %, which is a worrisome result since the rates of bleeding and mortality are significantly higher in patients with TTR < 60 % when compared to those with TTR > 75 % [7]. Laboratorial monitoring is necessary to guide dose adjustments in order to achieve INR target. Interventions focused on INR control, and consequent TTR elevation, contribute to the increase of effectiveness of drug therapy and the minimization of adverse drug events. In this sense, the use of valid and reliable instruments allow health teams to evaluate the level of knowledge of patients and identify knowledge deficits on important aspects related to the treatment with warfarin substantiating appropriate decision for patient management [1].
Regarding the limitations of this study, it is noteworthy that, as there are no instruments aimed at evaluating patients’ knowledge about therapy with warfarin already validated for the Brazilian population, the criterion validity could not be established, as this would have required using a gold standard instrument. Added to this, the fact that although the sample size was sufficient to demonstrate the performance of the instrument, this relatively small number of individuals can also limit the ability to generalize the results for different groups of Brazilian patients on warfarin.
As another limitation of the study, OAK Test was originally designed to be self-administered by individuals with at least seven years of education. However, due to the low level of education presented by several individuals enrolled in this study, we preferred to administer the instrument by patient interview, which has extended the time of application.
However, the use of a rigorous methodological process to evaluate the psychometric properties of the Brazilian version of the OAK Test tried to maintain the reliability and validity of the original instrument in English. In addition, this instrument is useful, quick, and easy-to-apply in clinical practice, by means of individual interviews with patients with different levels of education. Based on a previous publication [12] we identified good content and face validity of the instrument. The instrument was proved to be effective in assessing what it proposes to measure, and that most of the items analyzed are relevant. This finding is in agreement with that of other studies with similar goals [8, 11]. Item 2 of the instrument was a reason of debate among the authors. Most of the population studied uses only the 5 mg warfarin tablet that belongs to the list of drugs distributed by the National Health System in Brazil, and that is not distinguishable by the color that identifies the drug concentration as it happens in the US. Thus, participants may present difficulties in understanding this question. However, we made a decision to retain Item 2 in the final version to provide a broader scope of the instrument to be used not only by public health services but also by private institutions where warfarin brands that allow patients to distinguish different drug concentrations by color may be preferably prescribed.
The results of the test may assist the stratification of patients and the improvement of planning and individualization of educational practices according to the needs of the patients. The need for further studies to check the performance of the instrument in individuals with characteristics different from the study group is indicated.
Conclusion
The Brazilian version of the OAK Test showed good psychometric properties and was found to be a reliable and valid instrument for evaluating the knowledge of Brazilian patients on oral anticoagulant therapy using warfarin. The results generated with the use of the instrument may be incorporated into the practice of public health care, in order to structure health education activities to ensure improvement in the knowledge of the therapy proposed, which may help increase the effectiveness and safety of the treatment.
The use of the instrument will also aid the analysis of the relationship between the knowledge of the patients and the quality of the anticoagulation treatment using warfarin, as well as the comparison of research findings between different countries. It favors, then, the communication between health professionals and patients, and contributes to the rapid identification of problems and priority needs, as well as to decision-making.
Abbreviations
ICC, intraclass correlation coefficients; INR, international normalized ratio; OAK, oral anticoagulation knowledge; PT, prothrombin time; Rs, Spearman correlation; SD, standard deviation; SPSS, Statistical Package for Social Sciences TTR, time within therapeutic range
References
Ageno W, Gallus AS, Wittkowsky A, Crowther M, Hylek EM, Palareti G. Oral anticoagulant therapy: antithrombotic therapy and prevention of thrombosis, 9th ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest. 2012;141:e44S–88S.
You JJ, Singer DE, Howard PA, Lane DA, Eckman MH, Fang MC, et al. Antithrombotic therapy for atrial fibrillation: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest. 2012;141:531S–75S.
Whitlock RP, Sun JC, Fremes SE, Rubens FD, Teoh KH. Antithrombotic and thrombolytic therapy for valvular disease: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest. 2012;141:576S–600S.
Holbrook A, Schulman S, Witt DM, Vandvik PO, Fish J, Kovacs MJ, et al. Evidence-based management of anticoagulant therapy: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest. 2012;141:152–84.
Wiedermann CJ, Stockner I. Warfarin-induced bleeding complications – clinical presentation and therapeutic options. Thromb Res. 2008;122:13–8.
Ansell J, Hirsh J, Hylek E, Jacobson A, Crowther M, Palareti G. American College of Chest Physicians Pharmacology and management of the vitamin K antagonists: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest. 2008;133:160–98.
White HD, Gruber M, Feyzi J, Kaatz S, Tse HF, Husted S, Albers GW. Comparison of outcomes among patients randomized to warfarin therapy according to anticoagulant control: results from SPORTIF III and V. Arch Intern Med. 2007;167:239–45.
Zeolla MM, Brodeur MR, Dominelli A, Haines ST, Allie ND. Development and validation of an instrument to determine patient knowledge: the oral anticoagulation knowledge test. Ann Pharmacother. 2006;40:633–8.
Delgado PG, Garralda MAG, Parejo MIB, Lozano FF, Martínez FM. Validación de un cuestionario para medir el conocimiento de los pacientes sobre sus medicamentos. Aten Primaria. 2009;41:661–9.
Baker JW, Pierce KL, Ryals CA. INR goal attainment and oral anticoagulation knowledge of patients enrolled in an anticoagulation clinic in a Veterans Affairs medical center. J Manag Care Pharm. 2011;17:133–42.
Matalaqah LM, Radaideh K, Sulaiman SASS, Hassali MA, Kader MASAK. An instrument to measure anticoagulation knowledge among Malaysian community: a translation and validation study of the Oral Anticoagulation Knowledge (OAK) Test. J Pharm Biomed Sci. 2013;3:30–7.
Martins MAP, Praxedes MFS, Paiva S, Ribeiro DD, Marcolino MS, Guimarães MHN. Adaptação Transcultural do Oral Anticoagulation Knowledge Test para o português do Brasil. Cienc Saude Colet. 2016:23. [in press]. Available at: http://www.cienciaesaudecoletiva.com.br/artigos/artigo_int.php?id_artigo=15389. Accessed 15 June 2016.
Beaton D, Bombardier C, Guillemin F, Ferraz MB. Recommendations for the Cross- Cultural Adaptation of the DASH & Quick DASH Outcome Measures. Institute for Work & Health. 2007. Available at http://www.dash.iwh.on.ca/sites/dash/files/downloads/cross_cultural_adaptation_2007.pdf. Accessed 1 June 2016.
Roberts P, Priest H, Traynor M. Reability and validity in research. Nurs Stand. 2006;20:41–5.
Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. New York: Oxford University Press; 1989.
Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. Chichester: John Wiley & Sons; 2007.
Rosendaal FR, Cannegieter SC, van der Meer FJ, Briët E. A method to determine the optimal intensity of oral anticoagulant therapy. Thromb Haemost. 1993;69:236–9.
Sijtsma K. On the use, the misuse, and the very limited usefulness of Cronbach’s Alpha. Psychometrika. 2009;74:107–20.
Ajzen I, Fishbein M. Understanding Attitudes and Predicting Social Behavior. Englewood Cliffs (NJ): Prentice Hall; 1998.
Khudair IF, Hanssens YI. Evaluation of patients’ knowledge on warfarin in outpatient anticoagulation clinics in a teaching hospital in Qatar. Saudi Med J. 2010; 31:672–7.
Mayet AY. Association between oral anticoagulation knowledge, anticoagulation control, and demographic characteristics of patients attending an anticoagulation clinic in Saudi Arabia: a cross-sectional prospective evaluation. Trop J Pharm Res. 2015;14:1285–91.
Alphonsa A, Sharma KK, Sharma G, Bhatia R. Knowledge regarding oral anticoagulation therapy among patients with stroke and those at high risk of thromboembolic events. J Stroke Cerebrovasc Dis. 2015;24:668–72.
Janoly-Duménil A, Bourne C, Loiseau K, Luauté J, Sancho PO, Ciancia S, et al. Oral anticoagulant treatment - evaluating the knowledge of patients admitted in physical medicine and rehabilitation units. Ann Phys Rehabil Med. 2011;54:172–80.
Gras-Champel V, Voyer A, Lematte C, Pakula P, Roussel B, Lefrère JJ, Andréjak M. Assessment of the quality of oral anticoagulation management in patients admitted to Amiens University Hospital. Therapie. 2005;60:149–57.
Hasan SS, Shamala R, Syed IA, Basariah N, Chong DW, Mei TK, Chin OH. Factors affecting warfarin-related knowledge and INR control of patients attending physician- and pharmacist-managed anticoagulation clinics. J Pharm Pract. 2011;24:485–93.
Acknowledgements
This study was supported by the National Council for Scientific and Technological Development (CNPq), the Coordination for the Improvement of Higher Level Education Personnel (CAPES) and the State of Minas Gerais Research Foundation (FAPEMIG), Brazil, and the Pró-reitoria Pesquisa of the Universidade Federal Minas Gerais, Brasil.
Authors’ contribution
MFSP was responsible for the acquisition of data, analyse and interpretation of the data. MHNGA conceived and designed the study, analyse and interpretation of the data. SMP and JVMM were responsible for analysis and interpretation of data. MSM was responsible for the acquisition of data. MAPM conceived and designed the study, was responsible for the acquisition, analysis and interpretation of data. All authors contributed to the writing of the manuscript revising it critically for important intellectual content. All authors give final approval of the version to be published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Competing interest
The authors declare that they have no competing interests.
Ethics approval and consent to participate
All participants signed an informed consent form prior to the start of the study. The present study was conducted in accordance with the Declaration of Helsinki [17]. The Universidade Federal Minas Gerais Ethics Committee approved the study in 2013 (Number 08136613.4.0000.5149).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Praxedes, M.F.d.S., de Abreu, M.H.N.G., Paiva, S.M. et al. Assessment of psychometric properties of the Brazilian version of the oral anticoagulation knowledge test. Health Qual Life Outcomes 14, 96 (2016). https://doi.org/10.1186/s12955-016-0498-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12955-016-0498-3