Correlation between adherence rates measured by MEMS and self-reported questionnaires: a meta-analysis
© Shi et al; licensee BioMed Central Ltd. 2010
Received: 9 May 2010
Accepted: 13 September 2010
Published: 13 September 2010
It is vital to understand the associations between the medication event monitoring systems (MEMS) and self-reported questionnaires (SRQs) because both are often used to measure medication adherence and can produce different results. In addition, the economic implication of using alternative measures is important as the cost of electronic monitoring devices is not covered by insurance, while self-reports are the most practical and cost-effective method in the clinical settings. This meta-analysis examined the correlations of two measurements of medication adherence: MEMS and SRQs.
The literature search (1980-2009) used PubMed, OVID MEDLINE, PsycINFO (EBSCO), CINAHL (EBSCO), OVID HealthStar, EMBASE (Elsevier), and Cochrane Databases. Studies were included if the correlation coefficients [Pearson (rp) or Spearman (rs)] between adherences measured by both MEMS and SRQs were available or could be calculated from other statistics in the articles. Data were independently abstracted in duplicate with standardized protocol and abstraction form including 1) first author's name; 2) year of publication; 3) disease status of participants; 4) sample size; 5) mean age (year); 6) duration of trials (month); 7) SRQ names if available; 8) adherence (%) measured by MEMS; 9) adherence (%) measured by SRQ; 10) correlation coefficient and relative information, including p-value, 95% confidence interval (CI). A meta-analysis was conducted to pool the correlation coefficients using random-effect model.
Eleven studies (N = 1,684 patients) met the inclusion criteria. The mean of adherence measured by MEMS was 74.9% (range 53.4%-92.9%), versus 84.0% by SRQ (range 68.35%-95%). The correlation between adherence measured by MEMS and SRQs ranged from 0.24 to 0.87. The pooled correlation coefficient for 11 studies was 0.45 (p = 0.001, 95% confidence interval [95% CI]: 0.34-0.56). The subgroup meta-analysis on the seven studies reporting rp and four studies reporting rs reported the pooled correlation coefficient: 0.46 (p = 0.011, 95% CI: 0.33-0.59) and 0.43 (p = 0.0038, 95% CI: 0.23-0.64), respectively. No differences were found for other subgroup analyses.
Medication adherence measured by MEMS and SRQs tends to be at least moderately correlated, suggesting that SRQs give a good estimate of medication adherence.
Medical adherence is defined as the extent to which a patient's medication taking coincides with medical or health advice . Despite the proven efficacy of prescription drugs in reducing illness symptoms and preventing or minimizing associated complications, adherence rates to long-term pharmacotherapy tend to be approximately 50%, regardless of the illness, regimen or measurement criteria [2, 3]. In addition, the adherence rate varies with disease conditions, ranging from 15% to 93% as reported in the literature . Failure to adhere to medication regimens in the United States may cost as much as $300 billion annually, mediated by ineffectiveness of treatment and worsening of disease progression to poor outcomes, disease complications, medication adverse events, hospitalizations and re-hospitalizations, emergency department visits, and even death .
Measuring patient adherence to prescribed therapies is a first step towards developing a greater understanding of the potential for non-adherence and adverse outcomes. Two methods often used for this purpose are medication event monitoring systems (MEMS) and self-reported questionnaires (SRQs) . In spite of the availability of these measures, they present several technical challenges in measuring adherence. The MEMS is a medication vial cap that electronically records the date and time of bottle opening. It is also known as the "imperfect gold standard,"  due to its recording effectiveness in measurement of patient adherence. However, it could be time consuming, expensive, resource intensive and may not be suitable for all medications/formulations. Alternatively, self-reported questionnaires (SRQs) could be a very convenient choice for certain study designs. However, SRQs are subject to measurement bias such as social desirability, recall bias, and response bias; there have been mixed reports about the accuracy of self-reported adherence [8, 9]. Therefore, the accuracy in measuring medication adherence is uncertain for SRQs. This uncertainty further limits the credibility and validity of results obtained using SRQs. The previous literature reviews have focused on some qualitative work examining the correlation between SRQs and other measures such as pharmacy refill records, and interview [8–10]. Hence, it is vital to understand their associations relative to electronic measures of adherence such as MEMS. In addition, the economic implication of using alternative measures such as SRQs is also important as the cost of electronic monitoring devices is not covered by insurance, and thus these devices are not in routine use while self-reports are the most useful method in the clinical setting for practical interventions on non-adherence.
To advance the knowledge on relationships between different measurements, this study was the first study attempting to assess and quantify the correlation between MEMS and SRQs used for the measurement of medication adherence. Hence the objective of this study was to perform a meta-analysis to examine the correlation between MEMS and SRQs.
The literature search for monitoring devices citations from 1980-April 2009 was performed using search terms: patient compliance, medication adherence, treatment compliance, drug monitoring, drug therapy, electronic, digital, computer, monitor, monitoring, drug, drugs, pharmaceutical preparations, compliance, and medications. The search time frame was determined appropriately because the MEMS technology is available in 1980 s. We searched the following databases: PubMed, OVID MEDLINE, PsycINFO (EBSCO), CINAHL (EBSCO), OVID HealthStar, EMBASE (Elsevier), and Cochrane Databases of Systematic Reviews. The search was restricted to only human studies. All results of database search were merged in a single file for monitoring devices after the duplicates from the citation list were removed using the Endnote reference management tool. The initial search was performed in October of 2008, and updated in April 2009.
Inclusion criteria were (1): an article measuring medication adherence in clinical trials using both MEMS and SRQs; (2): the correlation coefficients (Pearson correlation coefficient (rp) or Spearman correlation coefficient (rs)) between the adherence rates measured by 2 different methods were available or could be calculated based on data published in the study reports.
Data were independently abstracted in duplicate with the standardized protocol and abstraction form. The study characteristics recorded were as follows: 1) first author's name; 2) year of publication; 3) disease status of participants; 4) sample size; 5) mean age (year); 6) duration of trials (month); 7) SRQ names if available or anonymous if a specific name is unavailable in the article; 8) adherence (%) measured by MEMS; 9) adherence (%) by SRQ; 10) correlation coefficient and relative information, including p-value, 95% confidence interval (CI). If data concerning the outcome were missing from an article, the investigators attempted to contact the primary author in order to obtain this missing data.
This meta-analysis was conducted according to the QUOROM guidelines  and PRISMA statements for the conduct and reporting of meta-analyses. Standard methods were used to calculate the pooled variance , which were calculated using CIs, p-values, t-statistics, or individual variances for the 2 types of adherence measurements. When a paper reported p < 0.05, p < 0.01, p < 0.001 or NS, we computed standard error of correlation coefficient with p values of 0.025, 0.005, 0.0005, 0.50, respectively, which likely gained a highly conservative estimate of the correlation coefficient . Both fixed-effects and DerSimonian and Laird's random effects models were used to calculate the pooled correlation coefficient . The 2 models approximate each other in the absence of heterogeneity. Heterogeneity was assessed using the chi-square test statistic. The random effect model was selected in this meta-analysis to synthesize correlation coefficient due to heterogeneity among the reviewed studies. We presented data for random-effects models throughout because of the different demographic characteristics, measurement methods, and study durations that were involved in the original trials. Publication bias was examined using the Begg-adjusted rank correlation test based on Kendall's score and Egger regression asymmetric test . Two subgroup post-hoc meta-analyses (studies reporting Pearson correlation coefficient and Spearman rank correlation coefficient; HIV studies vs. non-HIV studies) were also conducted to investigate potential differences, to address these naturally occurring groups in the population of studies. All analyses were conducted in STATA version 10.1 (Stata Corp., College Station, TX). The significance was set at 2-tailed p-values of 0.05.
Basic characteristics of studies
Basic characteristics of studies investigating the correlation between adherence rates measured by MEMS and SRQs
(rp or rs)
MOS, Morisky, VAS
Hypertension Diabetes Dysdipidemia
The correlation between adherence measured by MEMS and self-report questionnaires ranged from 0.24 to 0.87 for the 11 articles. We found 7 (63.6%) articles reporting Pearson correlation coefficient (rp) [17, 19–22, 24, 26] and 4 (36.4%) using Spearman rank correlation coefficient (rs) [16, 18, 23, 25].
The test for heterogeneity among the reviewed studies showed statistically significance in both categories (both p-values < 0.05) and the overall analysis (p = 0.001). Given the heterogeneity statistics presented, we only reported the results of the random-effects models as appropriate models for combining the individual studies.
As to publication bias, the Egger test showed the intercept in the regression of the standardized effect estimates against their precision was -0.75 (p = 0.40, 95% CI: -2.69-1.19) while the Begg test showed a marginally statistical significance (p = 0.052).
This is the first study to our best knowledge to quantify the correlation between the MEMS and SRQs for measuring adherence. We only found a small number of studies which have met the inclusion criteria for meta-analysis. We have found at least moderate correlation using a meta-regression model to pool the correlation coefficients from a total of 11 studies. These findings are consistent with previous studies on the moderate-to-high correlation of self-report with other measures of medication adherence [8–10, 27].
The systematic measurement of medication adherence is not routinely performed in outpatient settings due to a lack of reliable, convenient, economical methods for measuring adherence. The key advantages and limitations of various methods have been well summarized in the literature . The selection of medication adherence measures should tailor to the goals and resources available for the intended use and attributes of each type of measures. The 2 methods (MEMS and SRQs) collect different sets of information using different approaches and perspectives. When used together, the 2 methods complement each other giving confidence to the results, and tend to support the same conclusion. The meta-analysis summarizes and advances the field of adherence research through a side-to-side examination on two types of measurements within a study. Our finding of the pooled correlation coefficient of approximate 0.45 supports the need of multiple measures in the future adherence research because neither the MEMs nor SRQs can replace each other.
Furthermore, we have found that most of SRQs used in the meta-analysis were generic measures for medication adherence. For example, among these questionnaires, the Adult AIDS Clinical Trials Group (AACTG) instruments were most frequently used to evaluate clinical interventions, including the efficacy of drugs and drug combinations for treating HIV infection and HIV-associated illnesses . This is a standard self-administered questionnaire based on previous research on adherence. The questionnaire has been in use for over 10 years and patients demonstrated high satisfaction with its length [30, 31]. Similarly, the Morisky Scale is widely used to measure medication adherence in various populations (e.g., asthma , cancer , osteoporosis ). It was originally developed to measure hypertension and demonstrated high concurrent and predictive validity with regard to blood pressure control. The 4 items scale and its modified versions: 8- and 5-item scales are relatively simple to use and could be utilized to measure adherence [35, 36]. The Medication Adherence Self-Report Inventory (MASRI) is a 12-item questionnaire originally developed for HIV  and systemic lupus . However, in contrast to those well-known SRQs, most of the reviewed anonymous questionnaires (4 studies) also found low correlation with MEMS. Therefore, the validity of these anonymous questionnaires was not satisfactory for further development.
These findings must be interpreted in the context of the methodological weaknesses of this study, particularly for the heterogeneity of SRQs in the limited number of included studies. First, some studies have different definitions of adherence, in addition to the variations in study populations, disease states, and study duration. For example, most studies were in HIV patients where adherence is very high. In contrast, for 2 studies that examined non-symptomatic disease such as hypertension, correlation was low. Relatively recent methodological work has been published to assess adherence-response relationships, particularly when adherence is subject to measurement error [38, 39]. Secondly, the information on some SRQs is limited in the study reports, even without a specific name for the SRQs in 4 articles. Thirdly, 2 simplistic correlation measures, Pearson correlation coefficients and Spearman correlation coefficients, have been used in the meta-analysis. With the focus on the correlation coefficients, we had an implicit assumption that the association between electronically measured and self-reported adherence rates is linear. Obviously, a non-linear association is possible in the true association for research in the future. Additionally, we have tested the heterogeneity among the studies with a finding of significance. To address the issue of heterogeneity, which is quite common in meta-analysis, we have adopted random-effect models in the meta-analysis due to heterogeneity. We have also done two subgroup analyses to explore some possible influences of heterogeneity. The results of subgroup analyses did not find substantial differences because the results of 95% CI were overlapping for the pooled estimates. Lastly, measuring the level of agreement (not just association) between the MEMS and questionnaire data should be considered in future studies. The Pearson product-moment correlation is a measure of association, not agreement. Perhaps we may also extract an indicator such as the intraclass correlation.
Other limitations should also be mentioned. Although the authors have made attempts to identify all available studies for meta-analysis, there could have been studies that were missed. For example, a recent study was excluded due to the use of different measure of correlation coefficient Kendall tau . Inclusion of other self-reported methods such as diary, claims data, and clinical opinion could potentially be explored in the future. Lastly, the generalizability of the study results is limited as majority of the studies identified as measuring adherence were in HIV and few were in hypertension, schizophrenia and diabetes.
Based on the pooled estimate using meta-analysis, at least moderate correlation was found between adherences measured by MEMS and SRQs. Therefore, SRQs provide a good estimate of patient medication adherence. If possible, MEMS and SRQs should be used complementarily to get accurate measure for patient adherence.
- Osterberg L, Blaschke T: Adherence to medication.[see comment]. N Engl J Med 2005,353(5):487–497. 10.1056/NEJMra050100PubMedView ArticleGoogle Scholar
- World Health Organization: Adherence to Long-term Therapies--Evidence for Action. WHO Publications, Geneva 2003.Google Scholar
- DiMatteo MR: Variations in Patients' Adherence to Medical Recommendations: A Quantitative Review of 50 Years of Research. Medical Care 2004,42(3):200–209. 10.1097/01.mlr.0000114908.90348.f9PubMedView ArticleGoogle Scholar
- Singh N, Squier C, Sivek C, Wagener M, Nguyen MH, Yu VL: Determinants of compliance with antiretroviral therapy in patients with human immunodeficiency virus: prospective assessment with implications for enhancing compliance. AIDS Care 1996,8(3):261–269. 10.1080/09540129650125696PubMedView ArticleGoogle Scholar
- Bender BG, Rand C: Medication non-adherence and asthma treatment cost. Curr Opin Allergy Clin Immunol 2004,4(3):191–195. 10.1097/00130832-200406000-00009PubMedView ArticleGoogle Scholar
- Farmer KC: Methods for measuring and monitoring medication regimen adherence in clinical trials and clinical practice. Clinical Therapeutics 1999,21(6):1074–1090. 10.1016/S0149-2918(99)80026-5PubMedView ArticleGoogle Scholar
- Claxton AJ, Cramer J, Pierce C: A systematic review of the associations between dose regimens and medication compliance. Clinical Therapeutics 2001,23(8):1296–1310. 10.1016/S0149-2918(01)80109-0PubMedView ArticleGoogle Scholar
- Garber MC, Nau DP, Erickson SR, Aikens JE, Lawrence JB: The Concordance of Self-Report with Other Measures of Medication Adherence: A Summary of the Literature. Medical Care 2004,42(7):649–652. 10.1097/01.mlr.0000129496.05898.02PubMedView ArticleGoogle Scholar
- Cook CL, Wade WE, Martin BC, Perri M: Concordance among three self-reported measures of medication adherence and pharmacy refill records. Journal of the American Pharmacists Association: JAPhA 2005,45(2):151–159. 10.1331/1544345053623573PubMedView ArticleGoogle Scholar
- Wang PS, Benner JS, Glynn RJ, Winkelmayer WC, Mogun H, Avorn J: How well do patients report noncompliance with antihypertensive medications?: a comparison of self-report versus filled prescriptions. Pharmacoepidemiology and Drug Safety 2004,13(1):11–19. 10.1002/pds.819PubMedView ArticleGoogle Scholar
- Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF: Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses.[see comment]. Lancet 1999,354(9193):1896–1900. 10.1016/S0140-6736(99)04149-5PubMedView ArticleGoogle Scholar
- Rice J: Mathematical Statistics and Data Analysis. Belmont, MA: Duxbury Press; 1988.Google Scholar
- Chida Y, M H: An association of adverse psychosocial factors with diabetes mellitus: a meta-analytic review of longitudinal cohort studies. Diabetologia 2008,51(12):2168–2178. 10.1007/s00125-008-1154-1PubMedView ArticleGoogle Scholar
- DerSimonian R, Laird N: Meta-analysis in clinical trials. Control Clin Trials 1986,7(3):177–188. 10.1016/0197-2456(86)90046-2PubMedView ArticleGoogle Scholar
- Begg C: Publication Bias. In The Handbook of Research Synthesis. Edited by: Cooper H, Hedges L. New York, NY: Russell Sage Foundation; 1994:399–409.Google Scholar
- Hugen PWH, Langebeek N, Burger DM, Zomer B, Van Leusen R, Schuurman R, Koopmans PP, Hekster YA: Assessment of adherence to HIV protease inhibitors: Comparison and combination of various methods, including MEMS (electronic monitoring), patient and nurse report, and therapeutic drug monitoring. Journal of Acquired Immune Deficiency Syndromes 2002,30(3):324–334.PubMedView ArticleGoogle Scholar
- Walsh JC, Mandalia S, Gazzard BG: Responses to a 1 month self-report on adherence to antiretroviral therapy are consistent with electronic data and virological treatment outcome. AIDS 2002,16(2):269–277. 10.1097/00002030-200201250-00017PubMedView ArticleGoogle Scholar
- Hamilton GA: Measuring adherence in a hypertension clinical trial. European Journal of Cardiovascular Nursing 2003,2(3):219–228. 10.1016/S1474-5151(03)00058-6PubMedView ArticleGoogle Scholar
- Oyugi JH, Byakika-Tusiime J, Charlebois ED, Kityo C, Mugerwa R, Mugyenyi P, Bangsberg DR: Multiple validated measures of adherence indicate high levels of adherence to generic HIV antiretroviral therapy in a resource-limited setting. Journal of Acquired Immune Deficiency Syndromes 2004,36(5):1100–1102. 10.1097/00126334-200408150-00014PubMedView ArticleGoogle Scholar
- Fletcher CV, Testa MA, Brundage RC, Chesney MA, Haubrich R, Acosta EP, Martinez A, Jiang H, Gulick RM: Four measures of antiretroviral medication adherence and virologic response in AIDS clinical trials group study 359. Journal of Acquired Immune Deficiency Syndromes 2005,40(3):301–306. 10.1097/01.qai.0000180078.53321.6aPubMedView ArticleGoogle Scholar
- Halkitis PN, Kutnick AH, Slater S: The Social Realities of Adherence to Protease Inhibitor Regimens: Substance Use, Health Care and Psychological States. Journal of Health Psychology 2005,10(4):545–558. 10.1177/1359105305053422PubMedView ArticleGoogle Scholar
- Jasti S, Siega-Riz AM, Cogswell ME, Hartzema AG: Correction for errors in measuring adherence to prenatal multivitamin/mineral supplement use among low-income women. J Nutr 2006,136(2):479–483.PubMedGoogle Scholar
- Byerly MJ, Nakonezny PA, Rush AJ: The Brief Adherence Rating Scale (BARS) validated against electronic monitoring in assessing the antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder. Schizophrenia Research 2008,100(1–3):60–69. 10.1016/j.schres.2007.12.470PubMedView ArticleGoogle Scholar
- Lu M, Safren SA, Skolnik PR, Rogers WH, Coady W, Hardy H, Wilson IB: Optimal recall period and response task for self-reported HIV medication adherence. AIDS and Behavior 2008,12(1):86–94. 10.1007/s10461-007-9261-4PubMedView ArticleGoogle Scholar
- Zeller A, Schroeder K, Peters TJ: Electronic pillboxes (MEMS) to assess the relationship between medication adherence and blood pressure control in primary care. Scandinavian Journal of Primary Health Care 2007,25(4):202–207. 10.1080/02813430701651954PubMed CentralPubMedView ArticleGoogle Scholar
- Arnsten JH, Demas PA, Farzadegan H, Grant RW, Gourevitch MN, Chang CJ, Buono D, Eckholdt H, Howard AA, Schoenbaum EE: Antiretroviral therapy adherence and viral suppression in HIV-infected drug users: Comparison of self-report and electronic monitoring. Clinical Infectious Diseases 2001,33(8):1417–1423. 10.1086/323201PubMed CentralPubMedView ArticleGoogle Scholar
- Velligan DI, Wang M, Diamond P, Glahn DC, Castillo D, Bendle S, Lam YWF, Ereshefsky L, Miller AL: Relationships among subjective and objective measures of adherence to oral antipsychotic medications. Psychiatric Services 2007,58(9):1187–1192. 10.1176/appi.ps.58.9.1187PubMedView ArticleGoogle Scholar
- Hawkshead J, Krousel-Wood MA: Techniques for measuring medication adherence in hypertensive patients in outpatient settings. Advantages and limitations. Disease Management & Health Outcomes 2007,15(2):109–118.View ArticleGoogle Scholar
- Chesney MA, Ickovics JR, Chambers DB, Gifford AL, Neidig J, Zwickl B, AW. W: Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: the AACTG adherence instruments. Patient Care Committee & Adherence Working Group of the Outcomes Committee of the Adult AIDS Clinical Trials Group (AACTG). AIDS Care 2000,12(3):255–266. 10.1080/09540120050042891PubMedView ArticleGoogle Scholar
- Fang MC, Machtinger EL, Wang F, Schillinger D: Health Literacy and Anticoagulation-related Outcomes Among Patients Taking Warfarin. Journal of General Internal Medicine 2006,21(8):841–846. 10.1111/j.1525-1497.2006.00537.xPubMed CentralPubMedView ArticleGoogle Scholar
- Reinhard MJ, Hinkin CH, Barclay TR, Levine AJ, Marion S, Castellon SA, Longshore D, Newton T, Durvasula RS, Lam MN, et al.: Discrepancies Between Self-Report and Objective Measures for Stimulant Drug Use in HIV: Cognitive, Medication Adherence and Psychological Correlates. Addict Behav 2007,32(12):2727–2736. 10.1016/j.addbeh.2007.04.006PubMed CentralPubMedView ArticleGoogle Scholar
- Joshi AV, Madhavan SS, Ambegaonkar A, Smith M, Scott V, Dedhia H: Association of Medication Adherence with Workplace Productivity and Health-Related Quality of Life in Patients with Asthma. Journal of Asthma 2006,43(7):521–526. 10.1080/02770900600857010PubMedView ArticleGoogle Scholar
- Larizza MA, Dooley MJ, Stewart K, Kong DCM: Factors influencing adherence to molecular therapies in haematology-oncology outpatients. Journal of Pharmacy Practice and Research 2006,36(2):115–118.View ArticleGoogle Scholar
- Guilera M, Fuentes M, Grifols M, Ferrer J, Badia X, OPTIMA study investigators: Does an educational leaflet improve self-reported adherence to therapy in osteoporosis? The OPTIMA study.Google Scholar
- Morisky DE: Nonadherence to medical recommendations for hypertensive patients: Problems and potential solutions. Journal of Compliance in Health Care 1986,1(1):5–20.Google Scholar
- Morisky DE, Ang A, Krousel-Wood M, Ward HJ: Predictive Validity of a Medication Adherence Measure in an Outpatient Setting. The Journal of Clinical Hypertension 2008,10(5):348–354. 10.1111/j.1751-7176.2008.07572.xPubMed CentralPubMedView ArticleGoogle Scholar
- Koneru S, Shishov M, Ware A, Farhey Y, Mongey AB, Graham TB, Passo MH, Houk JL, Higgins GC, Brunner HI: Effectively measuring adherence to medications for systemic lupus erythematosus in a clinical setting. Arthritis Rheum 2007,57(6):1000–1006. 10.1002/art.22898PubMedView ArticleGoogle Scholar
- Goetghebeur E, Vansteelandt S: Structural mean models for compliance analysis in randomized clinical trials and the impact of errors on measures of exposure. Statistical Methods in Medical Research 2005,14(4):397–415. 10.1191/0962280205sm407oaPubMedView ArticleGoogle Scholar
- Graham D: The problem of measurement error in modelling the effect of compliance in a randomized trial. Statistics in Medicine 1999,18(21):2863–2877. 10.1002/(SICI)1097-0258(19991115)18:21<2863::AID-SIM238>3.0.CO;2-IView ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.