Strategies to use tablet computers for collection of electronic patient-reported outcomes
Health and Quality of Life Outcomes volume 13, Article number: 2 (2015)
Mobile devices are increasingly being used for data collection in research. However, many researchers do not have experience in collecting data electronically. Hence, the purpose of this short report was to identify issues that emerged in a study that incorporated electronic capture of patient-reported outcomes in clinical settings, and strategies used to address the issues.
The issues pertaining to electronic patient-reported outcome data collection were captured qualitatively during a study on use of electronic patient-reported outcomes in two home dialysis units. Fifty-six patients completed three surveys on tablet computers, including the Kidney Disease Quality of Life-36, the Edmonton Symptom Assessment Scale, and a satisfaction measure. Issues that arose throughout the research process were recorded during ethics reviews, implementation process, and data collection. Four core issues emerged including logistics of technology, security, institutional and financial support, and electronic design.
Although use of mobile devices for data collection has many benefits, it also poses new challenges for researchers. Advance consideration of possible issues that emerge in the process, and strategies that can help address these issues, may prevent disruption and enhance validity of findings.
Electronic capture of patient-reported outcomes (ePROs) data [1-4], offers new opportunities to both researchers and clinicians. The benefits and challenges of ePROs previously addressed in the literature [1-16] are summarized in the below list. Although guidelines for reporting patient-reported outcomes have been established [13,17-19], utilization of ePROs is a more recent development. Over the past decade, researchers have described how they worked with electronic data collection systems, platforms or registries of ePROs [3,4,6,11,12,15,16,20]. However, pragmatic issues and potential strategies that a researcher might use when conducting a study to collect ePRO data are not often discussed, and this paper addresses this gap in the literature.
Benefits and challenges of administering ePROs
Resistance in a healthcare culture to electronic administration 
Integrating ePRO data into clinic workflow and interpretation of clinical relevance 
Purpose and research question
The purpose of this research was to identify issues that emerged in a study that incorporated ePROs in clinical settings and to identify possible solutions/strategies to address the issues.
The overall focus of the pilot project was the feasibility of using tablet computers to collect ePROs in two outpatient dialysis clinics . People attending a clinic, in one of two cities in Western Canada, were invited to participate if they were over 19 years old, did not have mild-severe cognitive challenges, could speak and read English, and were medically stable. Fifty-six dialysis patients volunteered. They were primarily older adults with a mean age of 66 ± 12 (36–91 years old), and only two had used a tablet computer previously. Ethics approval was attained from a University Human Research Ethics Board and a health authority. While in the clinic waiting room, participants completed two ePROs (Kidney Disease Quality of Life-36 ((KDQOL-36)) ), and Edmonton Symptom Assessment Scale ((ESAS)) ) and a patient satisfaction measure. The tablet computer used in this study was the iPad™ (iPad is a trademark of Apple, Inc). The ESAS was designed as a new application (app), and the KDQOL-36 was completed through KDQOL Complete  providing online scores in real time. The ESAS app had minor modifications, and measurement equivalence was addressed through cognitive debriefing and usability testing [9,14].
Interpretive description, an established qualitative methodology used in applied clinical fields, is an inductive approach that offers ways to consider themes/patterns from subjective descriptions, and create descriptions that inform clinical understanding . It was used to identify the core ePRO issues that arose and strategies that were used during the design and conduct of the project. The researchers qualitatively recorded issues that arose during ethics reviews and data collection. Researchers used field notes and emails to each other to formally document the issues. Notes were updated when issues were resolved and new matters arose [24,25]. Analysis was undertaken at a conceptual/thematic descriptive level, identifying thematic patterns developed in situ from the data . If another researcher wanted to examine and verify our decision-making and interpretation of findings in the interests of rigor, audit trails (documentation and reflexive writing) were created throughout the analytic process.
The themes that emerged relating to issues and strategies to address ePRO implementation included: logistics of technology, security, support, and design. Table 1 provides excerpts from field notes to support these themes.
The logistics for use of the technology in the clinical environment required considerable time and attention. In our study, the health authority did not allow researchers to access the healthcare institution’s intranet, and because one survey was conducted online, tablets needed to have 3G (cellular) capacity. Given that health settings often have variable cellular receptivity, the tablets needed to be tested in various clinical locations. In this project, it was not possible to synchronize ePRO and electronic health data because the clinics were using a hybrid of paper and electronic records. Logistical issues arose due to the need for wireless printing. Because clinicians requested that ePRO results be printed for their use in real time, printing options required advance onsite testing.
Health administrators, practitioners, and patients all asked about how infection control was being managed with the tablet. We used two approaches. First, participants were asked to use a hand sanitizer. Second, everyone used a stylus to prevent touching the screen. Older adults who had not used a touch-screen tablet previously found the stylus very helpful.
Technology security was paramount, and it came under scrutiny during ethics reviews. While KDQOL-36 and ESAS data was printed for clinicians, raw data was downloaded daily to the university data repository so that no surveys were stored on the tablet. As a result, a file transfer protocol was required.
KDQOL-36 data was stored on the KDQOL Complete database, housed on a US server. Under the US Patriot Act, the US Government could access this data without knowledge/consent by researchers or participants. To uphold confidentiality, ID codes were used to identify participants and this was explained in the informed consent forms.
To ensure tablets were securely stored, they were kept in a locked cabinet in the researcher’s office when not being used. When being used, they were either with the researcher or kept in a locked storage container. To meet university/health authority ethical requirements, tablet computers needed 8-digit passwords and the highest levels of encryption.
Institutional and financial support was essential. Ethical review for the pilot project took approximately one year, primarily due to the novelty of the technology in the clinical setting. Approval was needed from the health authority’s Information Technology department to ensure that their security requirements were met and health privacy legislation was followed. Financial support was necessary to assist with purchasing tablets and printers, hiring computer programmers, procuring online survey support, and accessing cellular internet.
Elements of electronic design were issues that emerged in relation to the screen display, creation of new apps, and measurement equivalence. Because participants were primarily older adults without computer experience, screen displays that were easily readable were needed. We kept the app minimalistic with large font, black writing on white background, and no distracting graphics. Time stamps were also used to log data input as an audit trail. Given that both the ESAS and KDQOL-36 were originally designed for paper, there was a need for only minor modification of the instruments for the tablet .
This study offers strategies that researchers/practitioners might consider useful when employing ePROs, a topic rarely discussed in the literature. The findings are presented in Table 1 along with a checklist of questions that researchers may ask themselves when using ePROs in clinical settings.
Given that every clinical application of ePROs will be unique, different issues may emerge in diverse contexts, and testing in real-world conditions is imperative . For example, technology logistics in clinical environments may dictate the capacity to which ePRO data may be linked to electronic health records. In light of the current expansion of organizations incorporating electronic patient records, combined with the desire for immediate retrieval of electronic data, the potential to integrate ePROs with electronic records will only continue to expand [1,6,12,13]. Strategies for such integration will be paramount as researchers strive for utility of ePROs in clinical practice.
Specific clinical settings may also direct logistics controlling the spread of infection. Strategies other than those used in this study could include sanitizing the tablet screen between each use or using disposable screen covers.
Institutional requirements regarding technology security will vary based on national/local policies and contexts. Specifically, passwords and encryption requirements will differ in various healthcare settings. Documentation of security features is an important consideration, especially during research ethics applications. Only two authors [4,12] mentioned security requirements in the ePRO literature. However, in our project, this was a significant consideration, one that required collaborative strategies with computer programmers/privacy analysts.
Findings from this study regarding institutional and financial support are substantiated in the literature. While some authors [1,6,12] acknowledge that there will be greater upfront costs and training for patients/professionals to use technology, electronic formats are still more economical both in terms of time and resources [3,4,6-8,12,15]. ePROs also offer an environmental, paperless option. Despite international movement towards use of ePROs for the purposes of creating retrievable data [1,6], linking with electronic records [6,12,13], and triggering automated patient alerts [1,3,6,11-13], institutional support was not recognized in the ePRO literature as a critical strategy with the exception of acknowledging support from clinical leaders [4,6]. Research addressing how ePRO data is used as a feedback mechanism to clinicians is an area in need of future development.
Health data privacy requirements will vary based on the legislation, policy, culture or health authority. Establishing early collaborations with local Information Technology departments may assist researchers to overcome challenges with institutional support.
The aspect of electronic design was rarely discussed, with a few exceptions [6,15]. Our findings emphasize the need to tailor design for the intended population. Diverse patient groups may have different technology comfort levels, and design will need to be adapted accordingly. One area of design that has been developed in the literature pertains to modification of measurements for electronic delivery . The level of modification needed for translation from paper to electronic version will determine the evidence needed to test measurement equivalence.
A limitation of this study is that the primary purpose of the overarching project related to the feasibility of ePROs in clinical settings  and not on issues that emerged in the use of the technology; thus, systematic questionnaires were not used to identify such issues. Nevertheless, this paper is a form of knowledge translation offered to researchers/clinicians who will use ePROs in future work. This study was only conducted in one health authority and in one country, limiting the breadth of issues encountered.
Although use of mobile devices, specifically tablet computers, for data collection has many benefits, it also poses new challenges for researchers. Advance consideration of possible issues that emerge in the process, and strategies to address these issues, may prevent disruption and enhance validity of findings.
Electronic patient-reported outcome(s)
Edmonton Symptom Assessment Scale
Kidney Disease Quality of Life
Basch E, Goldfarb S. Electronic patient-reported outcomes for collecting sensitive information from patients. J Support Oncol. 2009;7:98–9.
Dupont A, Wheeler J, Herndon II JE, Coan A, Zafar SY, Hood L, et al. Use of tablet personal computers for sensitive patient-reported information. J Support Oncol. 2009;7:91–7.
Smith SK, Rowe K, Abernethy AP. Use of an electronic patient-reported outcome measurement system to improve distress management in oncology. Palliat Support Care. 2014;12:69–73.
Zbrozek A, Hebert J, Gogates G, Thorell R, Dell C, Molsen E, et al. Validation of electronic systems to collect patient-reported outcome (PRO) data – Recommendations for clinical trial teams: Report of the ISPOR ePRO Systems Validation Good Research Practices Task Force. Value Health. 2013;16:48–489.
Abernethy AP, Ahmad A, Zafar SY, Wheeler JL, Reese JB, Lynerly HK. Electronic patient-reported data capture as a foundation of rapid learning cancer care. Med Care. 2010;48 Suppl 1:S32–8.
Bennett AV, Jensen RE, Basch E. Electronic patient-reported outcome systems in oncology clinical practice. CA-Cancer J Clin. 2012;62:337–47.
Chang Y-J, Chang C-H, Peng C-L, Wu H-C, Lin H-C, Wang J-Y, et al. Measurement equivalence and feasibility of the EORTC QLQ-PR25: paper-and-pencil versus touch-screen administration. Health Qual Life Out. 2014;12:1–10.
Coons SJ. ePRO systems validation: Clearly defining the roles of clinical trial teams and ePRO system providers. Value Health. 2013;16:457–8.
Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based Patient-Reported Outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health. 2009;12:419–29.
Dy SM, Roy J, Ott GE, McHale M, Kennedy C, Kutner JS, et al. Tell Us™: A web-based tool for improving communication among patients, families, and providers in hospice and palliative care through systematic data specification, collection and use. J Pain Symptom. 2011;42:526–34.
Espallargues M, Valderas JM, Alonso J. Provision of feedback on perceived health status to health care professionals: a systematic review of its impact. Med Care. 2000;38:175–86.
Holzner B, Giesinger JM, Pinggera J, Zugal S, Schöpf F, Oberguggenberger AS, et al. The computer-based health evaluation software (CHES): a software for electronic patient-reported outcome monitoring. BMC Med Inform Decis Mak. 2012;12:126–37.
Snyder CF, Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MH, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 2012;21:1305–14.
Schick-Makaroff K, Molzahn A. Use of iPads to assess health-related quality of life: a feasibility study in outpatient home dialysis clinics. CJKHD 2014, 1. doi:10.1186/s40697-014-0022-9
Ashley L, Jones H, Forman D, Newsham A, Brown J, Downing A, et al. Feasibility test of a UK-scalable electronic system for regular collection of patient-reported outcome measures and linkage with clinical cancer registry data: the electronic Patient-reported Outcomes from Cancer Survivors (ePOCS) system. BMC Med Inform Decis Mak. 2011;11:1–10.
Ashley L, Jones H, Thomas J, Newsham A, Downing A, Morris E, et al. Integrating patient reported outcomes with clinical cancer registry data: a feasibility study of the electronic Patient-reported Outcomes from Cancer Survivors (ePOCS) system. J Med Internet Res. 2013;15:e230.
Brundage M, Blazeby J, Revicki D, Bass B, de Vet H, Duffy H, et al. Patient-reported outcomes in randomized clinical trials: development of ISOQOL reporting standards. Qual Life Res. 2013;22:1161–75.
Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD, et al. Reporting of patient-reported outcomes in randomized trials. JAMA. 2013;309:814–22.
Reeve BB, Wyrwich KW, Wu AW, Velikova G, Terwee CB, Snyder CF, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res. 2013;22:1889–905.
Basch E, Iasonos A, Barz A, Culkin A, Kris MG, Artz D, et al. Long-term toxicity monitoring via electronic patient-reported outcomes in patients receiving chemotherapy. JCO. 2007;25:5374–80.
Davison SN, Jhandri GS, Johnson JA. Cross-sectional validity of a modified Edmonton symptom assessment system in dialysis patients: a simple assessment of symptom burden. Kidney Int. 2006;69:1621–5.
Hays RD, Kallich JD, Mapes DL, Coons SJ, Carter WB. Development of the Kidney Disease Quality of Life (KDQOL) instrument. Qual Life Res. 1994;3:329–38.
Kidney Disease Quality of Life (KDQOL) Complete [http://www.kdqol-complete.org/]
Thorne S. Interpretive Description. Walnut Creek, CA: Left Coast Press; 2008.
Sandelowski M, Barroso J. Classifying the findings in qualitative studies. Qual Health Res. 2003;13:905–23.
This study was funded by KRESCENT (Kidney Research Scientist Core Education and National Training Program) and the Faculty of Nursing, University of Alberta in Canada.
Development on this paper was also supported by The University of Maryland Online Dissemination and Implementation Institute funded by the University of Maryland and the John A. Hartford Foundation.
The authors declare that they have no competing interests.
KSM co-designed the study, wrote ethics applications, collected data, systematically recorded issues that arose with e-PRO data collection, strategies to address these issues, and drafted/revised the manuscript. AM substantially contributed to the design of the study and helped write and revise the manuscript for intellectual content. Both authors read and approved the final manuscript.
KSM is a Postdoctoral Fellow in the Faculty of Nursing at the University of Alberta, Canada.
AM is Dean and Professor at the Faculty of Nursing at the University of Alberta, Canada.
About this article
Cite this article
Schick-Makaroff, K., Molzahn, A. Strategies to use tablet computers for collection of electronic patient-reported outcomes. Health Qual Life Outcomes 13, 2 (2015). https://doi.org/10.1186/s12955-014-0205-1
- Electronic patient-reported outcomes
- Tablet computers