Skip to main content

Electronic reporting of patient-reported outcomes in a fragile and comorbid population during cancer therapy – a feasibility study



Electronic collection of patient-reported outcomes (ePROs) is becoming widespread in health care, but the implementation into routine cancer care during therapy remains to be seen. Especially, little is known of the use and success of electronic reporting during active cancer treatment in fragile and comorbid patients. The aim of this study was to test the feasibility of ePRO and its incorporation into routine cancer care, measured by physician compliance, for a fragile and comorbid bladder cancer (BC) population receiving chemo- or immunotherapy.


All BC patients initiating treatment for locally advanced or metastatic bladder cancer at Rigshospitalet or Herlev Hospital, Denmark, were approached during an 8 month period. Exclusion criteria were patients not speaking Danish or not being signed up for electronic communication with health authorities. Enrolled patients were prompted to complete weekly ePROs from home. Patients completed the European Organisation for Research and Treatment of Cancer’s general quality of life questionnaire, QLQ-C30, and the module for muscle-invasive bladder cancer QLQ-BLM30, the Hospital Anxiety and Depression Scale, HADS, and selected items from the Patient Reported-Outcomes version of the Common Terminology Criteria of Adverse Events (PRO-CTCAE), in total 158 questions weekly. If failing to report when prompted, patients were sent two e-mail reminders. Patients were informed that the physician would have an overview of the reported ePROs at their following clinical visits. Physicians were at all clinical visits informed to look at the ePROs in a software solution separate from the medical records. Physicians were logged to check their compliance to the task. No continuous surveillance of ePROs was established.


Of 91 patients screened for enrolment, 19 patients (21%) were not found eligible for standard treatment, eight patients (9%) were not signed up for electronic communication with the health authorities and nine patients (10%) declined participation. Another six patients did not meet other inclusion criteria. In total 49 BC patients were enrolled, 29 initiating chemotherapy and 20 initiating immunotherapy. A total of 466 electronic questionnaires were completed. The overall adherence of the patients to complete ePROs was at an expected level for an elderly cancer population (75%) and remained above 70% until the 6th cycle of treatment. The physician’ compliance was in contrast low (0–52%) throughout the course of treatment.


Electronic reporting of PROs is feasible in a fragile and comorbid population of patients during routine active cancer treatment. Despite clear implementation strategies the physician compliance remained low throughout the study proving the need for further implementation strategies.


In previous studies we have seen that bladder cancer patients have very poor clinical outcomes with 66% of the patients experiencing hospitalisation during oncological treatment and only 45–50% of patients completing the planned treatment, whether as neoadjuvant treatment or treatment for metastatic disease [1,2,3]. One of the explanations for this could be that many of these patients deal with comorbidities troubling completion of treatment [4]. Better handling of troublesome symptoms during treatment therefore seems reasonable. The active use of patient-reported outcomes (PROs) for timely monitoring and handling of symptoms during cancer treatment has proven advantageous for cancer patients illustrated by improvements in quality of life, satisfaction with care and survival [5,6,7,8,9]. The improvement in survival was hypothesized to be due to better adherence to treatment [7]. Bladder cancer patients may therefore benefit from the use of electronic PROs (ePROs) as an intervention to improve adherence to treatment and hence their outcomes. However, little is known of the adherence to ePROs by specifically frail and comorbid patients as previous studies focus on a broad population of cancer patients across diseases [5, 6, 10].

To obtain improvements in clinical outcomes as a result of active use of PROs requires staff dedicated to management and action on the reported symptoms, whether reported on paper or as ePROs [11,12,13]. The technological developments urge the health care systems to start implementation of ePROs [14,15,16] and many studies report on the feasibility and implementation of ePROs in different populations [10, 17,18,19]. However, as pointed out in previous studies several challenges exist with ePRO implementation in many of which the clinicians play a vital role in the success of implementation [11, 20, 21]. One of the key challenges found by Nordan et al. was the acceptance of a new workflow. The handling of ePROs was, instead of a route to better and more efficient patient care, perceived as an additional daily task not part of the available clinical guidelines [11, 22]. To study this reluctance from clinicians, qualitative research studying physicians’ perceptions of ePRO has been carried out [17, 23, 24]. However, these studies do not report quantitively on the clinician use of ePROs to estimate the impact of a given implementation. Before extensive costs across health care systems are used to implement a given software for this purpose, analysis on the feasibility and real-life use of such in different populations is warranted.

Working towards the possible use of PRO in clinical practice, the purpose of this study was to test the feasibility of ePRO from a patient and clinician perspective. This study therefore reports on the adherence to ePROs in a fragile and comorbid population of bladder cancer patients in which ePROs may have a substantial potential to improve clinical outcomes. This study will also present quantitative data on physician compliance with ePRO completion and review.


From 1 February 2018 all patients with bladder cancer initiating standard treatment with chemotherapy or immunotherapy outside clinical trials at the Departments of Oncology at University Hospitals of Copenhagen, Rigshospitalet and Herlev Hospital, Denmark, were approached. Patients fulfilling the following inclusion criteria were asked to participate:

  • Initiating standard treatment with chemotherapy (combination cisplatin-gemcitabine, carboplatin-gemcitabine or single agent vinflunine) or immunotherapy (pembrolizumab).

  • Able to read Danish.

  • No serious cognitive impairment as evaluated by the treating team of physician and/or nurse.

  • Access to electronic communication with health authorities (e-Boks – an electronic, closed two-code guarded system, see below) [25].

Inclusion was planned until a minimum of 40 patients (minimum 20 patients initiating chemotherapy and minimum 20 patients initiating immunotherapy), were enrolled. The sample size was planned according to recommendations for pilot studies [26, 27].

The electronic system for patient communication with authorities and companies, e-Boks, is a national system based on the civil registration number unique to every Danish citizen. More than 95% of the Danish population are active in the e-Boks communication system that ensures delivery of official communication from Danish authorities and companies regardless geographical home address. Exemption from electronic communication with the authorities can be granted due to physical or mental incapacity disabling electronic communication [28]. Access to documents can only be achieved by a two-factor log-in system with two different codes unique to the citizen. Ambuflex, a generic web-based PRO system, was used as the ePRO software system and generated e-mails to the e-Boks system with links to the questionnaires [29]. Originally developed in 2004 by author NHH the Ambuflex software platform has since been implemented for out-patient follow-up for 35 diagnosis groups across 10 hospitals in Denmark [20, 30]. If enrolled patients did not have access to a computer, tablet or smartphone at home, a tablet with free data access was provided from the department during the study period.

All patients completed the following questionnaires weekly electronically through e-Boks: European Organisation for Research and Treatment of Cancer (EORTC) core quality of life questionnaire: QLQ-C30 [31], EORTC module for muscle-invasive bladder cancer: QLQ-BLM30 [32], the Hospital Anxiety and Depression Scale (HADS) [33] and selected Patient-Reported Outcome version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) questions [34]; a total of 158 questions by each patient throughout the course of treatment. All questionnaires had a recall period of 1 week. The QLQ-C30 consists of 30 questions for assessing quality of life in cancer patients and comprises nine multi-item scales; five functional scales, three symptom scales and a global health scale [31]. It has been validated and translated into more than 100 languages, including Danish [35]. The bladder cancer specific module for muscle-invasive disease, QLQ-BLM30, can be used as a supplement to the QLQ-C30. It consists of 30 questions, divided into items concerning e.g. urinary, bowel and sexual functions [36]. The HADS is a brief but commonly used scale [33, 37]. The scale consists of 14 questions, divided into two subscales for anxiety and depression, all items rated on a 4-point scale. The PRO-CTCAE library was developed by the National Cancer Institute, US, in 2014 [34]. It consists of 124 items representing 78 symptomatic toxicities from which the relevant items are chosen depending on the study population. The PRO-CTCAE was translated into Danish in 2016 [38]. Responses are provided on a 5-point Likert scale (0–4). The selected 45 PRO-CTCAE symptoms explored by 84 PRO-CTCAE single items were chosen as part of an item selection process described in detail in a separate paper [39]. As all participating patients had existing access to e-Boks, no specific training in opening the questionnaire was found necessary. Reminders to complete questionnaires were sent one and 2 days after the weekly questionnaire was sent. The first questionnaire was sent on the day of initiating treatment. A screen shot of the login procedure and questionnaire friendliness is provided in Fig. 1. The estimated time for completion by the patients was 15–30 min, depending on pre-existing computer skills.

Fig. 1
figure 1

Patient view of log-on procedure and questionnaire. Shows the screen view for patients when logging into the system (a) with the study logo (iBLAD) in the top right corner and when completing questionnaires (b). The current and total number of screens is listed at the bottom of the questionnaire, so the patients are aware of their progress. Approximately 10 screens had to be completed in order to complete the full questionnaire, estimated duration 15–30 min depending on pre-existing computer skills. The Ambuflex telephone support number is listed at the bottom in case of difficulty logging in or when completing the questionnaire

Before study initiation, an implementation process for the clinical staff was undertaken. The process consisted of several initiatives: 1) Collective teaching for physicians and nurses about the purpose of the ePROs for this patient group and how to access the electronic module for viewing of the patients’ ePROs. These instructions were repeated twice within 2 months prior to enrolment start. 2) Instructions on how to access the ePRO system were also reviewed one-to-one for all physicians with out-patient duties in both uro-oncological out-patient clinics and 3) instruction cards were present and visible in all consultation rooms. Finally, 4) every patient visit in the ambulatory programme in the electronic health record system was marked with the study name (iBLAD) with a reminder to check the patient’s symptoms in the Ambuflex software. 5) The study investigator (GAT) furthermore provided everyday telephone hot-line service assistance, if needed.

For the physicians, access to the completed ePROs was through a browser-based existing clinical system that until recently was used for chemotherapy prescription, therefore well-known to all regular personnel. The estimated added workflow in opening and viewing the ePROs through this system was estimated to 2–5 min per patient per visit.

Patient adherence was measured as number of completed PRO-questionnaires divided by the number of PRO-questionnaires sent out during treatment. Physician ePRO views were logged and physician compliance was measured by number of ePROs opened in relation to a clinical visit for each patient; e.g. if 20 out of 30 patients had their ePROs viewed in relation to a given out-patient visit, the compliance would be 67%. The study took place at two university hospitals and patients were therefore seen by specialists in uro-oncology or fellow uro-oncologists in training at each clinical visit, every third week, in concordance with the length of all treatment cycles for standard treatment with chemo- or immunotherapy.

No surveillance or alert system was provided for the ePRO system, thus no alerts to patients nor clinic were provided while patients reported from home. The ePRO system therefore represented a passive collection of data from the patient with the possibility to incorporate these data into the out-patient consultation between physician and patient e.g. to improve physician-patient communication about symptoms during treatment or as an overview for the physician, Fig. 2. Eleven symptoms were chosen as the ‘front page’ in the ePRO system by two participating physicians, Fig. 2. All other responses could be seen by one additional click on the screen. All symptoms were marked with either green, yellow or red, depending on severity of the symptom, thereby enabling quick overview of troublesome symptoms.

Fig. 2
figure 2

Physician view of completed PROs in the Ambuflex system. Figure 1 shows the development of symptoms (left column) over time with dates at the top. The coloured bars (green, yellow, red) indicate severity of the reported PRO-CTCAE symptom or QLQ-C30 item. In this viewing only selected symptoms chosen by the study team (physicians) are shown. The selected QLQ-C30 or PRO-CTCAE items (F: frequency, S: severity, I: interference, P: presence) are in order of appearance: Fatigue (S), fatigue (I), global health (QLQ-C30, q29), global quality of life (QLQ-C30, q30), abdominal pain (F), abdominal pain (I), decreased appetite (S), decreased appetite (I), nausea (F), nausea (S), constipation (S), urinary urge (F), urinary urge (I), heart palpitations (F), heart palpitations (S), ringing in ears (S) and rash (P). By pressing ‘Vis besvarelse’(English: show completed questionnaire) the physician can access all responses to the 158 single items

The study was approved by The National Data Protection Agency (file no.: 2012-58-0004). All participants completed written informed consent in accordance with Danish ethical legislation. Patients could according to the Helsinki Declaration always decline further ePRO completions without providing a reason. Patients ceased ePRO completion after 6 cycles of treatment (18 weeks) or when terminating treatment, for whatever reason.


From February 2018 through September 2018, 91 patients referred to the Department of Oncology at Rigshospitalet or Herlev Hospital were screened for study enrolment 19 patients (21%) did not meet the prespecified inclusion criteria of initiating standard treatment either because of poor performance status preventing treatment initiation or due to inclusion into a clinical trial, eight patients (9%) did not have access to electronic communication, three patients (3%) declined treatment, three patients (3%) were missed at treatment initiation and nine patients (10%) declined participation, the majority due to reluctancy with information technology (IT) or not wanting to spend time completing questionnaires. The inclusion rate for patients initiating chemotherapy was higher than for immunotherapy and therefore study inclusion continued until 20 patients initiating immunotherapy were enrolled, thereby enrolling 29 patients initiating chemotherapy and 20 patients initiating immunotherapy.

Demographic data for participants is shown in Table 1. The distribution between patients starting neoadjuvant treatment vs. treatment for metastatic disease was 27 and 73%, respectively. Fifty-five percent of patients did not complete the planned treatment. The main reasons for early treatment cessation were disease progression (48%), death (11%), nephrological toxicity (7%), haematological toxicity (7%) or infection (7%).

Table 1 Patient characteristics

A total of 466/624 (75%) electronic questionnaires were completed by the patients. Baseline patient ePRO completion was achieved for 94%. The patient adherence remained above 70% throughout the 6 cycles of chemo- or immunotherapy only to drop after the sixth cycle of treatment. The completeness of data across all questionnaire items was 97% and in 76% of all questionnaires all 158 items were completed. Two patients withdrew consent to complete questionnaires during the study due to IT-problems with at-home completion but did not withdraw consent to participation in the study.

A total of 163 physician view logs were registered throughout the study. Multiple logs from the same physician on the same day for the same patient and logs by author GAT as principal investigator were filtered out leaving 40 physician view logs from a total of 24 different physicians participating at any given time during the course of the study. None of the other authors of this paper had log views to filter out and none of the authors had access to log-data while the study was being conducted.

The 49 patients completed a total of 208 cycles of treatment and thus had a total of 208 visits at the hospital at which the physician could view the patients’ responses. However, as patients adherence was not 100% only 128 clinical visits had a completed questionnaire in conjunction with the visit, see total number of completed questionnaires in Table 2. Each log was checked against the electronic health record (EHR). If the log view was in relation to a contact with the given patient the log counted as viewed in the final count. The amount of physician log views dispersed by outpatient visits are displayed in Table 2 and Fig. 3. The physician log showed a compliance with the ePRO system of 35% of questionnaires viewed at first outpatient consultation after initiating treatment. Physician compliance remained considerably lower (0–52%) than that of the patients throughout the course of the patients’ treatment.

Table 2 Patient adherence and clinician view of ePRO completion
Fig. 3
figure 3

Patient adherence and physician compliance with ePRO completion and views. The curve for patients indicates the percentage of questionnaires completed for patients still receiving treatment. The curve for physicians indicates the percentage of physicians viewing a completed questionnaire in relation to an out-patient visit


This study gives real-life insight into the implementation of ePROs in daily cancer care. Our data showed high patient adherence (70–94%) with ePRO completion throughout the 6 cycles of oncological therapy in an elderly and comorbid patient population while physician compliance was considerably lower (0–52%) despite a clear strategy for implementation. The results of this study are important to better understand the complexity of ePRO implementation and create strategies for further application.

The patient adherence with ePRO completion found in this study is similar to the findings in previous comparable studies examining ePRO adherence [10, 19, 40]. The drop seen in patient adherence after 6 cycles of chemo- or immunotherapy treatment may likely be due to the passive nature of the ePRO collection not creating alerts to neither patient nor clinic but simply gathering of data from the patient. Also, lack of feedback from the ePRO completion at clinical visits through the low clinician compliance seen in this study is known to make patient engagement drop [23].

Possible explanations for the low clinician compliance may be many. First, although the physicians were reminded before every enrolled patient and proposed to do so by the team management, the viewing of ePROs by physicians was not deemed mandatory by department management. As pointed out by Nordan et al. this may have affected physician adherence [11]. Second, the ePRO reporting started shortly after implementation of a new EHR system, into which the ePROs were not automatically transferred thus likely to be reflected in adherence by physicians [24]. Clinicians were likely reluctant to engage in additional electronic systems for care of patients due to the busy workflow thus resulting in poor clinician adherence to the study. Also academically trained personnel, e.g. physicians, may be opposed to implementing PROs based on the still sparse literature of the benefits on QoL and survival, two central elements in clinical oncology. Finally, the study took place at two large university hospitals in which specialist training and multiple clinical trials take place. Many oncologist residents and fellows not permanently allocated to the hospital or department therefore take part in the daily out-patient clinics perhaps disturbing continuity of permanent staff or not being fully introduced to the purpose of this study. This could have affected clinician adherence to view of the ePROs [11, 24].

This study meets the recommendations for successful implementation as depicted by Snyder et al. in the ISOQOL User’s Guide for Implementing Patient-Reported Outcomes Assessment in Clinical Practice [13] and the majority of the criteria related to successful implementation by Nordan et al. but may have come short on two issues despite the multifaceted strategy for implementation: 1) engaging providers and staff in a new workflow and 2) overcoming the EHR data management issues when the ePROs are not merged into the EHR. The turning point for Nordan et al. had been engaging a “physician champion” to engage colleagues by demonstrating the flexibility of the tool and highlighting efficiencies in workflows [11]. While this may be an important factor to build future learnings upon, one may also hypothesize that more quantitative research informing the physicians of e.g. time savings or similar efficiency pathways in the care of patients when using ePROs could be an important factor. Especially as the time saved not having to go through checklists of symptoms in the physician’s office may be spent worthwhile with the patients thus improving satisfaction with the quality of care delivered [22]. Gaining a “physician champion” does not however compensate the need for management endorsement of ePROs as part of daily clinic. This issue may be overcome through implementation of ePRO into the clinical guidelines thus endorsing the clinical advantage with the use of ePROs [24].

Finally, despite intentions to test the feasibility of ePROs in a frail and comorbid population of patients with high rates of hospitalisations and treatment cessations this study is limited by the inclusion criteria necessitating existing access to electronic communication with authorities through e-Boks. This inclusion criteria ruled out 9% of possible study participants, most likely the 9% most frail patients as exception from e-Boks communication is only allowed in certain circumstances (substantial morbidity or disability) after approval from the authorities [28]. Future work on how to include these frail patients into similar studies is needed. Also, this study may have had better success in terms of high physician compliance through a more in-depth training of personnel or by restricting clinical consultations with the task of ePRO viewing in the current study to only a few physicians. However, these changes would require more allocated time away from clinic for a more thorough onboarding task and also a re-design of clinical duties for attending staff in the university hospitals which was not the objective of this study.

This fragile and comorbid population is in need of interventions to better their outcomes [1]. The results of this study have contributed to the planning of a national randomized trial in the bladder cancer population in which ePROs act as the intervention for timely symptom management (NCT03584659). The randomized study has incorporated the learnings of the current study e.g. by increasing patient involvement through alerts in real-time for patients at home instead of alerts to clinic thus minimizing the added ‘burden’ for physicians. The results will be reported following completion of enrolment, in the autumn 2020.


This study gives real-life insight into the implementation of ePROs in cancer care and highlights that physicians may need more quantitative documentation of the positive benefits of using ePROs in order to endorse the use of ePROs in daily clinic. Thus, this study urges for continuous feasibility testing before software implementation, at the very least in the respect of both patients and providers.

Availability of data and materials

The data that support the findings of this study are available from corresponding author Gry Assam Taarnhøj but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available.



Electronic patient-reported outcomes


European Organisation for Research and Treatment of Cancer


EORTC’s general core quality of life questionnaire


EORTC’s module for muscle-invasive bladder cancer


Hospital Anxiety and Depression Scale


Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events


Patient-reported outcome(s)


Electronic health record


International Society of Quality of Life


  1. Taarnhoej GA, Lindberg H, Johansen C, Pappot H. Patient-reported outcomes and quality of life in bladder cancer patients receiving chemo- or immunotherapy; a real-life experience. Under review in Quality of Life Research Nov 2019. 2019.

  2. Holmsten K, Jensen NV, Mouritsen LS, Jonsson E, Mellnert C, Agerbaek M, et al. Vinflunine/gemcitabine versus carboplatin/gemcitabine as first-line treatment in cisplatin-ineligible patients with advanced urothelial carcinoma: A randomised phase II trial (VINGEM). Eur J Cancer. 2020;127:173–82. Epub 2019 Oct 22.

  3. Laurent M, Brureau L, Demery ME, Flechon A, Thuaut AL, Carvahlo-Verlinde M, et al. Early chemotherapy discontinuation and mortality in older patients with metastatic bladder cancer: The AGEVIM multicenter cohort study. Urol Oncol. 2017;35(1):34 e9–.e16.

  4. Guancial EA, Roussel B, Bergsma DP, Bylund KC, Sahasrabudhe D, Messing E, et al. Bladder cancer in the elderly patient: challenges and solutions. Clin Interv Aging. 2015;10:939–49 PMC4467651.

    CAS  PubMed  PubMed Central  Google Scholar 

  5. Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol. 2004;22(4):714–24.

    Article  PubMed  Google Scholar 

  6. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom monitoring with patient-reported outcomes during routine Cancer treatment: a randomized controlled trial. J Clin Oncol. 2016;34(6):557–65.

    Article  CAS  PubMed  Google Scholar 

  7. Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, et al. Overall Survival Results of a Trial Assessing Patient-Reported Outcomes for Symptom Monitoring During Routine Cancer Treatment. JAMA. 2017;318(2):197–8.

  8. Denis F, Yossi S, Septans AL, Charron A, Voog E, Dupuis O, et al. Improving Survival in Patients Treated for a Lung Cancer Using Self-Evaluated Symptoms Reported Through a Web Application. Am J Clin Oncol. 2017;40(5):464–9.

  9. Berry DL, Hong F, Halpenny B, Partridge AH, Fann JR, Wolpin S, et al. Electronic self-report assessment for cancer and self-care support: results of a multicenter randomized trial. J Clin Oncol. 2014;32(3):199–205.

    Article  PubMed  Google Scholar 

  10. Judson TJ, Bennett AV, Rogak LJ, Sit L, Barz A, Kris MG, et al. Feasibility of long-term patient self-reporting of toxicities from home via the Internet during routine chemotherapy. J Clin Oncol. 2013;31(20):2580–5 PMC3699724.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Nordan L, Blanchfield L, Niazi S, Sattar J, Coakes CE, Uitti R, et al. Implementing electronic patient-reported outcomes measurements: challenges and success factors. BMJ Qual Saf. 2018;27(10):852–6.

    Article  PubMed  Google Scholar 

  12. Engelen V, Haverman L, Koopman H, Schouten-van Meeteren N. Meijer-van den Bergh E, Vrijmoet-Wiersma J, et al. development and implementation of a patient reported outcome intervention (QLIC-ON PROfile) in clinical paediatric oncology practice. Patient Educ Couns. 2010;81(2):235–44.

    Article  PubMed  Google Scholar 

  13. Snyder CF, Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MY, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 2012;21(8):1305–14.

    Article  PubMed  Google Scholar 

  14. Wintner LM, Giesinger JM, Zabernigg A, Rumpold G, Sztankay M, Oberguggenberger AS, et al. Evaluation of electronic patient-reported outcome assessment with cancer patients in the hospital and at home. BMC Med Inform Decis Making. 2015;15:110 PMC4690412.

    Article  CAS  Google Scholar 

  15. Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health. 2009;12(4):419–29.

    Article  PubMed  Google Scholar 

  16. Jensen RE, Rothrock NE, DeWitt EM, Spiegel B, Tucker CA, Crane HM, et al. The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Med Care. 2015;53(2):153–9 PMC4801509.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Howell D, Molloy S, Wilkinson K, Green E, Orchard K, Wang K, et al. Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol. 2015;26(9):1846–58.

    Article  CAS  PubMed  Google Scholar 

  18. Baeksted C, Pappot H, Nissen A, Hjollund NH, Mitchell SA, Basch E, et al. Feasibility and acceptability of electronic symptom surveillance with clinician feedback using the Patient-Reported Outcomes version of Common Terminology Criteria for Adverse Events (PRO-CTCAE) in Danish prostate cancer patients. J Patient Reported Outcomes. 2017;1(1):1 PMC5934932.

    Article  Google Scholar 

  19. Basch E, Artz D, Iasonos A, Speakman J, Shannon K, Lin K, et al. Evaluation of an online platform for cancer patient self-reporting of chemotherapy toxicities. J Am Med Inform Assoc. 2007;14(3):264–8 PMC2244885.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Mejdahl CT, Schougaard LMV, Hjollund NH, Riiskjaer E, Lomborg K. Exploring organisational mechanisms in PRO-based follow-up in routine outpatient care - an interpretive description of the clinician perspective. BMC Health Serv Res. 2018;18(1):546 PMC6044066.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Santana MJ, Haverman L, Absolom K, Takeuchi E, Feeny D, Grootenhuis M, et al. Training clinicians in how to use patient-reported outcome measures in routine clinical practice. Qual Life Res. 2015;24(7):1707–18.

    Article  PubMed  Google Scholar 

  22. Rotenstein LS, Huckman RS, Wagle NW. Making patients and doctors happier - the potential of patient-reported outcomes. N Engl J Med. 2017;377(14):1309–12.

    Article  PubMed  Google Scholar 

  23. Velikova G, Awad N, Coles-Gale R, Wright EP, Brown JM, Selby PJ. The clinical value of quality of life assessment in oncology practice-a qualitative study of patient and physician views. Psychooncology. 2008;17(7):690–8.

    Article  PubMed  Google Scholar 

  24. Calvert M, Kyte D, Price G, Valderas JM, Hjollund NH. Maximising the impact of patient reported outcome assessment for patients and society. BMJ (Clinical research ed). 2019;364:k5267.

    Article  Google Scholar 

  25. E-boks. Available from:

  26. Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.

    Article  PubMed  Google Scholar 

  27. Viechtbauer W, Smits L, Kotz D, Bude L, Spigt M, Serroyen J, et al. A simple formula for the calculation of sample size in pilot studies. J Clin Epidemiol. 2015;68(11):1375–9.

    Article  PubMed  Google Scholar 

  28. e-Boks. Electronic communication with Danish authorities. Available from:

  29. Schougaard LM, Larsen LP, Jessen A, Sidenius P, Dorflinger L, de Thurah A, et al. AmbuFlex: tele-patient-reported outcomes (telePRO) as the basis for follow-up in chronic and malignant diseases. Qual Life Res. 2016;25(3):525–34 PMC4759231.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Hjollund NHI. Fifteen Years' use of patient-reported outcome measures at the group and patient levels: trend analysis. J Med Internet Res. 2019;21(9):e15856.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85(5):365–76.

    Article  CAS  PubMed  Google Scholar 

  32. European Organisation of Research in the Treatment of Cancer Module for muscle-invasive bladder cancer. Available from:

  33. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361–70.

    Article  CAS  PubMed  Google Scholar 

  34. Basch E, Reeve BB, Mitchell SA, Clauser SB, Minasian LM, Dueck AC, et al. Development of the National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst. 2014;106(9):dju244. Print 2014 Sep.

  35. Gutierrez-Ibarluzea I, Simpson S, Benguria-Arrate G. Early awareness and alert systems: an overview of EuroScan methods. Int J Technol Assess Health Care. 2012;28(3):301–7.

    Article  PubMed  Google Scholar 

  36. Porter MP, Penson DF. Health related quality of life after radical cystectomy and urinary diversion for bladder cancer: a systematic review and critical analysis of the literature. J Urol. 2005;173(4):1318–22.

    Article  PubMed  Google Scholar 

  37. Cosco TD, Doyle F, Ward M, McGee H. Latent structure of the hospital anxiety and depression scale: a 10-year systematic review. J Psychosom Res. 2012;72(3):180–4.

    Article  PubMed  Google Scholar 

  38. Baeksted C, Nissen A, Pappot H, Bidstrup PE, Mitchell SA, Basch E, et al. Danish translation and linguistic validation of the U.S. National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Pain Symptom Manag. 2016;52(2):292–7.

    Article  Google Scholar 

  39. Taarnhoej GA, Lindberg H, Johansen C, Pappot H. Patient-reported outcomes item selection for bladder cancer patients receiving chemo- or immunotherapy. J Patient Rep Outcomes. 2019;3(1):56.

  40. Benze G, Nauck F, Alt-Epping B, Gianni G, Bauknecht T, Ettl J, et al. PROutine: a feasibility study assessing surveillance of electronic patient reported outcomes and adherence via smartphone app in advanced cancer. Ann Palliat Med. 2019;8(2):104–11.

    Article  PubMed  Google Scholar 

Download references


A special thanks to the patients enrolled in this study for weekly completion of lengthy and time-consuming questionnaires. Also, we would like to express our gratitude to the secretaries and nursing staff of the uro-oncological in- and outpatient ward of Department of Oncology at Rigshospitalet and Herlev Hospital for daily being aware of new patients eligible for enrollment throughout the study period.


The study was funded by the Danish Cancer Society [grant number R150-A10114], Einar Willumsens Mindelegat, Dagmar Marshalls Fond and Christian Larsen og Dommer Ellen Larsens Legat. The funding parties played no role in any part of the design, conduction, analysis or publication plan for this study.

Author information

Authors and Affiliations



Conception and design: GAT, HL, NHH, CJ, HP. Data collection: GAT, HL, LHO, HP. Data analysis: GAT, NHH, HP. Manuscript drafting: GAT, HL, LHO, NHH, CJ, HP. All authors approved the final version of this manuscript.

Corresponding author

Correspondence to Gry Assam Taarnhøj.

Ethics declarations

Ethics approval and consent to participate

The study was approved by The National Data Protection Agency (file no.: 2012-58-0004). All participants completed written informed consent in accordance with Danish ethical legislation. In Denmark, no ethical approval is needed for scientific studies assembling questionnaire data only.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no disclosures or any competing interests (financial or non-financial) in relation to this study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Taarnhøj, G.A., Lindberg, H., Dohn, L.H. et al. Electronic reporting of patient-reported outcomes in a fragile and comorbid population during cancer therapy – a feasibility study. Health Qual Life Outcomes 18, 225 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: