The 2008 report “High Quality Health Care for All” by the Health Minister Lord Darzi [1] highlighted the importance of assessing the effectiveness of health care from the patient perspective, and suggested that this should be measured through routine collection of Patient Reported Outcome Measures (PROMs). A national programme to collect PROM data from patients receiving hip or knee replacements, varicose vein surgery or groin hernia surgery was established in April 2009. A subsequent UK Government White Paper published in 2010 confirmed the importance of PROMs within the National Health Service (NHS), and announced plans to extend their use wherever practical to other procedural interventions [2].
Gathering outcome feedback from patients in the form of PROMs has several purposes, not least in assessing whether the intervention provided actually improved the patients health or wellbeing. While a healthcare provider may judge a procedure or treatment to be a success using objective clinical measures, efforts should be made to assess whether this reflects an improvement in the patients’ perspective of their symptoms and quality of life. The aim is to use PROMs to support commissioning within the NHS, based on robust, patient centred evidence to fund those services with proven patient benefit. Patient responses should allow providers to benchmark their performance outcomes.
As the use of PROMs becomes more widespread within certain diseases, research is underway to identify other clinical areas where PROMs collection may be feasible. One area where researchers are currently aiming to develop the use of PROMs is in symptomatic cardiac arrhythmia patients who undergo ablation treatment. Cardiac arrhythmias occur sporadically and cause a range of symptoms including palpitations, shortness of breath, fatigue and dizziness, which can have a significant detrimental effect on the quality of life of sufferers. Cardiac ablation is a widely-used treatment option for symptomatic arrhythmias which are unresponsive to drug therapy, and several guidelines have been published on its use [3-13]. Cardiac ablation aims not only to improve outcome, but also to reduce or stop symptoms [9]; this can be most accurately assessed by the patients themselves, making PROMs an ideal tool for evaluating the clinical benefit of the procedure. PROMs tools are already available for some types of arrhythmias including atrial fibrillation [14] and supraventricular tachycardia [15]. To date, no tool has been validated in a UK population which combines measures of treatment expectation and experiences with symptom impact and severity in patients with any symptomatic arrhythmia. Such a tool allows data to be gathered to facilitate comparative analysis, improve knowledge and inform clinical practice.
This paper describes the methods used to establish preliminary validity of our PROM for a population treated with ablation of cardiac arrhythmias. Using responses gained via face to face patient interviews we aimed to improve the PROM iteratively by ensuring that the tool is fully inclusive of aspects of the disease important to patients and easy to comprehend and complete. These are important steps which will confirm face validity, ensuring it is clear and easy to read and understand. This allowed us to establish preliminary content validity of the tool to ensure it measures all aspects of the disease and treatment most important from a patient perspective.
Study aim
As part of an iterative process to further develop and validate several questionnaires used in a pilot study to form a single PROM tool, a multicentre, prospective observational study was carried out after approval by the Nottingham 1 Research Ethics Proportionate Review Sub-Committee. This paper reports on the first stages of the study which involved face- to- face interviews with arrhythmia patients, to begin the process of establishing content validity of the PROM tool. Our objective was to identify areas of the tool which required additional clarification; adding items deemed to be relevant to the patient population, and removing redundant items. The instructions and layout of the items on each questionnaire were also investigated to clarify wording and improve usability.
Retrospective pilot – methods and results
As has been previously published [16], a retrospective study was carried out at three clinical sites (University Hospital Wales, Cardiff; Queen Elizabeth Hospital, Birmingham; and Freeman Hospital, Newcastle) to determine the feasibility of administering PROMs in patients undergoing ablation treatment for a variety of symptomatic cardiac arrhythmias. The original tool comprised of a battery of three questionnaires: i) a new short, arrhythmia specific, patient expectation questionnaire developed by our multidisciplinary team; ii) an adaptation of the arrhythmia disease specific “Patient Perception of Arrhythmia” questionnaire (PPAQ) developed by Wood et al. [15]; iii) the EuroQol EQ-5D-5L questionnaire [17,18], and was sent out to 791 patients. A response rate of over 74% (n = 584) was achieved including 569 analysable responses, with a high proportion of participants reporting an improvement in symptoms and quality of life following ablation [16].
Prospective study – phase 1: PROM development and qualitative interviews
Data from the retrospective pilot study allowed a draft cardiac arrhythmia PROM tool to be developed. Questionnaire responses, feedback from free text observations and comments during telephone enquiries provided useful data to the study team for revisions to make the PROM more effective for future studies. Replies indicated that while most areas of the questionnaires appeared to be well understood, some instructions and items required clarification. Additionally, comments suggested that further items should be considered to ensure all relevant areas which are important to patients were included.
Changes were made to the questionnaires based on feedback received to produce a PROM with improved readability and clarity. Additionally, some new items were included to gather more information on fields such as co-morbidities, arrhythmia medication and complications following the procedure.
Patient enrolment
Patients under the care of physicians at two clinical sites (University Hospital Wales, Cardiff and Queen Elizabeth Hospital, Birmingham) were eligible for inclusion in this stage of the study if they had a diagnosis of symptomatic cardiac arrhythmia and were awaiting for, and/or had received a cardiac ablation. Patients were required to be at least 18 years of age, able to communicate in English or Welsh and willing to participate in interviews lasting approximately 1.5-2 hours.
Patients were initially approached by their treating physician and given a patient information sheet which explained the study. Those who were interested in taking part in interviews were asked to complete a permission sheet which authorised a researcher to contact them to discuss their involvement. Interested patients were given further details of the study and an appointment for the interview. In one case where the researcher was on-site, a patient wished to carry out the interview immediately following their clinic appointment. In all other cases the patients were given the opportunity to choose the time and the place of their interview. In 21 cases these interviews took place in the patients’ own homes; four, including the case above took place in a quiet room at a convenient hospital site.
Purposive sampling was used to obtain a representative selection of participants that included both sexes, a range of arrhythmia substrates and a wide age range of patients. Consecutive patients who met the sampling criteria were approached at four separate clinics. A total of 41 patients were invited to participate in the study, eleven declined or did not respond following initial contact. Of the remainder, two were subsequently withdrawn as their ablation procedure was cancelled and three were not required as data saturation had been reached. The remaining 25 patients took part in the interview process, 11 of these were receiving treatment in Cardiff, while 14 were under the care of physicians in Birmingham.
Prior to the interviews, study procedures were explained and informed written consent was obtained. All interviews were carried out by a single trained researcher (KLW) using a semi-structured interview guide, and were audio-taped with the patients’ permission (one patient declined, two were not recorded due to technical issues). The interview guide used a list of questions (developed by Withers et al. [16]) to direct the process. “Think aloud” techniques were used to elicit patient opinions on the PROM, with patients invited to read each question aloud, explain their understanding of the question and describe any areas which lacked clarity. Areas identified by the patient as being unclear were probed in detail and patients were asked to suggest improvements. At the end of the session, participants were asked if there were any topics they had expected to see that weren’t covered, or issues which needed to be discussed in more detail. They were also encouraged to discuss problems they experienced as a result of their arrhythmia. Recorded sessions lasted between 23 and 93 minutes (mean duration of 50 minutes). The interviews were transcribed verbatim, checked for accuracy and accompanied by field notes and annotated questionnaires. Additional interviews were carried out until the research team found no new issues were being identified, suggesting data saturation had been reached.
The process was iterative and used grounded theory methodology throughout the interviewing process, with information from initial interviews used to inform and guide later ones. The research team monitored responses and used content analysis to identify common themes from the qualitative data. Following interviews with 19 patients, the PROM was revised to reflect the feedback received. These changes were then tested in a further six patients. Figure 1 illustrates the steps taken in developing the PROM.