- Open Access
Development and validation of a condition-specific diary to measure severity, bothersomeness and impact on daily activities for patients with acute urinary tract infection in primary care
Health and Quality of Life Outcomes volume 15, Article number: 57 (2017)
Urinary tract infection (UTI) is a common condition in primary care. Patient-reported outcome measures (PROMs) are crucial in the evaluation of interventions to improve diagnosis, treatment and prognosis of UTI. The aim of this study was to identify an existing condition-specific PROM to measure symptom severity, bothersomeness and impact on daily activities for adult patients with suspected urinary tract infection in primary care; or, in the absence of such a PROM, to test items identified from existing PROMs for coverage and relevance in single and group interviews and to psychometrically validate the resulting PROM.
The literature was searched for existing PROMs covering the three domains. Items from the identified PROMs were tested in single and group interviews. The resulting symptom diary was psychometrically validated using the partial credit Rasch model for polytomous items in a cohort of 451 women participating in two studies regarding UTI.
No existing PROM fulfilled the inclusion criteria. Content validation resulted in one domain concerning symptom severity (18 items), one concerning bothersomeness (18 items), and one concerning impact on daily activities (7 items). Psychometrical validation resulted in four dimensions in each of the first two domains and one dimension in the third domain.
Domains were not unidimensional, which meant that we identified dimensions of patient-experienced UTI that differed substantially from those previously found. We recommend that future studies on UTI, in which PROMs are to be used, should ensure high content validity of their outcome measures and unidimensionality of the included dimensions.
Urinary tract infection (UTI) is a common condition and accounts for about 2% of consultations in general practice in Denmark . It mainly affects women, one in every two women experiences a UTI at least once in her life-time . Symptoms of UTI are known to be painful and bothersome, impacting quality of life [3–6]. In addition to the symptoms experienced by the patient, laboratory confirmation of a significant amount of bacteria in the urine is required for the diagnosis of a clinically relevant UTI . Patient-reported outcome measures (PROMs) are important both for the evaluation of the extent to which an intervention can improve the diagnosis and treatment of UTI, and for following the patient’s experience of symptoms and recovery. A PROM should be face and content validated to ensure that its items are relevant and covering for the construct that is to be measured. Moreover, items and instructions in the PROM should be clear and understandable for the target population [8–10]. If the PROM encompasses domains of items, then these should be psychometrically validated in a larger sample of the target population using item response theory (IRT) models to ensure unidimensionality of the domains allowing for sum scores . When items in a domain fit IRT Rasch models, invariant measurement is achieved [12–16]. A number of PROMs exist, but to our knowledge none of them have been tested for both content validity and unidimensionality of domains using IRT models [6, 17–19].
The aims of this study were to 1) Perform a literature search to identify an existing condition-specific PROM to measure symptom severity, bothersomeness and impact on daily activities over time for adult patients with uncomplicated and complicated UTI in primary care; or 2) in the absence of such a PROM, to test items identified from existing PROMs for relevance in single and group interviews with patients who had experienced UTI; and 3) to psychometrically validate the resulting PROM using Rasch models.
Aim 1: literature search for existing PROMs
We searched Medline and Embase for development and validation studies published before September 2014 in English, Swedish, Danish or Norwegian. Combinations of the words “urinary tract infection”, “cystitis”, “patient-reported outcome measure”, “psychometrics”, “PROM”, “instrument”, “validation” and “scale” were used.
PROM development and validation studies performed in primary care or a comparable setting investigating adult patients with symptoms of UTI including the three domains: Symptom severity, symptoms bothersomeness and impact on daily activities and reporting a sufficient content validation involving either single or groups interviews to ensure coverage and relevance of items and a sufficient content validation using IRT models and analysis of differential item functioning (DIF).
Aim 2: face and content validity
Overview of content validation procedure
The process of content validation involved two primary elements: 1) Item generation and construction of a draft PROM, 2) single and group interviews with members of the target population.
Items relevant to the three domains were selected from existing PROMs identified in the literature search. To narrow down the initial pool of items, the items that had proved most predictive of confirmed UTI in previous research were selected and some items were modified based on clinical experience [20, 21]. Double-barreled items (For example “pain or burning when passing urine”) were split into two individual items. The resulting draft version of the PROM was converted into a 7-day symptom diary, one of several types of PROMs.
Group interviews were aimed at expanding our knowledge on symptoms experienced by patients with UTI, their bothersomeness and impact on daily activities. The method of group interviews was chosen to ensure a dynamic generation of new items and an open discussion about the content and layout of the diary . The participants were encouraged to talk about their experience of having UTI using open-ended questions. When they had no more new symptoms or activities to add, the draft version of the diary was presented. The participants completed the draft version of the diary and it was corrected according to their suggestions. Participants were recruited from a general practice, an elderly activity center and the researchers’ network. The group interviews took about two hours and were audio recorded; the recordings were used to analyze the interviews and change the draft version of the diary.
The purpose of single interviews was to ensure relevance, coverage and understandability of the diary. The participant firstly told about his or her experience of the UTI. Afterwards, the participant was told to complete the diary and comment on relevance, coverage and understandability. Female participants were recruited from the researchers’ network and male participants were recruited at a urological department. Single interviews lasted about 30 min.
Aim 3: psychometric validation
Patients with symptoms of UTI participating in two ongoing studies [23, 24] were asked to complete the diary after seeing their general practitioner. The diary was handed out in the consultation and the participant returned it by post using a prepaid reply envelope. We used text-message reminders and telephone calls to remind patients to complete and return the diary.
Statistical analysis: Rasch analysis
The responses were analyzed using the partial credit Rasch model for polytomous items [25, 26]. If an item shows misfit to a Rasch model, it indicates that the item does not belong to the same theoretical dimension. We tested the three domains for unidimensionality. If items showed misfit we tested alternative configurations of items based on clinical, empirical or theoretical relations between symptoms rather than results from analyses. The resulting dimensions were tested for DIF. DIF indicates that other factors, such as age, affect the responses to a specific item, causing the scale to behave differently in the different subgroups [27, 28]. Finally, we tested for local dependency (LD) to evaluate whether individual items within the resulting dimensions were so closely linked that they, to some extent, were measuring the same nuances of the construct. If two items have high local dependency, they nearly correspond to a single item. Since individual symptoms are known to have poor predictive value for confirmed UTI, we did not test for discriminative ability of the identified dimensions [20, 21]. If an item did not fit any dimensions it was kept in the final questionnaire if it had high content validity.
Data management and statistics
The psychometric properties of the involved scales were tested for unidimensionality, homogeneity and DIF in relation to age, study group, and confirmed UTI, by using likelihood ratio tests on appropriately conditioned Rasch models . Confirmed UTI was defined as having significant growth of uropathogens in a reference culture. The patient was not aware of the result at the time of completing the diary. The reliability of the dimensions was examined using Cronbach’s alpha (Table 2). Statistical analyses were performed in DIGRAM . To adjust for multiple testing the false discovery rate was fixed at 5% for each set of analyses using the Benjamini-Hochberg method .
Aim 1: literature search for existing PROMs
No PROMs were identified measuring all three domains: symptom severity, bothersomeness and impact on daily activities. We identified four development and validation studies for patients with symptoms of UTI [6, 17–19]. None of these studies described the use of single or group interviews to ensure content validity. All of them were statistically validated, but only one study tested for unidimensionality using IRT but not for DIF . The identified studies are listed in Additional file 1.
Aim 2: face and content validity
The first draft version of the diary contained eight items regarding symptom severity, eight items regarding symptom bothersomeness and five items regarding daily activities (Fig. 1). Four response categories to these 21 items were drafted: 0 (none), 1 (a little), 2 (some) and 3 (a lot). Before interviewing men, six items regarding complicated UTI were added.
Single and group interviews
Two group interviews were conducted: one with four women aged 29–63 (six invited) and one with seven women from 70 to 89 (seven invited). The first group was in the latter part of the interview presented to a draft questionnaire including 21 items (Fig. 1). In the first group interview twelve new items (four on symptom severity, four on symptom bothersomeness and four on activities) were generated (Fig. 1). None of the symptoms in the draft version was considered irrelevant, but two items regarding activities were discarded (ability to concentrate and spare time activities). In the second group interview with elderly women, almost all of the same symptoms were repeated but two new items were generated: Severity and bothersomeness of feeling unwell. The elderly women had more problems identifying individual activities that were impacted when they had UTI but did not find the activities from the draft version irrelevant. They also found the diary quite long and repetitive to complete. They could, however, accept completing the items in all three scales in a single day and just the items in one scale on the following days. Both groups found the response categories sufficient and used all four options when completing the diary. We had planned to perform a group interview with men as well, but recruitment proved so difficult that we decided to conduct single interviews with men.
Six single interviews with women were performed – two after the changes following the first group interview and four after the second group interview. The result of these single interviews was minor corrections to phrasing and layout. No new items were generated in the single interviews with women.
Three men were interviewed; one interview was performed in person and two were conducted over the telephone after the diary had been sent in the post. All three men had experienced having both cystitis and pyelonephritis. Their vocabulary for describing their symptoms was different, but they found the vocabulary in the diary understandable and the items relevant. The first interview resulted in four new items related to complicated UTI. No additional items were generated in either of the other two interviews.
After four single interviews with women and two with men without any new items, we concluded that data saturation had been reached.
The result of the qualitative evaluation was three domains: one for symptom severity containing 18 items, one for symptom bothersomeness containing 18 items and one for impact on daily activities containing seven items. The process and result of the content validation can be seen in Fig. 1.
Aim 3: psychometric validation
Recruitment of patients
We included data on 451 female patients consulting their general practitioner with at least one UTI symptom. 209 of the women completed the full questionnaire regarding symptom severity, bothersomeness and daily activities. The remaining 242 only completed the items regarding symptom severity. Response rates in the two studies were 86 and 78%. We included age, job status, if the patient had confirmed UTI and which study she participated in as covariates. Inclusion of patients can be seen in Fig. 1 and characteristics of included patients in Table 1.
None of the three domains revealed unidimensionality in the initial Rasch analyses. Subsequent empirical analysis of the three domains revealed nine new dimensions covering symptom severity, symptom bothersomeness and impact on daily activities. 14 single items did not fit the dimensions, but could not be excluded from the diary without compromising content validity. The overall fit of the nine dimensions can be seen in Table 2 and the fit of individual items in Table 3.
Symptom severity (Domain S)
We suggested domain S to have a dimension regarding frequency and a dimension regarding pain (dysuria). Four symptom-items fitted the Rasch model in the frequency-dimension: Frequent urination – daytime, Increased urge for urination, Having to hurry to the toilet and Incontinence. Four combinations of items showed LD: Frequency and Urge, Frequency and Incontinence, Incontinence and Urge, and Incontinence and Having to hurry to the toilet. Three items fitted the Rasch model in the pain-dimension (Pain on urination, Difficult to empty bladder and Uncomfortable pressure around the bladder). One combination – Pain around the bladder and Difficult to empty bladder – showed LD. We suggested a dimension regarding symptoms from the lower back, and two items fitted this with no LD. Finally, we tested a dimension regarding general symptoms, which encompassed three items and had a high fit to the model and no LD. None of the final four dimensions showed DIF. Six items regarding symptom severity did not fit a dimension.
Symptom bothersomeness (Domain B)
Since the bothersomeness domain contained the same items as the symptom domain, but asked about bothersomeness instead of severity, we tested the dimensions identified in the analysis of domain S. All four dimensions fitted the model and showed no DIF. There were only two combinations of symptoms in the frequency dimension showing LD: Frequency and Urge, and Incontinence and Having to hurry to the toilet.
Impact on daily activities (Domain D)
The daily activities domain showed unidimensionality and no DIF if two items (Sleep and Sex) were removed. These two items were removed because they were related to nighttime, which the other items were not. These two items did not compose a separate dimension together. The item “Cycling” showed a low fit to the dimension, but removing it did not improve overall fit, so we decided to keep it in the dimension. The final dimension showed high levels of LD; only three combinations did not have LD: Work and Exercise, Social activities and Exercise, and Social activities and Domestic duties.
This study resulted in a substantially new symptom diary for patients with symptoms of UTI with high content validity and adequate psychometric properties, comprising four dimensions of symptom severity and bothersomeness – dysuria, frequency, lower back symptoms and general symptoms – as well as one dimension of impact on daily activities. This is to our knowledge the first symptom diary regarding UTI to have been both content and psychometrically validated.
Strengths and limitations of this study
The diary was developed through interviews with patients attending general practice, thus yielding high content validity for patients in this setting. The domains were psychometrically analyzed using a large cohort obtained through two different studies. The psychometric validation ensured unidimensionality of the scales within the three domains and no DIF. We found corresponding scales in the symptom severity and symptom bothersomeness domains, suggesting the scales to be a solid finding.
It is a limitation in this study that we were unable to recruit men for a group interview or for the psychometrical validation. However, single interviews with men showed good relevance and coverage of the identified items and the items generated in the interviews with men were not gender-specific, but related to complicated UTI. In the psychometrical validation, we did not have sufficient sociodemographic data to include covariates, such as job status and education, in all analyses. This does not compromise the identified domains, but we do not have data to confirm whether any of the items possessed DIF in relation to sociodemographics. Another weakness is the high level of LD in the scale regarding daily activities. However, this finding corresponds with data from our second group interview, where participants stated that all activities were equally affected when they had UTI. This PROM is for research purposes and the fit-statistics indicate it should not be used for individual patients.
Findings in relation to other studies
Previous instruments regarding symptoms of UTI have also covered aspects such as frequency and dysuria . However, our content validation process showed that patients do not see frequency as a uniform aspect that can be scored in a single item, but as a group of symptoms and experiences of having to hurry to the toilet, having to void often in both the daytime and the nighttime and having incontinence problems. The psychometric validation showed that most of these items – but not all – were part of the same construct. Urgency, which is usually investigated separately, turned out to be part of the frequency scale. The term dysuria was even more differently perceived by patients than by us, the clinicians. The content validation resulted in several new items dealing with different aspects of pain, since the term “pain” turned out to be too broad a concept. In the psychometric validation we found a three-item dimension comprising “pain on urination”, “difficulties emptying the bladder” and “uncomfortable pressure around the bladder”; but the items “a burning sensation on urination” and “pain around the bladder” were not part of this dimension and the patients must have perceived these as fundamentally different symptoms.
Unanswered questions and future research
This study demonstrates that patient-experienced symptoms differ from the ways in which professionals perceive them as has been previously shown . It indicates that patient interviews with the target population should always be conducted before introducing a new instrument. The study has identified new dimensions of patient-experienced UTI that differ, in terms of content, from those previously been found. The symptom diary is a robust instrument when used in studies investigating women with UTI symptoms in general practice, but we do not have sufficient data to determine whether it could be used in a male population. Before using it in a study on male patients, we would suggest performing a psychometric validation on men. We recommend that future studies on UTI, in which PROMs are to be used, should ensure high content validity of their outcome measures and unidimensionality of the included dimensions.
Several instruments have been validated to measure symptoms in patients with suspected UTI. Items and dimensions are usually generated by the researcher and statistical validation does not test for unidimensionality, but assumes, rather, that each item represents a different feature of the same construct. This study has content and psychometrically validated a new symptom diary for UTI, identifying nine unidimensional scales measuring different constructs of symptom severity, bothersomeness and impact on daily activities in patients with UTI. These scales differ substantially from those previously described in the scientific literature.
Differential item functioning
Item response theory
Patient-reported outcome measure
Urinary tract infection
Forskningsenheden for Almen Praksis Aarhus Universitet, Afdeling for Almen Medicin Aarhus Universitet. Kontakt- Og Sygdomsmønsterundersøgelse KOS 2008. 2008.
Hummers-Pradier E, Kochen M. Urinary tract infections in adult general practice patients. Br J Gen Pract. 2002;52:752–61.
Leydon GM, Turner S, Smith H, Little P. The journey from self-care to GP care: a qualitative interview study of women presenting with symptoms of urinary tract infection. Br J Gen Pract. 2009;59(564):e219–25.
Malterud K, Baerheim A. Peeing barbed wire. Symptom experiences in women with lower urinary tract infection. Scand J Prim Health Care. 1999;17(1):49–53.
Bermingham SL, Ashe JF. Systematic review of the impact of urinary tract infections on health-related quality of life. BJU Int. 2012;110(11 Pt C):E830–6.
Colgan R, Keating K, Dougouih M. Survey of symptom burden in women with uncomplicated urinary tract infections. Clin Drug Investig. 2004;24(1):55–60.
Aspevall O, Hallander H, Gant V, Kouri T. European guidelines for urinalysis: a collaborative document produced by European clinical microbiologists and clinical chemists under ECLM in collaboration with ESCMID. Scand J Clin Lab Invest. 2000;60:1–96.
Mosier CI. A critical examination of the concepts of face validity. Educ Psychol Meas. 1947;7(2):191–205.
Sireci SG. The construct of content validity. Soc Indic Res. 1998;45:83–117.
Comins JD, Krogsgaard MR, Brodersen J. Ensuring face validity in patient-related outcome scores--a matter of content. Knee. 2013;20(2):72–8.
Cano SJ, Hobart JC. The problem with health measurement. Patient Prefer Adherence. 2011;5:279–90.
Hobart J, Cano S. Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods, vol. 13. 2009.
Comins JD, Krogsgaard MR, Brodersen J. Development of the Knee Numeric-Entity Evaluation Score (KNEES-ACL): a condition-specific questionnaire. Scand J Med Sci Sports. 2013;23(5):e293–301.
Brodersen J, McKenna SP, Doward LC, Thorsen H. Measuring the psychosocial consequences of screening. Health Qual Life Outcomes. 2007;5:1–4.
Aabenhus R, Thorsen H, Siersma V, Brodersen J. The development and validation of a multidimensional sum-scaling questionnaire to measure patient-reported outcomes in acute respiratory tract infections in primary care: the acute respiratory tract infection questionnaire. Value Health. 2013;16(6):987–92.
Comins JD, Krogsgaard MR, Kreiner S, Brodersen J. Dimensionality of the Knee Numeric-Entity Evaluation Score (KNEES-ACL): a condition-specific questionnaire. Scand J Med Sci Sport. 2013;23(5):302–12.
Wild D, Clayson D. Validation of a patient-administered questionnaire to measure the activity impairment experienced by women with uncomplicated urinary tract infection: the Activity Impairment. Heal Qual Life Outcomes. 2005;3(42).
Clayson D, Wild D, Doll H, Keating K, Gondek K. Validation of a patient-administered questionnaire to measure the severity and bothersomeness of lower urinary tract symptoms in uncomplicated urinary tract infection (UTI): the UTI Symptom Assessment questionnaire. BJU Int. 2005;96(3):350–9.
Alidjanov JF, Abdufattaev UA, Makhsudov SA. New self-reporting questionnaire to assess urinary tract infections and differential diagnosis: Acute cystitis symptom score. Urol Int. 2014;92:230–6.
Bent S, Nallamothu B, Simel D. Does this woman have an acute uncomplicated urinary tract infection? JAMA. 2002;287(20):2701–10.
Giesen L, Cousins G, Dimitrov BD, van de Laar F, Fahey T. Predicting acute uncomplicated urinary tract infection in women: a systematic review of the diagnostic accuracy of symptoms and signs. BMC Fam Pract. 2010;11(78).
Vogt DS, King DW, King L. a. Focus groups in psychological assessment: enhancing content validity by consulting members of the target population. Psychol Assess. 2004;16(3):231–43.
Holm A, Cordoba G, Sørensen TM, Jessen LR, Siersma V, Bjerrum L. Point of care susceptibility testing in primary care - does it lead to a more appropriate prescription of antibiotics in patients with uncomplicated urinary tract infections? Protocol for a randomized controlled trial. BMC Fam Pract. 2015;16(1):106.
Cordoba G, Sørensen TM, Holm A, Bjørnvad CR, Bjerrum L, Jessen LR. Exploring the feasibility and synergistic value of the One Health approach in clinical research: protocol for a prospective observational study of diagnostic pathways in human and canine patients with suspected urinary tract infection. Pilot Feasibility Stud. 2015;1(1):38.
Tennant A, Conaghan PG. The Rasch measurement model in rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Care Res. 2007;57(8):1358–62.
Turk DC, Dworkin RH, Burke LB, et al. Developing patient-reported outcome measures for pain clinical trials: IMMPACT recommendations. Pain. 2006;125(3):208–15.
Bjorner JB, Kreiner S, Ware JE, Damsgaard MT, Bech P. Differential item functioning in the Danish translation of the SF-36. J Clin Epidemiol. 1998;51(11):1189–202.
Brodersen J, Meads D, Kreiner S, Thorsen H, Doward L, McKenna S. Methodological aspects of differential item functioning in the Rasch model. J Med Econ. 2007;10(3):309–24.
Andersen EB. A goodness of fit test for the rasch model. Psychometrika. 1973;38(1):123–40.
Kreiner S. Introduction to DIGRAM, Department of Biostatistics Research Report 03/10. Copenhagen: Biostatistisk Afdeling University of Copenhagen; 2003.
Hochberg YB, Benjamini Y. Controlling the false discovery rate : a practical and powerful approach to multiple testing. J R Stat Soc Ser B. 1995;57(1):289–300.
We would like to thank all patients who took the time to participate in this study.
This study is funded by: a) 2016, University of Copenhagen b) Læge Sofus Carl Emil Friis og Hustru Olga Doris Friis’ legat.
Availability of data and materials
The datasets generated and analyzed during the current study are not publicly available due to privacy issues but are available in anonymized form from the corresponding author on reasonable request.
All authors have made substantial contributions to the scientific work and the manuscript and have approved the final version of the manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Participation in the single and group interviews was voluntary and anonymous. The two trials received approval from the Ethical Committees for the Capital Region of Denmark (approval no H-3-2014-107 and H-4-2014-097). All patients gave written informed consent to participate in the study. There were no safety issues regarding this study. The projects were presented to the Danish Data Protection Agency, but did not require additional approval since approval was obtained from the Ethical Committees.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Holm, A., Cordoba, G., Siersma, V. et al. Development and validation of a condition-specific diary to measure severity, bothersomeness and impact on daily activities for patients with acute urinary tract infection in primary care. Health Qual Life Outcomes 15, 57 (2017). https://doi.org/10.1186/s12955-017-0629-5
- Urinary tract infections
- Validation studies
- Item-response theory
- Rasch analysis
- Patient-reported outcomes
- Patient-reported outcome measures
- Primary care