Skip to main content

Table 1 Characteristics of studies included in the meta–analysis

From: Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013

  Year Study description Equivalence indices Rigour
Authors and study ID   eMode D R AP N Population Mean age (SD) eMode comparison Measure(s) SMD   Corr K Time lag PC/95 % CIs
Araujo et al. [16] 2012 Web/PC C Y N 21 Asthma 29(10) Paper–web/PC ACQ, AQLQ 0.025 4 weeks PC , CI
Ashley et al. [17] 2012 Web/PC C Y N 111 Cancer 57(13.2) Paper–web/PC SDI–21 0.006 0.89 19 days Unknown
Basnov et al. [18] 2009 Web/PC C Y N 41 Cancer 47.2(9) Paper–web/PC SF–36 0.77 ≈2 weeks CI
Beaumont et al. [19] 2011 Web/PC P Y N 1006 COPD 55(11.4) Paper–web/PC COPD–PS 0.012 0.82 5–7 days PC , CI
Bennett et al. [20] 2013 Web/PC C* Y Y–A 170 Cancer 56(11) Paper–web/PC BFI, NRS LASA QOL 0.97 Next day Unknown
IVRS        Paper–IVRS 0.89
Web/PC–IVRS 0.88
Bernabe–Ortiz et al. [21] 2008 PDA W N N 200 Sexual Health 22.9(3.4) Paper–PDA [STD Symptoms] 0.86 Immediate No
Bernstein et al. [22] 2013 Web/PC W N N 116 Sexual Health   Paper–web/PC SHIM 0.002 0.87 1 week Unknown
Bishop et al. [23] 2010 Web/PC C Y N 167 Back Pain 46.28 Paper–web/PC RMDQ 0.001 0.97 Immediate PC
Bushnell et al. [24] 2012 Web/PC C Y Y–P 314 General 53(12.5) Paper–web/PC ENSEMBLE MDS 0.87 24 hours Unknown
Bushnell et al. [25] (a) 2013 Web/PC C Y Y–P 228 General 44.3(13.5) Paper–web/PC PDHCO 0.90 1 week Unknown
Bushnell et al. [26] (b) 2013 Web/PC C Y N 63 Dermatology 50.2(13.6) Paper–web/PC PSI 0.023 0.96 24 hours Unknown
Carlbring et al. [27] 2007 Web/PC C Y N 494 Panic disorder 37.6(10.9) Paper–web/PC BSQ, ACQ, MI, BAI, 0.021 0.90 <36 hours PC
BDI, QOLI, MADRAS
Chen et al. [28] 2007 Web/PC C Y N 150 General 30.8 Paper–web/PC SF–36 0.013 10 minutes PC , CI
Clayer et al. [29] 2011 Web/PC W N N 46 Cancer 53.5(13.9) Paper–web/PC TESS 0.005 0.97 7 days CI
Coles et al. [30] 2007 Web/PC C Y N 105 OCD 19.01(1.41) Paper–web/PC OCI, OBQ–44 0.027 0.78 ≈2 days No
Cook et al. [31] 2007 Tablet C Y N 80 Depression 44.1(11.6) Paper–Tablet QIDS–SR16 0.007 0.99 Immediate CI
Cubo et al. [32] 2012 Web/PC C Y N 42 Parkinson’s 64.7(9) Paper–web/PC PDQ–39, NMSQ, UPDRS 0.82 <5 days PC , CI
II, UPDRS IV
Dalal et al. [33] 2011 Web/PC IVRS C Y N 149 COPD 53.1(10) Paper–web/PC LFQ 0.014 0.81 1 week CI
Paper–IVRS 0.005 0.93
Dunn et al. [34] 2007 IVRS W N N 99 Sexual function 31 Paper–IVRS CSFQ 0.91 Unknown Unknown
Dupont et al. [35] 2009 Tablet W Y N 56 Cancer 54(13) Paper–Tablet FACT–G (Social Well–Being Subscale) 0.035 1 minute PC , CI
Gibbons et al. [36] 2011 PDA C Y N 12 Appetite 25.6(6.3) Paper–PDA VAS 0.93 30 minutes CI
Godfrey et al. [37] 2013 Web/PC C Y N 35 Shoulder injury 48 Paper–web/PC WORC 0.016 0.89 Immediate PC
Griffiths–Jones et al. [38] 2012 Web/PC C Y Y–A 47 Hip injury   Paper–web/PC Oxford hip score, 0.004 0.97 1 week PC , CI
McCarthy hip score, UCLA activity, howRu
Gudbergsen et al. [39] 2011 Tablet C Y N 20 Osteoarthritis 66.5(7) Paper–Tablet KOOS, VAS (pain, 0.024 0.95 5 minutes PC , CI
function, global), SF– 36, PainDirect
Handa et al. [40] 2008 Web/PC C Y N 43 Gynecology 52(13) Paper–web/PC PFDI–20, PFIQ–7 0.86 <6 weeks PC , CI
Heiberg et al. [41] 2007 PDA C N N 38 Rheumatology 58.4(12.9) Paper–PDA VAS, SF–36, m–HAQ 0.010 3 weeks Unknown
Hollandare et al. [42] 2010 Web/PC C Y N 87 Depression 41.1(13) Paper–web/PC MADRS–S, BDI–II 0.012 0.87 ≈10 days No
Hollen et al. [43] 2013 PDA W N N 86 Cancer 67 Paper–PDA LCSS 0.92 15 minutes CI
Inman et al. [44] 2012 Web/PC W N Y–A 1439 Rheumatology 56.5(14) Paper–web/PC HAQ–II, Pain, GA, 0.006 6 months Unknown
Fatigue
Jaspan et al. [45] 2007 PDA C Y N 212 Sexual beh. 14.5(2.75) Paper–PDA Individual items 0.035 0.72 2 weeks CI
Jones et al. [46] 2008 Web/PC P N N 183 Mental health 20.1(2.6) Paper–web/PC PIQ, LSHS–R 0.016 N/A No
Juniper et al. [47] 2007 PDA C Y N 70 Rhino–conjunctivitis 41 Paper–PDA RQLQ 0.014 0.90 2 hours No
Juniper et al. [48] 2009 PDA C Y N 68, 27 Asthma & Rhino– conjunctivitis 41 Paper–PDA AQLQ(S), ACQ, RQLQ(S) 0.89 0.89 2 hours CI
Junker et al. [49] 2008 PDA C Y N 200 Chronic pain 57 Paper–PDA Average, present, worst pain & PainDetect 0.026 <1 day PC , CI
Kajander et al. [50] 2007 PDA W N Y–A 15 IBS 42 Paper–PDA IBS 0.008 0.96 Unknown CI
Koolen et al. [51] 2011 Web/PC C* Y N 156 Asthma 11.25(1.9) Paper–web/PC C–ACT, ACT 0.83 <5 days PC , CI
Lam et al. [52] 2009 IVRS C Y N 64 IBD 43.1(13.8) Paper–IVRS SIBDQ 0.013 0.89 ≈7 days No
Lee et al. [53] 2009 Tablet C Y N 261 Asthma 40.8(12.1) Paper–Tablet A–QOL 0.85 Unknown Unknown
Luce et al. [54] 2007 Web/PC C* Y N 74 Eating disorders 15.4(0.3) Paper–web/PC risk for eating disorders 0.025 0.74 1 week Unknown
Lundy and Coons [55] 2011 IVRS C Y N 113 General 61.5 Paper–IVRS EQ–5D index & VAS 0.018 0.89 0.71 3 days Unknown
Lundy et al. [56] 2013 IVRS C Y N 139 Cancer 61.5 Paper–IVRS QLQ–C30 0.015 0.82 2 days PC , CI
Mackenzie et al. [57] 2011 Web/PC C Y N 56–63 Psoriatic 53 Paper–web/PC HAQ, SF–36, mFSS, 0.003 0.95 Consecutive CI
Arthritis FACIT–F, DLQI, BASDAI, BASFI, BASG, BASQoL, EQ–5D
Marceau et al. [58] 2007 PDA C Y N 36 Chronic Pain 48 (8) Paper–PDA Pain diary 0.020     2 weeks No
Matthew et al. [59] 2007 PDA C* Y N 39–53 Cancer 67.2(10.3) Paper–PDA IPSS 0.042 0.85    30 minutes No
McCarrier et al. (a) [60] 2011 Web/PC C* Y Y–A 258 Mental Health 48.6(13.5) Paper–web/PC PHQ– 4 0.86 1 week Unknown
McCarrier et al. (b) [61] 2011 Web/PC C* Y Y–P 256 General 48.6(13.5) Paper–web/PC MOS–SSS 0.89 1 week Unknown
McCarrier et al. [62] 2013 Web/PC C* Y Y–P 230 Multiple 44.3(13.5) Paper–web/PC DBS 0.020 0.88 24 hours PC , CI
Mundt et al. [63] 2010 IVRS W N N 62 Mental health   Paper–IVRS DAS–A 0.019 0.87 0.89 24 hours Unknown
Parnell et al. [64] 2011 Web/PC C Y N 50 Pelvic floor 50.4(11.6) Paper–web/PC PISQ–12 0.008 0.88 2 weeks PC , CI
Raat et al. (a) [65] 2007 Web/PC P Y N 933 Child health 14.7(0.68) Paper–web/PC CHQ 0.010 Unknown
Raat et al. (b) [66] 2007 Web/PC P Y N 933 Asthma 14.7(0.68) Paper–web/PC ISAAC 0.008 N/A PC
Ramachandran et al. [67] 2008 Tablet C Y N 314 General 35.5(14) Paper–Tablet EQ VAS 0.014 0.75 Unknown Unknown
Read et al. [68] 2009 Web/PC C N N 38 Trauma 19.6(1.5) Paper–web/PC TLEQ, PCL–C 0.058 0.69 1 week No
Richardson et al. [69] 2009 Web/PC P N N 354 Smoking Dx 16(1.55) Paper–web/PC Social and emotional dependence, physical and sensory 0.052 N/A No
Richter et al. [70] 2008 Tablet C Y N 153 Rheumatology 45.7(14.4) Paper–Tablet FFbH. HAQ. BASDAI, SF–36 0.001 0.97 Unknown Unknown
Salaffi et al. [71] 2009 Tablet C Y N 87 Rheumatology 65 Paper–Tablet VAS–GH/Pain/PGA, ROAD, TJC 0.008 0.92 60 minutes CI
Saunders et al. [72] 2007 Web/PC C* N N 50 Hearing loss 65.6(8.9) Paper–web/PC ALHQ 0.74 9–10 days CI
Shervin et al. [73] 2011 Web/PC W N N 61 Osteoarthritisa 63 Paper–web/PC The Harris hip score, 0.017 0.85 Immediate PC
Tablet Others Paper–Tablet WOMAC, SF–36, EQ– 0.005 0.84
Tablet–web/PC 5D, UCLA activity score 0.013 0.90
Swartz et al. [74] 2007 PDA C Y N 756 Mental Health 55(13) Paper–PDA CES–D 0.023 Immediate Unknown
Thoren et al. [75] 2012 Web/PC C Y N 53 Hearing loss 68.3(11.3) Paper–web/PC HHIE, IOI–HA, SADL, HADS 0.016 0.73 3 weeks No
Tiplady et al. [76] 2010 PDA C Y N 43 Rheumatoid Arthritis 57 Paper–PDA HAQ–DI, EQ–5D, BPI, MPQ–SF, FACIT–F, SF–36, SARA 0.009 0.88 1 hour CI
Turvey et al. [77] 2012 IVRS W N N 51 Mental Health 68(8) Paper–IVRS PHQ–9 0.036 0.65 1 week No
Vallejo et al. [78] 2007 Web/PC W N N 185 Mental Health 27.4(10.0) Paper–web/PC GHQ–28, SCL–90–R 0.020 0.69 ≈17 days No
Vallejo et al. [79] 2008 Web/PC C Y N 40 Mental Health 22.2 Paper–web/PC GHQ–28, SCL–90
(Spanish)
0.84 <1 week No
Varni et al. [80] 2008 Web/PC C Y N 92 Diabetes 13.2(3.42) Paper–web/PC PedsQL 4.0 0.007 0.89    <5 minutes No
Vinney et al. [81] 2011 PDA C Y N 19 Speech 9.58(1.22) Paper–PDA PedsQL 4.0. 0.008 0.86 3 weeks PC
Whitehead et al. [82] 2011 Web/PC P Y N 1034 Mental Health 24.07(8.5) Paper–web/PC HADS, SF–36v2, FSI and Fatigue item 0.012 N/A CI
Wijndaele et al. [83] 2007 Web/PC W N N 130 Mental health 46.5 Paper–web/PC GHQ–12, SCL–90–R, MOS–SSS, PSS, UCL 0.76 1 week No
Wu et al. [84] 2009 Web/PC C Y N 34 Heart failure 49(14.2) Paper–web/PC KCCQ, MLHFQ, SCHFI 0.027 2 weeks CI
Young et al. [85] 2009 Web/PC C Y N 69 Child health 11(1.55) Paper–web/PC ASK, PedsQL 0.013 0.81 2 weeks CI
Yu and Yu, 2007 [86] 2007 Web/PC P Y N 1171 Mental health   Paper–web/PC CES–D Chinese 0.019 N/A No
Zimmerman & Martinez [87] 2012 Web/PC W N N 53 Mental health 45.1(12.3) Paper–web/PC CUDOS 0.009 0.96 <2 days No
  1. C crossover, C* 3/4–group crossover, P parallel, W within subjects, D design, R randomisation, AP abstract/poster, K kappa (weighted or unweighted), SMD scaled mean difference (study average), Time lag time between administrations, PC/95 % CIs power calculation or precise 95 % confidence intervals, Dx diseases
  2. ACQ Agoraphobic Cognitions Questionnaire, ACT Asthma Control Test, ALHQ The Attitudes towards Loss of Hearing Questionnaire, AQA Asthma Control Questionnaire, AQLQ(S) Asthma Quality of Life Questionnaire, ASK Activities Scale for Kids, A–QOL Asthma–specific Quality of Life, BAI Beck Anxiety Inventory (BAI), BASDAI Bath Ankylosing Spondylitis Disease Activity Index, BASFI Bath Ankylosing Spondylitis Functional Index, BASG Bath Ankylosing Spondylitis Global Score, BASQoL Ankylosing Spondylitis Quality of Life Instrument, BDI Beck Depression Inventory, BFI Bowel Function Instrument, BPI Brief Pain Inventory, BSQ Body Sensations Questionnaire, C–AC Childhood Asthma Control Test, CES–D Center for Epidemiologic Studies Depression, CHQ Child Health Questionnaire, COPD Chronic Obstructive Pulmonary Disease, COPD–PS COPD Population Screener, CSFQ Changes in Sexual Functioning Questionnaire, CUDOS Clinically Useful Depression Outcome Scale, DAS–A Assessment Scale for Anxiety, DBS 4 item Disease Burden Scale, DLQI Dermatology Life Quality Index, ENSEMBLE MDS a battery of phenotypic patient–reported instruments administered at baseline in clinical studies, EQ–5D EuroQOL–5 Dimensions, EQ VAS EuroQOL Visual Analog Scale, FACT Functional Assessment of Cancer Therapy, FACIT–F The Functional Assessment of Chronic Illness Therapy Fatigue, FFbH Hannover Functional Ability Questionnaire, FSI Fatigue Symptom Inventory, GA Global Assessment, GHQ–12 General Health Questionnaire–12 items, GHQ–28 General Health Questionnaire–28 items, HAQ–II Health Assessment Questionnaire II, HAQ–DI Health Assessment Questionnaire Disability Index, HHIE Hearing Handicap Inventory for the Elderly, howRU a short generic tool for measuring patient–reported outcomes, IBD Irritable Bowel Disease, IBS Irritable Bowel Syndrome, IOI–HA International Outcome Inventory for Hearing Aids, IPSS International Prostate Symptom Score, ISSAC Eight items from the International Study of Asthma and Allergies in Childhood, KCCQ Kansas City Cardiomyopathy Questionnaire, KOOS Knee injury and Osteoarthritis Outcome Score, LCSS Lung Cancer Symptom Scale, LFQ Lung Function Questionnaire, LSHS–R The revised Launay–Slade Hallucination Scale, MADRAS Montgomery Asberg Depression Rating Scale, MADRS–S Montgomery Asberg Depression Rating Scale, m–FSS The modified Fatigue Severity Scale, m–HAQ Modified Health Assessment Questionnaire, MI Mobility Inventory, MLHFQ Minnesota Living with Heart Failure Questionnaire, MOS–SSS Medical Outcomes Study Social Support Scale, MPQ–SF McGill Pain Questionnaire, NMSQ Non–Motor Symptoms Questionnaire; NRS LASA QoL Numerical Rating Scale Linear Analogue Self–Assessment of Quality of Life, OBQ–44 Obsessive Beliefs Questionnaire–44, OCI Obsessive Compulsive Inventory, PCL–C Posttraumatic Stress Disorder Checklist – Civilian Version, PDHCO The Provider–Dependent Health Care Orientation, PDQ–39 Parkinson´s Disease Questionnaire, PedsQL 4.0 Pediatric Quality of Life Inventory, PFDI–20 Pelvic Floor Distress Inventory–20, PFIQ–7 Pelvic Floor Impact Questionnaire–7, PHQ–4 Patient Health Questionnaire, PHQ–9 Patient Health Questionnaire, PIQ The Persecutory Ideation Questionnaire, PISQ–12 Pelvic Organ Prolapse/Urinary Incontinence Sexual Function Questionnaire, PSI Psoriasis Symptom Inventory, PSS Perceived Stress Scale, QLQ–C30 Quality of Life Questionnaire – Cancer 30 items, QOLI Quality Of Life Inventory, QUIDS–SR16 16–item Quick Inventory of Depressive Symptomatology Self–Rated, RMDQ Roland Morris Disability Questionnaire, ROAD Recent–Onset Arthritis Disability questionnaire, RQLQ(S) Rhinoconjunctivitis Quality of Life Questionnaire, SADL Satisfaction with Amplification in Daily Life, SARA Subjects Assessment of Rheumatoid Arthritis, SCHFI Self–Care of Heart Failure Index, SCL–90–R Symptoms Check–List–90–Revised, SDI–21 Social Difficulties Inventory, SF–36 Short Form Survey 36 items, SHIM Sexual Health Inventory for Men, SIBDQ The Short Inflammatory Bowel Disease Questionnaire, TESS Toronto Extremity Salvage Score, SS–5 Perceived Social Support, TESS The Toronto Extremity Salvage Score, TIBI Total Illness Burden Index Truncated Questionnaire, TJC Tender Joint Count, TLEQ Traumatic Life Events Questionnaire, UCLA–A University of California at Los Angeles activity score, UCLA–A Utrecht Coping List, UPDRS II Unified Parkinson's Disease Rating Scale II, UPDRS IV Unified Parkinson's Disease Rating scale IV, VAS Visual Analog Scale, VAS–GH Visual Analog Scale – General Health, WOMAC Western Ontario and McMaster Universities Osteoarthritis Index, WORC The Western Ontario Rotator Cuff Index
  3. aWhen reported as ≥0.85 this is recorded as 0.85