Skip to main content

Table 1 Characteristics of studies included in the meta–analysis

From: Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013

 

Year

Study description

Equivalence indices

Rigour

Authors and study ID

 

eMode

D

R

AP

N

Population

Mean age (SD)

eMode comparison

Measure(s)

SMD

 

Corr

K

Time lag

PC/95 % CIs

Araujo et al. [16]

2012

Web/PC

C

Y

N

21

Asthma

29(10)

Paper–web/PC

ACQ, AQLQ

0.025

–

–

–

4 weeks

PC ✓, CI ✓

Ashley et al. [17]

2012

Web/PC

C

Y

N

111

Cancer

57(13.2)

Paper–web/PC

SDI–21

0.006

0.89

–

–

19 days

Unknown

Basnov et al. [18]

2009

Web/PC

C

Y

N

41

Cancer

47.2(9)

Paper–web/PC

SF–36

–

0.77

–

–

≈2 weeks

CI ✓

Beaumont et al. [19]

2011

Web/PC

P

Y

N

1006

COPD

55(11.4)

Paper–web/PC

COPD–PS

0.012

0.82

–

–

5–7 days

PC ✓, CI ✓

Bennett et al. [20]

2013

Web/PC

C*

Y

Y–A

170

Cancer

56(11)

Paper–web/PC

BFI, NRS LASA QOL

–

0.97

–

–

Next day

Unknown

IVRS

      

Paper–IVRS

0.89

Web/PC–IVRS

0.88

Bernabe–Ortiz et al. [21]

2008

PDA

W

N

N

200

Sexual Health

22.9(3.4)

Paper–PDA

[STD Symptoms]

–

–

–

0.86

Immediate

No

Bernstein et al. [22]

2013

Web/PC

W

N

N

116

Sexual Health

 

Paper–web/PC

SHIM

0.002

0.87

–

–

1 week

Unknown

Bishop et al. [23]

2010

Web/PC

C

Y

N

167

Back Pain

46.28

Paper–web/PC

RMDQ

0.001

0.97

–

–

Immediate

PC ✓

Bushnell et al. [24]

2012

Web/PC

C

Y

Y–P

314

General

53(12.5)

Paper–web/PC

ENSEMBLE MDS

–

0.87

–

–

24 hours

Unknown

Bushnell et al. [25] (a)

2013

Web/PC

C

Y

Y–P

228

General

44.3(13.5)

Paper–web/PC

PDHCO

–

0.90

–

–

1 week

Unknown

Bushnell et al. [26] (b)

2013

Web/PC

C

Y

N

63

Dermatology

50.2(13.6)

Paper–web/PC

PSI

0.023

0.96

–

–

24 hours

Unknown

Carlbring et al. [27]

2007

Web/PC

C

Y

N

494

Panic disorder

37.6(10.9)

Paper–web/PC

BSQ, ACQ, MI, BAI,

0.021

–

0.90

–

<36 hours

PC ✓

BDI, QOLI, MADRAS

Chen et al. [28]

2007

Web/PC

C

Y

N

150

General

30.8

Paper–web/PC

SF–36

0.013

–

–

–

10 minutes

PC ✓, CI ✓

Clayer et al. [29]

2011

Web/PC

W

N

N

46

Cancer

53.5(13.9)

Paper–web/PC

TESS

0.005

0.97

–

–

7 days

CI ✓

Coles et al. [30]

2007

Web/PC

C

Y

N

105

OCD

19.01(1.41)

Paper–web/PC

OCI, OBQ–44

0.027

–

0.78

–

≈2 days

No

Cook et al. [31]

2007

Tablet

C

Y

N

80

Depression

44.1(11.6)

Paper–Tablet

QIDS–SR16

0.007

0.99

–

–

Immediate

CI ✓

Cubo et al. [32]

2012

Web/PC

C

Y

N

42

Parkinson’s

64.7(9)

Paper–web/PC

PDQ–39, NMSQ, UPDRS

–

0.82

–

–

<5 days

PC ✓, CI ✓

II, UPDRS IV

Dalal et al. [33]

2011

Web/PC IVRS

C

Y

N

149

COPD

53.1(10)

Paper–web/PC

LFQ

0.014

0.81

–

–

1 week

CI ✓

Paper–IVRS

0.005

0.93

Dunn et al. [34]

2007

IVRS

W

N

N

99

Sexual function

31

Paper–IVRS

CSFQ

–

–

0.91

–

Unknown

Unknown

Dupont et al. [35]

2009

Tablet

W

Y

N

56

Cancer

54(13)

Paper–Tablet

FACT–G (Social Well–Being Subscale)

0.035

–

–

–

1 minute

PC ✓, CI ✓

Gibbons et al. [36]

2011

PDA

C

Y

N

12

Appetite

25.6(6.3)

Paper–PDA

VAS

–

–

0.93

–

30 minutes

CI ✓

Godfrey et al. [37]

2013

Web/PC

C

Y

N

35

Shoulder injury

48

Paper–web/PC

WORC

0.016

0.89

–

–

Immediate

PC ✓

Griffiths–Jones et al. [38]

2012

Web/PC

C

Y

Y–A

47

Hip injury

 

Paper–web/PC

Oxford hip score,

0.004

0.97

–

–

1 week

PC ✓, CI ✓

McCarthy hip score, UCLA activity, howRu

Gudbergsen et al. [39]

2011

Tablet

C

Y

N

20

Osteoarthritis

66.5(7)

Paper–Tablet

KOOS, VAS (pain,

0.024

0.95

–

–

5 minutes

PC ✓, CI ✓

function, global), SF– 36, PainDirect

Handa et al. [40]

2008

Web/PC

C

Y

N

43

Gynecology

52(13)

Paper–web/PC

PFDI–20, PFIQ–7

–

0.86

–

–

<6 weeks

PC ✓, CI ✓

Heiberg et al. [41]

2007

PDA

C

N

N

38

Rheumatology

58.4(12.9)

Paper–PDA

VAS, SF–36, m–HAQ

0.010

–

–

–

3 weeks

Unknown

Hollandare et al. [42]

2010

Web/PC

C

Y

N

87

Depression

41.1(13)

Paper–web/PC

MADRS–S, BDI–II

0.012

–

0.87

–

≈10 days

No

Hollen et al. [43]

2013

PDA

W

N

N

86

Cancer

67

Paper–PDA

LCSS

–

0.92

–

–

15 minutes

CI ✓

Inman et al. [44]

2012

Web/PC

W

N

Y–A

1439

Rheumatology

56.5(14)

Paper–web/PC

HAQ–II, Pain, GA,

0.006

–

–

–

6 months

Unknown

Fatigue

Jaspan et al. [45]

2007

PDA

C

Y

N

212

Sexual beh.

14.5(2.75)

Paper–PDA

Individual items

0.035

–

0.72

–

2 weeks

CI ✓

Jones et al. [46]

2008

Web/PC

P

N

N

183

Mental health

20.1(2.6)

Paper–web/PC

PIQ, LSHS–R

0.016

–

–

–

N/A

No

Juniper et al. [47]

2007

PDA

C

Y

N

70

Rhino–conjunctivitis

41

Paper–PDA

RQLQ

0.014

0.90

–

–

2 hours

No

Juniper et al. [48]

2009

PDA

C

Y

N

68, 27

Asthma & Rhino– conjunctivitis

41

Paper–PDA

AQLQ(S), ACQ, RQLQ(S)

–

0.89

0.89

–

2 hours

CI ✓

Junker et al. [49]

2008

PDA

C

Y

N

200

Chronic pain

57

Paper–PDA

Average, present, worst pain & PainDetect

0.026

–

–

–

<1 day

PC ✓, CI ✓

Kajander et al. [50]

2007

PDA

W

N

Y–A

15

IBS

42

Paper–PDA

IBS

0.008

0.96

–

–

Unknown

CI ✓

Koolen et al. [51]

2011

Web/PC

C*

Y

N

156

Asthma

11.25(1.9)

Paper–web/PC

C–ACT, ACT

–

0.83

–

–

<5 days

PC ✓, CI ✓

Lam et al. [52]

2009

IVRS

C

Y

N

64

IBD

43.1(13.8)

Paper–IVRS

SIBDQ

0.013

–

0.89

–

≈7 days

No

Lee et al. [53]

2009

Tablet

C

Y

N

261

Asthma

40.8(12.1)

Paper–Tablet

A–QOL

–

–

–

0.85

Unknown

Unknown

Luce et al. [54]

2007

Web/PC

C*

Y

N

74

Eating disorders

15.4(0.3)

Paper–web/PC

risk for eating disorders

0.025

–

–

0.74

1 week

Unknown

Lundy and Coons [55]

2011

IVRS

C

Y

N

113

General

61.5

Paper–IVRS

EQ–5D index & VAS

0.018

0.89

–

0.71

3 days

Unknown

Lundy et al. [56]

2013

IVRS

C

Y

N

139

Cancer

61.5

Paper–IVRS

QLQ–C30

0.015

0.82

–

–

2 days

PC ✓, CI ✓

Mackenzie et al. [57]

2011

Web/PC

C

Y

N

56–63

Psoriatic

53

Paper–web/PC

HAQ, SF–36, mFSS,

0.003

0.95

–

–

Consecutive

CI ✓

Arthritis

FACIT–F, DLQI, BASDAI, BASFI, BASG, BASQoL, EQ–5D

Marceau et al. [58]

2007

PDA

C

Y

N

36

Chronic Pain

48 (8)

Paper–PDA

Pain diary

0.020

   

2 weeks

No

Matthew et al. [59]

2007

PDA

C*

Y

N

39–53

Cancer

67.2(10.3)

Paper–PDA

IPSS

0.042

0.85

  

30 minutes

No

McCarrier et al. (a) [60]

2011

Web/PC

C*

Y

Y–A

258

Mental Health

48.6(13.5)

Paper–web/PC

PHQ– 4

–

0.86

–

–

1 week

Unknown

McCarrier et al. (b) [61]

2011

Web/PC

C*

Y

Y–P

256

General

48.6(13.5)

Paper–web/PC

MOS–SSS

–

0.89

–

–

1 week

Unknown

McCarrier et al. [62]

2013

Web/PC

C*

Y

Y–P

230

Multiple

44.3(13.5)

Paper–web/PC

DBS

0.020

0.88

–

–

24 hours

PC ✓, CI ✓

Mundt et al. [63]

2010

IVRS

W

N

N

62

Mental health

 

Paper–IVRS

DAS–A

0.019

0.87

0.89

–

24 hours

Unknown

Parnell et al. [64]

2011

Web/PC

C

Y

N

50

Pelvic floor

50.4(11.6)

Paper–web/PC

PISQ–12

0.008

0.88

–

–

2 weeks

PC ✓, CI ✓

Raat et al. (a) [65]

2007

Web/PC

P

Y

N

933

Child health

14.7(0.68)

Paper–web/PC

CHQ

0.010

–

–

–

–

Unknown

Raat et al. (b) [66]

2007

Web/PC

P

Y

N

933

Asthma

14.7(0.68)

Paper–web/PC

ISAAC

0.008

–

–

–

N/A

PC ✓

Ramachandran et al. [67]

2008

Tablet

C

Y

N

314

General

35.5(14)

Paper–Tablet

EQ VAS

0.014

0.75

–

–

Unknown

Unknown

Read et al. [68]

2009

Web/PC

C

N

N

38

Trauma

19.6(1.5)

Paper–web/PC

TLEQ, PCL–C

0.058

–

0.69

–

1 week

No

Richardson et al. [69]

2009

Web/PC

P

N

N

354

Smoking Dx

16(1.55)

Paper–web/PC

Social and emotional dependence, physical and sensory

0.052

–

–

–

N/A

No

Richter et al. [70]

2008

Tablet

C

Y

N

153

Rheumatology

45.7(14.4)

Paper–Tablet

FFbH. HAQ. BASDAI, SF–36

0.001

–

0.97

–

Unknown

Unknown

Salaffi et al. [71]

2009

Tablet

C

Y

N

87

Rheumatology

65

Paper–Tablet

VAS–GH/Pain/PGA, ROAD, TJC

0.008

0.92

–

–

60 minutes

CI ✓

Saunders et al. [72]

2007

Web/PC

C*

N

N

50

Hearing loss

65.6(8.9)

Paper–web/PC

ALHQ

–

–

0.74

–

9–10 days

CI ✓

Shervin et al. [73]

2011

Web/PC

W

N

N

61

Osteoarthritisa

63

Paper–web/PC

The Harris hip score,

0.017

–

0.85

–

Immediate

PC ✓

Tablet

Others

Paper–Tablet

WOMAC, SF–36, EQ–

0.005

0.84

Tablet–web/PC

5D, UCLA activity score

0.013

0.90

Swartz et al. [74]

2007

PDA

C

Y

N

756

Mental Health

55(13)

Paper–PDA

CES–D

0.023

–

–

–

Immediate

Unknown

Thoren et al. [75]

2012

Web/PC

C

Y

N

53

Hearing loss

68.3(11.3)

Paper–web/PC

HHIE, IOI–HA, SADL, HADS

0.016

–

0.73

–

3 weeks

No

Tiplady et al. [76]

2010

PDA

C

Y

N

43

Rheumatoid Arthritis

57

Paper–PDA

HAQ–DI, EQ–5D, BPI, MPQ–SF, FACIT–F, SF–36, SARA

0.009

0.88

–

–

1 hour

CI ✓

Turvey et al. [77]

2012

IVRS

W

N

N

51

Mental Health

68(8)

Paper–IVRS

PHQ–9

0.036

0.65

–

–

1 week

No

Vallejo et al. [78]

2007

Web/PC

W

N

N

185

Mental Health

27.4(10.0)

Paper–web/PC

GHQ–28, SCL–90–R

0.020

–

0.69

–

≈17 days

No

Vallejo et al. [79]

2008

Web/PC

C

Y

N

40

Mental Health

22.2

Paper–web/PC

GHQ–28, SCL–90

(Spanish)

–

–

0.84

–

<1 week

No

Varni et al. [80]

2008

Web/PC

C

Y

N

92

Diabetes

13.2(3.42)

Paper–web/PC

PedsQL 4.0

0.007

0.89

  

<5 minutes

No

Vinney et al. [81]

2011

PDA

C

Y

N

19

Speech

9.58(1.22)

Paper–PDA

PedsQL 4.0.

0.008

0.86

–

–

3 weeks

PC ✓

Whitehead et al. [82]

2011

Web/PC

P

Y

N

1034

Mental Health

24.07(8.5)

Paper–web/PC

HADS, SF–36v2, FSI and Fatigue item

0.012

–

–

–

N/A

CI ✓

Wijndaele et al. [83]

2007

Web/PC

W

N

N

130

Mental health

46.5

Paper–web/PC

GHQ–12, SCL–90–R, MOS–SSS, PSS, UCL

–

0.76

–

–

1 week

No

Wu et al. [84]

2009

Web/PC

C

Y

N

34

Heart failure

49(14.2)

Paper–web/PC

KCCQ, MLHFQ, SCHFI

0.027

–

–

–

2 weeks

CI ✓

Young et al. [85]

2009

Web/PC

C

Y

N

69

Child health

11(1.55)

Paper–web/PC

ASK, PedsQL

0.013

0.81

–

–

2 weeks

CI ✓

Yu and Yu, 2007 [86]

2007

Web/PC

P

Y

N

1171

Mental health

 

Paper–web/PC

CES–D Chinese

0.019

–

–

–

N/A

No

Zimmerman & Martinez [87]

2012

Web/PC

W

N

N

53

Mental health

45.1(12.3)

Paper–web/PC

CUDOS

0.009

0.96

–

–

<2 days

No

  1. C crossover, C* 3/4–group crossover, P parallel, W within subjects, D design, R randomisation, AP abstract/poster, K kappa (weighted or unweighted), SMD scaled mean difference (study average), Time lag time between administrations, PC/95 % CIs power calculation or precise 95 % confidence intervals, Dx diseases
  2. ACQ Agoraphobic Cognitions Questionnaire, ACT Asthma Control Test, ALHQ The Attitudes towards Loss of Hearing Questionnaire, AQA Asthma Control Questionnaire, AQLQ(S) Asthma Quality of Life Questionnaire, ASK Activities Scale for Kids, A–QOL Asthma–specific Quality of Life, BAI Beck Anxiety Inventory (BAI), BASDAI Bath Ankylosing Spondylitis Disease Activity Index, BASFI Bath Ankylosing Spondylitis Functional Index, BASG Bath Ankylosing Spondylitis Global Score, BASQoL Ankylosing Spondylitis Quality of Life Instrument, BDI Beck Depression Inventory, BFI Bowel Function Instrument, BPI Brief Pain Inventory, BSQ Body Sensations Questionnaire, C–AC Childhood Asthma Control Test, CES–D Center for Epidemiologic Studies Depression, CHQ Child Health Questionnaire, COPD Chronic Obstructive Pulmonary Disease, COPD–PS COPD Population Screener, CSFQ Changes in Sexual Functioning Questionnaire, CUDOS Clinically Useful Depression Outcome Scale, DAS–A Assessment Scale for Anxiety, DBS 4 item Disease Burden Scale, DLQI Dermatology Life Quality Index, ENSEMBLE MDS a battery of phenotypic patient–reported instruments administered at baseline in clinical studies, EQ–5D EuroQOL–5 Dimensions, EQ VAS EuroQOL Visual Analog Scale, FACT Functional Assessment of Cancer Therapy, FACIT–F The Functional Assessment of Chronic Illness Therapy Fatigue, FFbH Hannover Functional Ability Questionnaire, FSI Fatigue Symptom Inventory, GA Global Assessment, GHQ–12 General Health Questionnaire–12 items, GHQ–28 General Health Questionnaire–28 items, HAQ–II Health Assessment Questionnaire II, HAQ–DI Health Assessment Questionnaire Disability Index, HHIE Hearing Handicap Inventory for the Elderly, howRU a short generic tool for measuring patient–reported outcomes, IBD Irritable Bowel Disease, IBS Irritable Bowel Syndrome, IOI–HA International Outcome Inventory for Hearing Aids, IPSS International Prostate Symptom Score, ISSAC Eight items from the International Study of Asthma and Allergies in Childhood, KCCQ Kansas City Cardiomyopathy Questionnaire, KOOS Knee injury and Osteoarthritis Outcome Score, LCSS Lung Cancer Symptom Scale, LFQ Lung Function Questionnaire, LSHS–R The revised Launay–Slade Hallucination Scale, MADRAS Montgomery Asberg Depression Rating Scale, MADRS–S Montgomery Asberg Depression Rating Scale, m–FSS The modified Fatigue Severity Scale, m–HAQ Modified Health Assessment Questionnaire, MI Mobility Inventory, MLHFQ Minnesota Living with Heart Failure Questionnaire, MOS–SSS Medical Outcomes Study Social Support Scale, MPQ–SF McGill Pain Questionnaire, NMSQ Non–Motor Symptoms Questionnaire; NRS LASA QoL Numerical Rating Scale Linear Analogue Self–Assessment of Quality of Life, OBQ–44 Obsessive Beliefs Questionnaire–44, OCI Obsessive Compulsive Inventory, PCL–C Posttraumatic Stress Disorder Checklist – Civilian Version, PDHCO The Provider–Dependent Health Care Orientation, PDQ–39 Parkinson´s Disease Questionnaire, PedsQL 4.0 Pediatric Quality of Life Inventory, PFDI–20 Pelvic Floor Distress Inventory–20, PFIQ–7 Pelvic Floor Impact Questionnaire–7, PHQ–4 Patient Health Questionnaire, PHQ–9 Patient Health Questionnaire, PIQ The Persecutory Ideation Questionnaire, PISQ–12 Pelvic Organ Prolapse/Urinary Incontinence Sexual Function Questionnaire, PSI Psoriasis Symptom Inventory, PSS Perceived Stress Scale, QLQ–C30 Quality of Life Questionnaire – Cancer 30 items, QOLI Quality Of Life Inventory, QUIDS–SR16 16–item Quick Inventory of Depressive Symptomatology Self–Rated, RMDQ Roland Morris Disability Questionnaire, ROAD Recent–Onset Arthritis Disability questionnaire, RQLQ(S) Rhinoconjunctivitis Quality of Life Questionnaire, SADL Satisfaction with Amplification in Daily Life, SARA Subjects Assessment of Rheumatoid Arthritis, SCHFI Self–Care of Heart Failure Index, SCL–90–R Symptoms Check–List–90–Revised, SDI–21 Social Difficulties Inventory, SF–36 Short Form Survey 36 items, SHIM Sexual Health Inventory for Men, SIBDQ The Short Inflammatory Bowel Disease Questionnaire, TESS Toronto Extremity Salvage Score, SS–5 Perceived Social Support, TESS The Toronto Extremity Salvage Score, TIBI Total Illness Burden Index Truncated Questionnaire, TJC Tender Joint Count, TLEQ Traumatic Life Events Questionnaire, UCLA–A University of California at Los Angeles activity score, UCLA–A Utrecht Coping List, UPDRS II Unified Parkinson's Disease Rating Scale II, UPDRS IV Unified Parkinson's Disease Rating scale IV, VAS Visual Analog Scale, VAS–GH Visual Analog Scale – General Health, WOMAC Western Ontario and McMaster Universities Osteoarthritis Index, WORC The Western Ontario Rotator Cuff Index
  3. aWhen reported as ≥0.85 this is recorded as 0.85