Scolaris Content Display Scolaris Content Display

Flowchart for study selection
Figuras y tablas -
Figure 1

Flowchart for study selection

Risk of bias graph: review authors' judgements about each risk of bias item, presented as percentages across all included studies. The blank spaces represent risk of bias criteria that were not applicable to the study design.
Figuras y tablas -
Figure 2

Risk of bias graph: review authors' judgements about each risk of bias item, presented as percentages across all included studies. The blank spaces represent risk of bias criteria that were not applicable to the study design.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study. The blank cells represent risk of bias criteria that were not applicable to the study design.
Figuras y tablas -
Figure 3

Risk of bias summary: review authors' judgements about each risk of bias item for each included study. The blank cells represent risk of bias criteria that were not applicable to the study design.

Summary of findings for the main comparison. Public reporting of performance data versus no public reporting

People: Insurance plan beneficiaries, birthing mothers, GPs

Settings (countries and clinical settings): United States, Canada, South Korea, Netherlands, China / Community, primary care and hospitals

Intervention: Public release of performance data

Comparison: No public reporting

Outcomes

Impact

No of clinical encounters
(studies)

Certainty of the evidence
(GRADE)*

Changes in healthcare utilisation by consumers

Public release of performance data may make little or no difference to long‐term healthcare utilisation by consumers. However, two studies (one cNRT and one ITS) found that some population subgroups might be influenced by public release of performance data.

18,294 insurance plan beneficiariesa

(3: 1 cRT, 1 cNRT, 1 ITS)

⊕⊕⊝⊝
low

Changes in healthcare decisions taken by healthcare providers (professionals and organisations)

Public release of performance data may make little or no difference to decisions taken by healthcare professionals. Two studies (2 cRTs) found that some decisions might be affected by public release of performance data. One study (ITS) found that decisions might be influenced by the initial release of data, but that subsequent releases might have less impact.

3,000,000 birthsb and 67 healthcare providers (4: 2 RTs, 2 ITS)

⊕⊕⊝⊝
lowc

Changes in the healthcare utilisation decisions of purchasers

No studies reported this outcome.

Changes in provider performance

Public release of performance data may make little or no difference to objective measures of provider performance.

82 healthcare providers

(1 cRT)

⊕⊕⊝⊝
lowd

Changes in patient outcome

Public release of performance data may slightly improve patient outcomes.

315,092 hospitalisations and 7503 healthcare providers (5: 1 RT, 3 ITS, 1 CBA)

⊕⊕⊝⊝
lowe

Adverse effects

No studies reported this outcome.

Impact on equity

Public release of performance data may have a greater effect on provider choice among advantaged populations.

Unknown (1 ITS)

⊕⊕⊝⊝
low

EPOC adapted statements for GRADE Working Group grades of evidence
High‐certainty. This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different is low.
Moderate‐certainty. This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different is moderate.
Low‐certainty. This research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high.
Very low‐certainty. This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different is very high.

Substantially different = a large enough difference that it might affect a decision

a Number was based only on Farley 2002a and Farley 2002b studies, as the total number of cases analysed in Romano 2004 was unclear

b Number of participants in Jang 2011 (3,000,000) estimated from data presented in Chung 2014

c Downgraded one level for inconsistency as effect shown by Zhang 2016, but not IkkersheJang 2011, Ikkersheim 2013, or Flett 201511

d Downgraded two levels for risk of bias, as there was attrition of participating hospitals, evidence of contamination of the intervention across intervention and control hospitals, and blinding was not possible given the nature of the intervention

e Downgraded two levels for inconsistency, as there was marked disagreement between studies, with two showing improvements in patient outcome (LiuTu 2009; Liu 20179), and three showing no such improvements (DeVoRinke 2015; DeVore 2016; Joynt 201615)

cluster‐randomised trial (cRT); cluster‐non‐randomised trial (cNRT); controlled before‐after (CBA) study; interrupted time series (ITS) study; randomised trial (RT)

Figuras y tablas -
Summary of findings for the main comparison. Public reporting of performance data versus no public reporting
Table 1. Summary of included studies

Study detailsa

Improvement by selection

Improvement by changes in care

Data available

Study

Design, setting, and participants

Intervention

Consumers

Providers

Purchasers

Provider performance

Patient outcome

Staff morale

Farley 2002a

cRT; USA; 13,077 insurance plan beneficiaries

Consumer Assessment of Healthcare Providers and Systems (CAHPS) report

X

X

Farley 2002b

cNRT; USA; 5217 insurance plan beneficiaries

Consumer Assessment of Healthcare Providers and Systems (CAHPS) report

X

X

Romano 2004

ITS; USA; ‐

Report cards with risk‐adjusted patient outcomes produced by state agencies

X

b

Flett 2015

ITS; USA; 21 hospitals

State‐based mandatory public reporting of healthcare‐associated infections

X

X

Rinke 2015

CBA; USA; 3207 hospitals

Mandatory public reporting of healthcare‐associated infections

X

X

DeVore 2016

ITS; USA; 315,092 hospitalisations

Online reporting of risk‐adjusted 30‐day re‐admission rates (Hospital Compare)

X

b

Joynt 2016

ITS; USA; 3970 hospitals

Online reporting of risk‐adjusted 30‐day mortality rates (Hospital Compare)

X

X

Liu 2017

ITS; USA; 244 hospitals

Mandatory public reporting of healthcare‐associated infections

X

c

Tu 2009

cRT; Canada; 82 hospital organisations

Report cards with risk‐adjusted patient outcomes and a press conference

X

X

X

Jang 2011

ITS; South Korea; 3,000,000 live births

Repeated public release of information (online, press releases) on hospital caesarean rates

X

X

Ikkersheim 2013

cRT; The Netherlands; 26 general practitioners

Report cards with risk‐adjusted patient outcomes sent to GPs for discussion with patients

X

b

Zhang 2016

cRT; China; 20 primary care providers

Public display of prescription information on outpatient department bulletin boards

X

X

controlled before‐after (CBA) study; cluster‐randomised trial (cRT); cluster‐non‐randomised trial (cNRT); Consumer Assessment of Healthcare Providers and Systems (CAHPs); general practitioners (GPs); interrupted time series (ITS) study

Column headers: changes in healthcare utilisation by consumers (Consumers); changes in healthcare decisions taken by healthcare providers (professionals and organisations; (Providers)); changes in healthcare decisions of purchasers (Purchasers); changes in provider performance (Provider performance); changes in patient outcome (Patient outcome); changes in staff morale (Staff morale); impact on equity (Equity)

Order of studies: listed in chronological order USA, then chronological order for other countries of study

a Studies grouped by intervention, i.e. mode of public release of performance data

b No change in slope and so re‐analysis of the ITS data was uninformative

c Presented derived data (e.g. outputs of regression models) that were insufficient for re‐analysis

Figuras y tablas -
Table 1. Summary of included studies
Table 2. Changes in the healthcare utilisation decisions of consumers

Intervention

Outcome

Study

Type of study

Absolute post‐intervention difference

Absolute pre‐intervention difference

Post‐intervention level in control group

Relative effect

Dissemination of consumer reports directly to consumers

Assigned to high‐rated HMO (2 choices)

Farley 2002a

cRT

1.5

0

15.9

0.0943

Assigned to low‐rated HMO (2 options)

0.4

0

25

0.0160

Assigned to high‐rated HMO (1 option)

1.3

0

29.5

0.0441

Assigned to low‐rated HMO (1 option)

0.1

0

23.7

0.0042

Proportion choosing plan

Farley 2002b

cNRT

0.01

0

0.69

0.0145

cluster‐randomised trial (cRT); cluster‐non‐randomised trial (cNRT); health maintenance organization (HMO)

Figuras y tablas -
Table 2. Changes in the healthcare utilisation decisions of consumers
Table 3. Changes in the healthcare utilisation decisions of healthcare providers (professionals and organisations)

Intervention

Outcome

Study

Type of study

Absolute post‐intervention difference

Absolute pre‐intervention difference

Post‐intervention level in control group

Relative effect

Public reporting of injection prescribing rates in outpatient areas

Average expenditure per prescription

Zhang 2016

cRT

3.4

2.2

41.2

0.0291

Percentage of prescriptions requiring antibiotics

4.6

6.1

62.8

‐0.0249

Percentage of prescriptions requiring combined antibiotics

2.1

4.1

18.6

‐0.1083

Percentage of prescriptions requiring injections

9.0

13.2

64.9

‐0.0643

Average expenditure per prescription

7.2

6.9

44.3

0.0070

Mandatory public reporting of healthcare‐associated infections

Pediatric quality indicator per 1000 eligible discharges

Rinke 2015

CBA

0.6

0.5

1.0

0.1000

Intervention

Outcome

Study

Type of study

Absolute level effect (95% CI)

Relative change at 3 months (95% CI)

Relative change at 6 months (95% CI)

Relative change at 9 months (95% CI)

Relative change at 12 months (95% CI)

Relative change at 24 months (95% CI)

Repeated public release of hospital caesarean section rates

Caeserean section rate

Jang 2011

ITS

‐0.52 (‐0.77 to ‐0.26)

‐0.04 (‐1.23 to 1.18)

‐1.49 (‐2.55 to ‐0.40)

‐2.92 (‐4.50 to 1.30)

‐4.34 (‐6.61 to ‐1.95)

Mandatory public reporting of healthcare‐associated infections

PICU blood cultures

Flett 2015

ITS

7.48 (1.09 to 13.87)

6.21 (‐2.84 to 17.10)

9.90 (‐0.45 to 22.64)

13.87 (1.42 to 29.82)

18.17 (2.90 to 38.77)

22.87 (4.11 to 49.86)

PICU antibiotics

7.29 (4.46 to 10.12)

‐0.11 (‐2.03 to 1.89)

1.61 (‐0.45 to 3.75)

3.36 (0.96 to 5.87)

5.15 (2.26 to 8.20)

6.98 (2.50 to 10.70)

NICU antibiotics

‐5.79 (‐9.17 to ‐2.42)

8.12 (4.11‐12.46)

6.06 (2.08 to 10.35)

4.05 (‐0.35 to 8.85)

1.90 (‐3.17 to 7.53)

‐0.36 (‐6.25 to 6.33)

NICU blood cultures

‐1.14 (‐1.90 to ‐0.39)

2.49 (‐0.51 to 5.67)

1.06 (‐2.07 to 4.39)

‐0.42 (‐3.93 to 3.36)

‐1.95 (‐6.02 to 2.49)

‐3.53 (‐8.26 to 1.72)

cluster‐randomised trial (cRT); controlled before‐after (CBA) study; 95% confidence interval (95% CI); interrupted time series (ITS) study; neonatal intensive care unit (NICU); paediatric intensive care unit (PICU)

Figuras y tablas -
Table 3. Changes in the healthcare utilisation decisions of healthcare providers (professionals and organisations)
Table 4. Changes in provider performance

Intervention

Outcome

Study

Type of study

Absolute post‐intervention difference

Absolute pre‐intervention difference

Postintervention level in control group

Relative effect

Public release of a range of quality indicators

All AMI processes

Tu 2009

cRT

2.0

0.9

65.6

0.0168

Use of standard admission orders

6.1

0.7

72.5

0.0745

Left ventricular function assessment

2.9

6.3

49.8

‐0.0683

Lipid test < 24 hours arrival

3.8

1.6

51.1

0.0431

Fibrinolytics < 30 mins after arrival

2.6

3.1

45.7

‐0.0109

Fibrinolytics decided by ED physician

2.0

4.4

84.3

‐0.0285

Fibrinolytics prior to transfer to CCU

3.8

2.9

95.7

0.0094

Aspirin < 6 hours arrival

5.5

3.1

82.6

0.0291

B blockers < 12 hours arrival

2.4

3.9

73.7

‐0.0204

Aspirin at discharge

0.9

0.0

84.0

0.0107

B blockers at discharge

0.6

0.0

85.6

0.0070

ACEi, ARB for LV dysfunction

4.7

3.4

81.7

0.0159

Statin at discharge

0.3

0.2

85.5

0.0012

All CHF processes

1.0

3.0

54.6

‐0.0366

LVF assessment

2.7

4.5

55.2

‐0.0326

Daily weights recorded

1.3

0.3

24.0

0.0417

Counselling on > 1 aspect of CHF

0.9

1.7

55.3

‐0.0145

ACEi, ARB for LV dysfunction

6.3

1.7

92.4

0.0498

B blockers for LV dysfunction

4.0

1.7

71.7

0.0321

Warfarin for AF

0.6

3.1

64.2

‐0.0389

atrial fibrillation (AF); acute myocardial infarction (AMI); angiotensin‐converting enzyme inhibitor (ACEi); angiotensin‐2 receptor blockers (ARB); beta‐adrenergic blocking agents (B blockers); cluster‐randomised trial (cRT); coronary care unit (CCU); congestive heart failure (CHF); emergency department (ED); left ventricular (LV); left ventricular failure (LVF); minutes (mins)

Figuras y tablas -
Table 4. Changes in provider performance
Table 5. Changes in patient outcome

Intervention

Outcome

Study

Type of study

Absolute postintervention difference

Absolute pre‐intervention difference

Postintervention level in control group

Relative effect

Public release of a range of quality indicators

AMI 30‐day mortality

Tu 2009

cRT

2.4

0.5

9.8

0.1939

AMI 1‐year mortality

3.1

1

19.4

0.1082

STEMI 30‐day mortality

3.1

0.4

8.3

0.3253

STEMI 1‐year mortality

3.9

1.2

13.5

0.2000

NSTEMI 30‐day mortality

2.3

0.3

10.5

0.1905

NSTEMI 1‐year mortality

3

0.9

22.6

0.0929

CHF 30‐day mortality

1

0.9

9.6

0.0104

CHF 1‐year mortality

2.6

0.6

30.3

0.0660

CHF and LV dysfunction 30‐day mortality

0.9

0.6

8.5

0.0353

CHF and LV dysfunction 1‐year mortality

6.3

1.8

26.3

0.1711

Mandatory reporting of healthcare‐associated infections

Pediatric quality indicator per 1000 eligible discharges

Rinke 2015

CBA

0.6

0.5

1

0.1000

Intervention

Outcome

Study*

Type of study

Absolute level effect (95% CI)

Relative change at 4 months (95% CI)

Relative change at 8 months (95% CI)

Relative change at 12 months (95% CI)

Relative change at 24 months (95% CI)

Hospital quality process and outcome metrics reported on a public website

30‐day risk‐adjusted mortality

Joynt 2016

ITS

0.12 (0.03 to 0.21)

1.57 (‐4.28 to 8.18)

‐2.47 (‐8.20 to 4.03)

3.71 (‐3.25 to 11.74)

7.18 (‐1.91 to 18.13)

Public reporting of risk‐standardised hospital re‐admission rates

30‐day re‐admission (AMI)

DeVore 2016

ITS

0.00 (0.00 to 0.00)

‐2.04 (‐8.56 to 5.48)

‐1.36 (‐7.92 to 6.20)

‐0.69 (‐7.34 to 7.00)

0.72 (‐6.32 to 8.90)

30‐day re‐admission (heart failure)

0.00 (0.00 to 0.00)

‐1.39 (‐4.17 to 1.56)

‐1.84 (‐4.59 to 1.08)

‐1.88 (‐4.68 to 1.10)

‐2.78 (‐6.42 to 1.15)

30‐day re‐admission (pneumonia)

0.00 (0.00 to 0.00)

‐4.44 (‐13.61 to 6.91)

‐5.07 (‐14.17 to 6.20)

‐5.69 (‐14.71 to 5.47)

‐7.45 (‐18.10 to 6.37)

30‐day re‐admission (COPD)

0.00 (0.00 to 0.00)

‐6.66 (‐11.42 to ‐1.37)

‐0.76 (‐6.11 to 5.23)

‐7.64 (‐12.31 to ‐2.44)

‐9.06 (‐13.62 to ‐4.00)

30‐day re‐admission (diabetes)

0.00 (‐0.00 to 0.01)

‐0.65 (‐13.66 to 16.96)

0.00 (‐13.13 to 17.81)

0.65 (‐12.44 to 18.35)

1.98 (‐13.57 to 24.36)

30‐day mortality (AMI)

0.00 (0.00 to 0.00)

34.38 (2.71 to 94.32)

35.83 (2.79 to 100.17)

37.38 (2.88 to 106.67)

43.06 (3.20 to 133.08)

30‐day mortality (heart failure)

0.00 (0.00 to 0.00)

6.04 (‐5.86 to 21.37)

13.78 (‐0.56 to 32.94)

9.98 (‐3.46 to 27.77)

13.31 (‐0.54 to 31.64)

30‐day mortality (pneumonia)

0.00 (0.00 to 0.00)

‐3.96 (‐23.10 to 27.85)

‐3.72 (‐16.70 to 14.05)

2.94 (‐18.04 to 19.00)

‐3.84 (‐22.51 to 26.69)

30‐day mortality (COPD)

0.00 (0.00 to 0.00)

20.89 (5.51 to 41.52)

21.63 (5.68 to 43.24)

20.99 (5.54 to 41.75)

22.00 (5.77 to 44.13)

30‐day mortality (diabetes)

0.00 (0.00 to 0.00)

‐14.73 (‐34.83 to 23.29)

‐15.10 (‐35.48 to 24.12)

‐14.78 (‐34.92 to 23.40)

‐19.39 (‐42.65 to 35.66)

Acute Myocardial Infarction (AMI); ST‐Elevation Myocardial Infarction (STEMI); Non‐ST‐Elevation Myocardial Infarction (NSTEMI); Congestive Heart Failure (CHF); Left Ventricular (LV); Chronic Obstructive Pulmonary Disease (COPD); Cluster Randomised Trial (cRT); Controlled Before‐After (CBA) study; Interrupted Time Series (ITS) study; 95% Confidence Interval (95% CI)

* Joynt 2016 and DeVore 2016 provided outcomes in quarters rather than months and so have been presented as 4‐ and 8‐months rather than the pre‐specified 3‐ and 6‐months.

Figuras y tablas -
Table 5. Changes in patient outcome