Scolaris Content Display Scolaris Content Display

改善住院病患抗生素處方的介入措施

Contraer todo Desplegar todo

Background

Antibiotic resistance is a major public health problem. Infections caused by multidrug‐resistant bacteria are associated with prolonged hospital stay and death compared with infections caused by susceptible bacteria. Appropriate antibiotic use in hospitals should ensure effective treatment of patients with infection and reduce unnecessary prescriptions. We updated this systematic review to evaluate the impact of interventions to improve antibiotic prescribing to hospital inpatients.

Objectives

To estimate the effectiveness and safety of interventions to improve antibiotic prescribing to hospital inpatients and to investigate the effect of two intervention functions: restriction and enablement.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) (the Cochrane Library), MEDLINE, and Embase. We searched for additional studies using the bibliographies of included articles and personal files. The last search from which records were evaluated and any studies identified incorporated into the review was January 2015.

Selection criteria

We included randomised controlled trials (RCTs) and non‐randomised studies (NRS). We included three non‐randomised study designs to measure behavioural and clinical outcomes and analyse variation in the effects: non‐ randomised trials (NRT), controlled before‐after (CBA) studies and interrupted time series (ITS) studies. For this update we also included three additional NRS designs (case control, cohort, and qualitative studies) to identify unintended consequences. Interventions included any professional or structural interventions as defined by the Cochrane Effective Practice and Organisation of Care Group. We defined restriction as 'using rules to reduce the opportunity to engage in the target behaviour (or increase the target behaviour by reducing the opportunity to engage in competing behaviours)'. We defined enablement as 'increasing means/reducing barriers to increase capability or opportunity'. The main comparison was between intervention and no intervention.

Data collection and analysis

Two review authors extracted data and assessed study risk of bias. We performed meta‐analysis and meta‐regression of RCTs and meta‐regression of ITS studies. We classified behaviour change functions for all interventions in the review, including those studies in the previously published versions. We analysed dichotomous data with a risk difference (RD). We assessed certainty of evidence with GRADE criteria.

Main results

This review includes 221 studies (58 RCTs, and 163 NRS). Most studies were from North America (96) or Europe (87). The remaining studies were from Asia (19), South America (8), Australia (8), and the East Asia (3). Although 62% of RCTs were at a high risk of bias, the results for the main review outcomes were similar when we restricted the analysis to studies at low risk of bias.

More hospital inpatients were treated according to antibiotic prescribing policy with the intervention compared with no intervention based on 29 RCTs of predominantly enablement interventions (RD 15%, 95% confidence interval (CI) 14% to 16%; 23,394 participants; high‐certainty evidence). This represents an increase from 43% to 58% .There were high levels of heterogeneity of effect size but the direction consistently favoured intervention.

The duration of antibiotic treatment decreased by 1.95 days (95% CI 2.22 to 1.67; 14 RCTs; 3318 participants; high‐certainty evidence) from 11.0 days. Information from non‐randomised studies showed interventions to be associated with improvement in prescribing according to antibiotic policy in routine clinical practice, with 70% of interventions being hospital‐wide compared with 31% for RCTs. The risk of death was similar between intervention and control groups (11% in both arms), indicating that antibiotic use can likely be reduced without adversely affecting mortality (RD 0%, 95% CI ‐1% to 0%; 28 RCTs; 15,827 participants; moderate‐certainty evidence). Antibiotic stewardship interventions probably reduce length of stay by 1.12 days (95% CI 0.7 to 1.54 days; 15 RCTs; 3834 participants; moderate‐certainty evidence). One RCT and six NRS raised concerns that restrictive interventions may lead to delay in treatment and negative professional culture because of breakdown in communication and trust between infection specialists and clinical teams (low‐certainty evidence).

Both enablement and restriction were independently associated with increased compliance with antibiotic policies, and enablement enhanced the effect of restrictive interventions (high‐certainty evidence). Enabling interventions that included feedback were probably more effective than those that did not (moderate‐certainty evidence).

There was very low‐certainty evidence about the effect of the interventions on reducing Clostridium difficile infections (median ‐48.6%, interquartile range ‐80.7% to ‐19.2%; 7 studies). This was also the case for resistant gram‐negative bacteria (median ‐12.9%, interquartile range ‐35.3% to 25.2%; 11 studies) and resistant gram‐positive bacteria (median ‐19.3%, interquartile range ‐50.1% to +23.1%; 9 studies). There was too much variance in microbial outcomes to reliably assess the effect of change in antibiotic use.

Heterogeneity of intervention effect on prescribing outcomes

We analysed effect modifiers in 29 RCTs and 91 ITS studies. Enablement and restriction were independently associated with a larger effect size (high‐certainty evidence). Feedback was included in 4 (17%) of 23 RCTs and 20 (47%) of 43 ITS studies of enabling interventions and was associated with greater intervention effect. Enablement was included in 13 (45%) of 29 ITS studies with restrictive interventions and enhanced intervention effect.

Authors' conclusions

We found high‐certainty evidence that interventions are effective in increasing compliance with antibiotic policy and reducing duration of antibiotic treatment. Lower use of antibiotics probably does not increase mortality and likely reduces length of stay. Additional trials comparing antibiotic stewardship with no intervention are unlikely to change our conclusions. Enablement consistently increased the effect of interventions, including those with a restrictive component. Although feedback further increased intervention effect, it was used in only a minority of enabling interventions. Interventions were successful in safely reducing unnecessary antibiotic use in hospitals, despite the fact that the majority did not use the most effective behaviour change techniques. Consequently, effective dissemination of our findings could have considerable health service and policy impact. Future research should instead focus on targeting treatment and assessing other measures of patient safety, assess different stewardship interventions, and explore the barriers and facilitators to implementation. More research is required on unintended consequences of restrictive interventions.

PICO

Population
Intervention
Comparison
Outcome

El uso y la enseñanza del modelo PICO están muy extendidos en el ámbito de la atención sanitaria basada en la evidencia para formular preguntas y estrategias de búsqueda y para caracterizar estudios o metanálisis clínicos. PICO son las siglas en inglés de cuatro posibles componentes de una pregunta de investigación: paciente, población o problema; intervención; comparación; desenlace (outcome).

Para saber más sobre el uso del modelo PICO, puede consultar el Manual Cochrane.

改善醫院醫師開立抗生素處方的方式

文獻回顧目標

此篇考科藍文獻回顧的目標,在於探討改善醫院醫師開立抗生素處方的方法。我們收集並分析所有相關的研究來回答這個問題,共找到了221個研究。

主要訊息

使用抗生素政策改善了處方開立方式並縮短了抗生素的治療期間。

改善醫師開立抗生素處方的介入措施,令受試者住院的時間縮短了1.12天(根據15個研究的結果),也沒有增加死亡的風險(根據29個研究結果)。提供建議或回饋給醫師的介入措施,和不提供這些資訊給醫師相比,更能有效的改善實際處方。而由7個研究的證據顯示,通過使用規範使醫師正確開立處方的介入措施,會使得治療延遲,並使感染專家和臨床團隊之間的信任受到破壞。

本文獻回顧研究了什麼?

抗生素用於治療細菌感染,例如肺炎。隨著時間過去,許多細菌對抗生素產生了抗藥性。抗生素的抗藥性對病患及健康照護體系都是非常嚴重的問題。因為有抗藥性的細菌會導致較高的死亡率及較長的住院期。細菌抗藥性時常因為在不需使用抗生素的時候使用抗生素而產生。研究顯示,大約在一半的例子中,醫院的醫師並沒有正確的開立抗生素。

我們調查了能幫助醫師正確開立抗生素的介入措施之有效性及安全性,以及何種改變行為的技術能夠影響介入措施的成功。

主要結果

我們發現了221個相關研究。其中96個研究出自於北美,其餘125個研究出自於歐洲(87個)、亞洲(19個)、南美(8個)、澳洲(8個)以及東亞(3個)。研究測試的介入措施大致分成2類:限制型技術,即實施規範使醫師正確開立處方;以及支持型技術,提供建議或回饋,以幫助醫師正確開立處方。

我們發現高度可信的證據,介入措施使更多住院病患根據抗生素處方政策接受適當的治療。中度可信的證據顯示,在不增加病患死亡數的情形下,介入措施縮短了住院時間。限制型及支持型的技術兩者都成功有效地介入。我們不需要更多研究來回答這些介入措施是否減少非必要抗生素的使用,但我們還需要做更多研究以了解使用限制性介入措施的非預期後果。

儘管事實上大多數人沒有廣泛地採取行改變為的技術—稽核與提供實施績效回饋,介入措施仍成功安全地減少了醫院內非必要抗生素的使用。有效傳播文獻回顧的結果將可能對健康服務及政策產生相當大的影響。

本文獻回顧資料更新日期為何?

我們搜尋至2015年1月發表的研究。

Authors' conclusions

disponible en

Implications for practice

Reducing antimicrobial resistance and hospital‐associated infection is a public health priority. Our review shows that antimicrobial stewardship interventions can safely reduce unnecessary antibiotic use in hospitals, despite the fact that the majority of interventions did not use the most effective behaviour change techniques. Consequently, effective dissemination of the review results could have considerable health service and policy impact through greater use of interventions that enhance enablement.

The randomised controlled trials provided high‐certainty evidence that the interventions we have assessed are effective in increasing compliance with guidelines to reduce unnecessary treatment without increasing the risk of mortality. Furthermore, the interventions were associated with reduction in length of stay. The evidence from this review should inform implementation decisions regarding antimicrobial stewardship interventions in hospitals.

In randomised controlled trials and interrupted time series studies, enablement consistently increased the effectiveness of interventions, including restrictive interventions; however, feedback was used in only a minority of enablement interventions, and very few included goal setting or action planning. Antimicrobial management teams might consider using evidence about effective feedback from other clinical settings (Ivers 2012). Training in the design and reporting of behaviour change interventions should be a priority for antimicrobial management teams.

Implications for research

Given the high certainty of evidence for our primary outcome, we believe that additional trials comparing antibiotic stewardship with no intervention are unlikely to change our conclusions or build on our understanding of the current evidence. Future research should instead focus on measuring clinical outcomes and assessing other measures of patient safety and different stewardship interventions and explore the barriers and facilitators to implementation.

We included 163 NRS but only 11 of these were about unintended consequences. Moreover only one NRS used qualitative methods, which are likely to be required in addition to survey methods for the investigation of unanticipated consequences (Rogers 1995). Future research should make greater use of qualitative methods for investigation of consequences of interventions, for example in process evaluation alongside clinical trials (Grant 2013). Anticipated, undesirable consequences should be regarded as trade‐offs which may need to be accepted in exchange for a greater good (Ash 2007). Future research should examine how decisions are made about the acceptability of trade‐offs (SISCC 2016). The purpose, design, and use of balancing measures in quality and safety improvement has been identified as a priority for research on methods in improvement science (SISCC 2016). Antimicrobial stewardship is an important topic for further research because of the clear competing risks of excessive use of antibiotics and delayed or ineffective treatment of life threatening infection.

Antibiotic stewardship requires clinicians to change their infection control behaviours. Given that the extent to which current antibiotic stewardship programs have incorporated insights and approaches from behavioural science is limited, there is an urgent need to bring together key stakeholders in the design and delivery of stewardship programmes and research experts in improvement and social sciences to develop more impactful stewardship programmes. We propose three key questions, which a Transnational Working Group within the Joint Programming Initiative in Antimicrobial Resistance will address in 2017 (JPIAMR 2016):

  1. What behaviour change approaches can be recommended now to optimise hospital stewardship programmes?

  2. How can hospital stewardship programmes be designed to maximise implementation across countries?

  3. What is the research agenda to optimise efficient implementation of antibiotic stewardship programmes worldwide?

We were unable to perform reliable evidence synthesis on the relationship between prescribing and microbial outcomes with segmented regression analysis of interrupted time series studies from single hospitals. There is an urgent need for co‐ordinated, multicentre research studies.

We found consistent evidence of reduced length of stay as an unanticipated beneficial consequence of interventions that targeted either choice of antibiotic or duration of antibiotic treatment. Further research is required to understand the mechanism for this effect.

Summary of findings

Open in table viewer
Summary of findings for the main comparison. Effects of interventions to improve use of antibiotics on prescribing, clinical outcomes, adverse events, and effect modifiers (heterogeneity)

Patient or population: adults or children undergoing inpatient antibiotic prophylaxis or treatment

Settings: mainly high‐income countries (North America or Western Europe)

Intervention: any intervention targeting healthcare professionals that aimed to improve antibiotic prescribing to hospital inpatients

Comparison: usual care (varied across studies)

Effectiveness: prescribing outcomes from RCTs

Outcomes

Absolute effect*

No of participants

(No of studies)

Certainty of the evidence (GRADE)

Comments

Without intervention

With intervention

Proportion of participants who were treated according to antibiotic prescribing guidelines

Follow‐up to end of study

43 per 100

58 per 100

23,394 participants

(29 RCTs)

⊕⊕⊕⊕
High

We have graded the certainty of evidence as high because heterogeneity was explained by prespecified effect modifiers (see below). The intervention effect varied between the studies, but the direction of effect was consistent. Restricting the analysis to studies at low risk of bias gave a similar result (RD 11%, 95% CI 10% to 12%).

Difference: 15 more participants per 100 (95% CI 15 to 23) received appropriate treatment following intervention.

Duration of all antibiotic treatment

11.0 days

9.1 days

3318 participants

(14 RCTs)

⊕⊕⊕⊕
High

Difference: 1.95 fewer days per participant (95% CI 2.22 to 1.67)

Mortality

Follow‐up to end of study

11 per 100

11 per 100

15,827 participants

28 (RCTs)

⊕⊕⊕⊝1
Moderate

Mortality and length of stay were measured to determine the impact of reduced antibiotic use on clinical outcomes. The results were similar for studies that targeted antibiotic choice or exposure.

Only 1 of the interventions in the RCTs with mortality or length‐of‐stay outcomes had a restrictive component (Singh 2000). This evidence is therefore at high risk of indirectness because 7 studies in the next section of the table (see below) raise concerns about the safety of restrictive interventions. Moreover, the ITS studies showed that restrictive components were included in 42 (34%) of 123 hospital interventions.

Difference: 0 more deaths per 100 participants (95% CI 1 to 0 fewer)

Mean length of hospital stay per participant

12.9 days

11.8 days

3834 participants

15 (RCTs)

⊕⊕⊕⊝1
Moderate

Difference: 1.1 fewer days per participant (95% CI 1.5 to 0.7 fewer)

Delay in treatment

Restrictive interventions increased the risk of delay in all 3 studies. The risk to patients resulted in termination of the RCT by the Trial Monitoring Committee.

1 RCT, 2 cohort

⊕⊕⊝⊝2
Low

The evidence from these 7 studies of unintended consequences raises concerns about the directness of the evidence of safety from the 29 RCTs in the previous section of the table (see above).

Negative professional culture

Loss of trust in infection specialists because of failure to record approvals for restricted drugs or provide warning about stopping treatment

Misleading or inaccurate information from prescribers in order to meet criteria for restricted drugs. In 1 hospital, misdiagnosis of hospital‐acquired infection was large enough to trigger an outbreak investigation.

1 case control, 2 cohort, 1 qualitative

⊕⊕⊖⊖3
Low

Effect modifiers (heterogeneity) for immediate effect of intervention on prescribing outcomes:
impact of behaviour change functions (enablementor restriction) and additional impact of feedback, RCTs and ITS studies. A positive value for Beta means the modifier is associated with increased effect

Effect modifier

Adjusted effect in meta‐regression
Beta
(95% CI)

Number of studies

Certainty of the evidence (GRADE)

Comments

Enablement

15.12

(8.45 to 21.8)

29 RCTs

⊕⊕⊕⊕
High

The effect of enablement and restriction is similar in the RCTs and ITS studies. Of the 29 RCTs, only 8 (31%) of interventions were hospital‐wide, the majority being in single units. In contrast, 64 (70%) of the interventions in ITS studies were hospital‐wide.

12.86

(4.11 to 21.6)

91 ITS

Restriction

34.91

(13.52 to 56.29)

29 RCTs

⊕⊕⊕⊕
High

24.69

(13.74 to 35.64)

91 ITS

Addition of feedback to enablement

10.88

(7.16 to 19.32)

23 RCTs

⊕⊕⊕⊝2
Moderate

Feedback was included in 4 (17%) of 23 RCTs and 20 (47%) of 43 ITS studies with interventions that included enablement. There were not enough interventions with goal setting and action planning to analyse as effect modifiers.

15.63

(0.56 to 30.70)

43 ITS

Addition of enablement to restriction

38.36

(18.94 to 57.78)

29 ITS

⊕⊕⊖⊖3
Low

Enablement was included in 13 (45%) of 29 ITS studies with restrictive interventions.

*The risk WITHOUT the intervention is based on the median control group risk across studies. The corresponding risk WITH the intervention (and the 95% confidence interval for the difference) is based on the overall relative effect (and its 95% confidence interval).
CI: confidence interval; ITS: interrupted time series; RCT: randomised controlled trial; RD: risk difference

GRADE Working Group grades of evidence
High certainty: Further research is very unlikely to change our confidence in the estimate of effect.
Moderate certainty: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.
Low certainty: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.
Very low certainty: We are very uncertain about the estimate.

Details of five GRADE criteria for all outcomes from RCTs are in Appendix 2.

1We downgraded the evidence to moderate because of indirectness.
2We downgraded the evidence because most studies are non‐randomised studies.
3We graded the evidence as low because it is all from non‐randomised studies.
4We graded the evidence as very low because it is all from non‐randomised studies and there was too much heterogeneity for reliable evidence synthesis.

Background

disponible en

Description of the condition

Antibiotic resistance is a major public health problem. In comparison with infections caused by susceptible bacteria, those caused by multidrug‐resistant bacteria are associated with higher incidences of mortality and prolonged hospital stay (de Kraker 2011). Clostridium difficile infection (CDI) is another manifestation of the collateral damage caused by antimicrobial prescribing (Davey 2010). Such infections are also associated with increased costs resulting from the need to use more expensive antibiotics, prolonged hospital stay (the principal contributor), and expenses related to screening and surveillance, eradication regimens, and consumables (the gloves, gowns, and aprons used to prevent cross‐infection) (de Kraker 2011). The UK 5 Year Antimicrobial Resistance Strategy 2013 to 2018 recognises the importance of reducing inappropriate antibiotic prescribing (Department of Health 2013), the implication being that antibiotic resistance is largely a consequence of the selective pressures of antibiotic usage, and that reducing these pressures by the judicious administration of antibiotics will facilitate a return of susceptible bacteria or, at least, will prevent or slow the pace of the emergence of resistant strains.

At the same time, sepsis is a major cause of avoidable mortality in hospitals, with an estimated 100,000 cases per year in the UK alone (NCEPOD 2015).

Description of the intervention

We included any intervention to improve antibiotic prescribing to hospital inpatients. Antibiotic stewardship has two aims: first, to ensure effective treatment of patients with infection, and second, to minimise collateral damage from antimicrobial use (Davey 2010). Hence the UK Department of Health's Guidance on Antimicrobial Stewardship emphasises the need for urgent treatment of serious infections in addition to minimising unnecessary use of antibiotics (Department of Health 2013). We compared interventions to change professional behaviour with standard practice (no intervention). We classified interventions by their intervention function (Michie 2011). The previous version of this review suggested that restrictive interventions had greater immediate effect on prescribing than interventions that used education or persuasion (Davey 2013). For this update, we identified interventions that were designed to increase enablement, defined as 'increasing means/reducing barriers to increase capability or opportunity' (Michie 2011).

How the intervention might work

In this update of the review we used new data extraction sheets to classify the intervention functions and to identify the behaviour change functions that are used in antimicrobial stewardship interventions (Michie 2013). In particular, we assessed the relative effectiveness of interventions according to how they used enablement and restriction to change behaviour (Michie 2011). We divided the interventions into four groups: enablement without restriction; restriction without enablement; both enablement and restriction; and neither enablement nor restriction.

Why it is important to do this review

This review is an update of Davey 2005 and Davey 2013. It complements a review of interventions to improve prescribing of antibiotics to patients in ambulatory care (Arnold 2005).

Objectives

disponible en

To estimate the effectiveness and safety of interventions to improve antibiotic prescribing to hospital inpatients and investigate the effect of two intervention functions: restriction and enablement.

Methods

disponible en

Criteria for considering studies for this review

Types of studies

We included randomised controlled trials (RCTs) and non‐randomised studies (NRS). We included three NRS study designs to measure behavioural and clinical outcomes and analyse variation in the effects: non‐randomised trials (NRT), controlled before‐after (CBA) studies and interrupted time series (ITS) studies. We used Cochrane Effective Practice and Organisation of Care (EPOC) Group eligibility guidance for CBAs and NRTs (EPOC 2016). In addition, for the assessment of unintended consequences, we included three additional NRS designs (case control, cohort, and qualitative studies) to identify additional evidence about long‐term effects and harms of interventions in order to enhance the directness of evidence from RCTs (Schünemann 2013).

Types of participants

Healthcare professionals who prescribe antibiotics to hospital inpatients receiving acute care (including elective inpatient surgery). We excluded interventions targeted at residents in nursing homes or other long‐term healthcare settings.

Types of interventions

We included interventions relevant to improving antibiotic prescribing as outlined in the EPOC taxonomy (EPOC 2015).

  1. Audit and feedback defined as any summary of clinical performance of health care over a specified period of time.

  2. Education through meetings or distribution of educational materials.

  3. Educational outreach through academic detailing or review of individual patients with recommendation for change.

  4. Reminders provided verbally, on paper, in the workplace environment (e.g. posters or messages printed on equipment) or on computer.

  5. Structural: the influence on antibiotic prescribing of changing from paper to computerised records and of the introduction of new technology for rapid microbiology testing or measurement of inflammatory markers.

In addition, we included the following restrictive interventions: selective reporting of laboratory susceptibilities; formulary restriction; requiring prior authorisation (expert approval) therapeutic substitution; and automatic stop orders.

Enabling interventions were: audit and feedback; educational outreach through review of individual patients with recommendation for change; and circumstantial reminders that were targeted at doctors who were managing specific patients (Table 1). We classified reminders in the form of posters or pocket cards summarising antibiotic policies as environmental restructuring but not as enabling (Table 1). Terms used to describe interventions are described in more detail in the Data extraction and management section.

Open in table viewer
Table 1. Definition of behaviour change techniques and intervention functions

Intervention Function

Definition

Intervention components

Education

Increasing knowledge or understanding

Educational meetings;

Dissemination of educational materials;

Educational outreach

Persuasion

Using communication to induce positive or negative feelings or to stimulate action

Educational outreach by academic detailing or review and recommend change

Restriction

Using rules to reduce the opportunity to engage in the target behaviour (or increase the target behaviour by reducing the opportunity to engage in competing behaviours)

Restrictive

Environmental restructuring

Changing the physical context

Reminders (physical) such as posters, pocket‐size or credit card‐size summaries or on laboratory test reports;

Structural (e.g. new laboratory tests or rapid reporting of results)

Enablement

Increasing means/reducing barriers to increase capability or opportunity

Audit and feedback;

Decision support through computerised systems or through circumstantial reminders that were triggered by actions or events related to the targeted behaviour;

Educational outreach by review and recommend change

We did not consider studies that compared the effectiveness of antibiotic treatments (e.g. intravenous versus oral administration of antibiotics) as eligible for this review.

Types of outcome measures

Primary outcomes

The effect of interventions on antibiotic prescribing measured as either compliance with antibiotic guidelines or policies, the duration of antibiotic treatment, decision to treat, or total duration of treatment. We included studies without reliable or adequate information addressing the primary outcome measure, but we did not use these studies in data synthesis.

Secondary outcomes

Mortality, length of stay, or other clinical outcomes (e.g. surgical‐site infection or acute kidney injury), microbial outcomes (CDI, colonisation or infection with antimicrobial‐resistant bacteria), unintended‐consequences measures (e.g. a delay in start of antibiotic treatment, a change in threshold for diagnosis of hospital‐acquired infection to justify existing prescribing practice). Note that clinical outcomes could be indicators of improved clinical outcomes associated with interventions to increase effective antibiotic treatment, or unintended consequences (e.g. to provide evidence about the safety of interventions to reduce unnecessary antibiotic treatment).

Search methods for identification of studies

Electronic searches

We searched the Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE) for related systematic reviews and the following databases for primary studies without language, publication year, or publication status restrictions in January 2015.

Databases

  • Cochrane Central Register of Controlled Trials (CENTRAL; 2015, Issue 1) in the Cochrane Library (searched 22 January 2015)

  • MEDLINE (1946 to 19 January 2015) (OvidSP)

  • Embase (1947 to 22 January 2015) (OvidSP)

The MEDLINE search strategy was developed by the Cochrane EPOC Group Information Specialist in consultation with the review authors and translated for use in other databases employing appropriate syntax and vocabulary. Results were limited by two methodological filters: the Cochrane Highly Sensitive Search Strategy (sensitivity‐ and precision‐maximising version, 2008 revision) to identify randomised trials (Higgins 2011), and a Cochrane EPOC Group study design filter to identify NRS. Full search strategies are provided in Appendix 1.

Searching other resources

We searched for additional studies using the bibliographies of included articles, personal files, and by contacting experts in the field regarding any unpublished work.

Data collection and analysis

Selection of studies

Two review authors (EB and PD) independently reviewed citations and abstracts retrieved in the search to identify all reports that included original data about interventions to change antibiotic prescribing. If either review author had doubts about eligibility, then both review authors reviewed the full papers. The review authors were not blinded to study author or location. We resolved disagreements by discussion and consensus.

We excluded studies that had no relevant and interpretable data presented or obtainable. We defined 'relevant data' as an intervention that included a change in antibiotic treatment for hospital inpatients and where at least one of the study's reported outcomes was directly attributable to change in antibiotic treatment. We defined 'interpretable data' as follows: CBA, NRT, or RCT designs had to include sufficient data to estimate effect size as change in at least one relevant outcome after the intervention. Interrupted time series studies had to include a clearly defined intervention point.

We did not exclude studies due to high risk of bias.

Data extraction and management

Working in pairs, five review authors (PD, CM, CS, EC, KM) independently performed data abstraction using data extraction sheets including information on: study design, type of intervention (intervention components and functions), presence of controls, type of targeted behaviour, participants, setting, methods (unit of allocation, unit of analysis, study power, methodological risk of bias, consumer involvement), outcomes, and results.

Explanation of terms used to describe interventions
Restriction

We defined restriction as 'using rules to reduce the opportunity to engage in the target behaviour (or increase the target behaviour by reducing the opportunity to engage in competing behaviours)'.

Enablement

We defined enablement as 'increasing means/reducing barriers to increase capability or opportunity'.

Goal setting

We documented the specific prescribing behaviour that was targeted by the intervention (e.g. switch participants from parenteral to oral antibiotics) and how this was incorporated into an aim for the intervention. Was the aim simply a directional change of the target behaviour (e.g. increase or decrease behaviour?), or did the intervention include a specific threshold to be reached (e.g. target behaviour performed more than 95% of the time) or the duration within which the target had to be achieved (e.g. more than 95% reliability within six months)? If the study reported a power calculation, we did not accept this as evidence of a specific threshold unless it was clearly communicated to the professionals who were the targets of the intervention. For example, a power calculation showing that the study could detect a 10% improvement in the targeted behaviour would have to be accompanied by some explicit statement about the intervention aim being at least 10% improvement.

Feedback

We classified interventions as including feedback only if they provided a "summary of clinical performance of healthcare over a specified period of time" (EPOC 2015). We found that some studies did not meet this definition, even though they described their intervention as including feedback in the title (e.g. Elligsen 2012 and Newland 2012) or in the methods (e.g. Palmay 2014). The intervention in these studies was educational outreach by review and recommended change, so the feedback was limited to the individual participants who were reviewed with no feedback about the treatment of other participants over time. In contrast, Buising 2008a is an example of an intervention in which "a formal feedback was provided to units regarding their compliance with the approval system over time" in addition to review and recommend change for individual participants. For studies that met our definition of feedback, we recorded frequency, format (verbal, written, or both) and whether it was delivered by a colleague, supervisor, or somebody external to the clinical team.

Action planning

We documented whether there was a reward for meeting a target, which could be material or social reward (either from self or others) and the use of action plans if the target was not met. Our definition of an action plan was: prompt, detailed planning of performance of the behaviour, which had to include at least one of context, frequency, duration, or intensity. If there was evidence of action planning, we recorded to whom the action plan was tailored (e.g. individual participant or group) and whether participants were involved in developing the action plan.

Intervention components and functions

In the Characteristics of included studies we have listed the intervention components (Types of interventions) and the intervention functions (Michie 2011; Michie 2013). Note that each intervention component may have more than one intervention function. We have presented definitions of intervention functions and their relationship to intervention components in Table 1.

Assessment of the impact of interventions

We have used meta‐analysis to assess the impact of RCTs of interventions and meta‐regression to understand variation in effect estimates for RCTs and ITS studies.

Assessment of risk of bias in included studies

We applied the 2013 EPOC 'Risk of bias' criteria to all papers in the review, including articles in the 2003 review (EPOC 2013). We scored each study for risk of bias as 'low' if all criteria were scored as 'low', 'medium' if one or two criteria were scored as 'unclear' or 'high', and 'high' if more than two criteria were scored as 'unclear' or 'high'.

We applied three additional criteria to studies with microbial outcomes, based on the ORION statement: Guidelines for transparent reporting of outbreak reports and intervention studies of nosocomial infection (Orion Statement; Stone 2007).

  1. Case definition: score as 'low' if there is a clear definition either of infection or of colonisation and there were no major changes in laboratory diagnostic methods during the study period.

  2. Planned intervention: score as 'low' if the intervention was planned to reduce endemic rates of colonisation or infection and was not implemented in response to an outbreak. Regression to the mean following an outbreak is an important risk of bias for estimates of the effect of interventions in ITS studies of infection (Davey‐Smith 2001; Stone 2007).

  3. Other infection control measures: score as 'low' if infection control practices (hand hygiene, gowning, or other personal protection) and isolation or cohorting policies are described and there were no changes coincident with the intervention to change antibiotic prescribing.

We have presented microbial 'Risk of bias' results in the Notes section of the Characteristics of included studies. We have not included them in the 'Risk of bias' tables unless there might also be a risk to prescribing outcomes (e.g. appointment of additional infection control practitioners who might have influenced prescribing).

We assessed risk of bias in case control or cohort studies of unintended consequences with ROBINS‐I: a tool for assessing Risk of Bias in Non‐randomised Studies of Interventions (Sterne 2016). We have reported these 'Risk of bias' assessments in the Notes section of the Characteristics of included studies.

Measures of treatment effect

We assessed the impact of interventions on clinical outcome for studies that provided reliable data about mortality, length of hospital stay, or other clinical outcomes such as acute kidney injury. We did not include clinical outcomes for studies that estimated the impact of their intervention based on modelling (Barlow 2007). We analysed dichotomous data (such as increase in desired practice and mortality) as risk differences and analysed continuous data (such as length of hospital stay) as mean differences.

We critically examined the methods of analysis of ITS data. The preferred method is a statistical comparison of time trends before and after the intervention. If the original paper did not include an analysis of this type, we extracted the data presented in tables or graphs in the original paper and used them to perform new analyses where possible. We used segmented time series regression analysis to estimate the effect of the intervention whilst taking account of time trend and autocorrelation among the observations. We obtained estimates for regression coefficients corresponding to two standardised effect sizes for each study: a change in level and a change in trend before and after the intervention. A change in level was defined as the difference between the observed level at the first intervention time point and that predicted by the pre‐intervention time trend. A change in trend was defined as the difference between post‐ and pre‐intervention slopes (Ramsay 2003). We evaluated the direct effect of the intervention using results reported one month after the start of the intervention. We also reported the level effects at six months, and yearly thereafter when possible. We standardised the results of some ITS studies so that they were on the same scale (per cent change in outcome), thereby facilitating comparisons of different interventions. To do this, we used the change in level and change in slope to estimate the effect size with increasing time after the intervention (one month, six months, one year, etc.) as the per cent change in level at each time point. We did not extrapolate beyond the end of data collection after the intervention. We anticipated that the eligible studies would exhibit significant heterogeneity, due to variations in target clinical behaviours, patient and provider populations, methodological features, characteristics of the interventions, and the contexts in which the interventions were delivered. To address the source of variation in results due to the use of enabling or restrictive interventions, we undertook a random‐effects meta‐regression analysis on study‐level summary effect size at each time point.

We assessed the impact of interventions on microbial outcomes if the study provided reliable data about colonisation or infection with Clostridium difficile or with antibiotic‐resistant bacteria. We did not include microbial outcomes for studies that estimated the future impact of their intervention based on modelling (Paul 2006), or that used clinical definitions of infection that did not distinguish between resistant and sensitive bacteria (Micek 2004; Singh 2000).

Unit of analysis issues

If an RCT did not take into account the effect of clustering in the analysis, we stated this in the 'Risk of bias' assessment. We incorporated consideration of unit of analysis issues as part of the sensitivity analyses.

We estimated intracluster correlation (ICC) for each outcome. The ICCs used reflect that process measures usually have higher ICC than outcome measures and were obtained from the database of ICCs held by the Health Services Research Unit, University of Aberdeen (Health Services Research Unit 2016).

  • Prescribing 0.2

  • Mortality 0.01

  • Length of stay 0.2

Average cluster size (m) = (total number of participants (intervention + control)) ≑ (total number of clusters). Inflation factor = 1 + (m‐1) x ICC. For dichotomous outcomes, we divided events and participants by the inflation factor for intervention and control groups. For continuous outcomes, we multiplied intervention and control standard deviation by the inflation factor.

Dealing with missing data

We have not attempted to account for missing data in the meta‐analysis of RCTs or meta‐regression of ITS studies. For ITS studies, we only analysed effects at a specified time point when data were available, we have not carried forward regression lines beyond the last observation or used regression lines to estimate missing data..

Assessment of heterogeneity

We quantified heterogeneity among studies using the I2 statistic and Cochran's Q test (Cochran 1954). The I2 statistic quantifies the percentage of the total variation across studies that is due to heterogeneity rather than chance (Higgins 2003); smaller percentages suggest less observed heterogeneity.

Assessment of reporting biases

We assessed publication and selective reporting bias.

Data synthesis

We have analysed the results for RCTs, CBAs, NRT, and ITS studies separately. For the RCT data, we employed a standard meta‐analysis approach using Review Manager 5 for binary (e.g. compliance with guidelines) and continuous (e.g. duration of treatment) outcomes. We analysed the data with a fixed‐effect model (Review Manager 5).

We used Stata 14 for all statistical re‐analyses and meta‐regressions (Stata 2015), and Review Manager 5 for all data synthesis (Review Manager 5).

Subgroup analysis and investigation of heterogeneity

We used meta‐regression to investigate potential effect modifiers. In meta‐regression, the outcome variable is the effect estimate (e.g. a mean difference or a risk difference). The explanatory variables are characteristics of studies that might influence the size of intervention effect (Higgins 2011).

We prespecified four subgroups as explanatory variables for the meta‐regression (Davey 2014):

  1. interventions that included enablement versus those that did not;

  2. interventions that included restriction versus those that did not;

  3. enabling interventions that included feedback versus those that did not;

  4. feedback interventions that included goal setting or action planning versus those that did not.

Definitions of these terms can be found in Data extraction and management and Table 1. We expected restriction, enablement, feedback goal setting and action planning to be associated with increased effectiveness of interventions (Ivers 2012).

We included the following three additional variables in the meta‐regression because they might influence the size of intervention effect and explain heterogeneity.

  1. Target: choice of antibiotic regimen versus time to first antibiotic dose or exposure to antibiotics, effects possibly greater for interventions targeting choice.

  2. Setting: single unit versus multiple wards, effects possibly greater in single unit.

  3. Intent: increase effective versus decrease excessive, effects possibly greater with increase effective.

The meta‐regression was performed using standard weighted (by standard error of estimate) linear regression (Higgins 2011).

Sensitivity analysis

We conducted sensitivity analyses by re‐analysing data to investigate the effect of two risks of bias.

  1. Lack of adjustment for the effect of clustering in cluster RCTs. We repeated all analyses that included cluster RCTs with adjusted numbers of events and total participants for dichotomous variables and adjusted standard deviation for continuous variables (Analysis 1.2; Analysis 1.5; Analysis 2.2; Analysis 2.5).

  2. Overall high risk of bias. We analysed all studies at medium and low risk of bias separately in sensitivity analyses (Analysis 1.3; Analysis 1.6; Analysis 2.3; Analysis 2.6).

Summary of findings

We summarised the findings of the main intervention comparison for the most important outcomes in summary of findings Table for the main comparison. Two review authors independently assessed the certainty of the evidence for each key outcome (high, moderate, low, and very low) using the five GRADE considerations (study limitations, consistency of effect, imprecision, indirectness, and publication bias) (Guyatt 2011). We assessed the following outcomes:

  1. compliance with desired practice;

  2. duration of antibiotic treatment;

  3. mortality;

  4. length of hospital stay;

  5. delay in treatment;

  6. negative professional culture.

We also assessed the evidence from the meta‐regression in terms of the extent to which we believed it helped explain variation of effect. We included the following effect modifiers in our analysis.

  1. Enablement (Yes/No)

  2. Restriction (Yes/No)

  3. Addition of feedback to enablement (Yes/No)

  4. Addition of enablement to restriction (Yes/No)

We used the methods and recommendations described in Section 8.5 and Chapter 12 of the Cochrane Handbook for Systematic Reviews of Interventions,Higgins 2011, and the EPOC worksheets (EPOC 2013a). Disagreements on certainty ratings were resolved by discussion, and justification for decisions to down‐ or upgrade the ratings are provided in footnotes in the table and comments made to aid readers' understanding of the review where necessary. We used plain language statements to report these findings in the review. Further details about each of the five GRADE criteria are in Appendix 2.

Evidence from randomised studies started at high certainty and was downgraded according to the five considerations described above. Evidence from non‐randomised studies started at low certainty and was assessed against the same five criteria. We only considered upgrading for non‐randomised evidence in the presence of a large treatment effect, dose response, or where plausible confounding would have reduced the observed effect.

Results

Description of studies

Results of the search

The combined results of all literature searches are described in the study flow diagram (Figure 1).


Figure 1 Study flow diagram.

Figure 1 Study flow diagram.

Included studies

The Characteristics of included studies table lists 221 studies, of which 211 used the following designs to evaluate the intended effect of interventions: 138 ITS studies, 58 RCTs (14 cluster RCTs), 6 CBAs, and 8 NRTs. The remaining 11 studies were designed to identify unintended consequences of interventions and used the following designs: 8 cohort (Connor 2007; Duvoisin 2014; Friedberg 2009; Kanwar 2007; LaRosa 2007; Linkin 2007; Welker 2008; Winters 2010), 1 case control (Calfee 2003), and 1 qualitative (semi‐structured interviews) (Baysari 2013) and 1 ITS (Bell 2014).

Geographical location of study

Ninety‐six studies were from North America. The remaining 125 were from Europe (87, includes Israel), Asia (19), South America (8), Australia (8), and East Asia (3). The number of studies by country (including the countries in four multinational studies) is: Argentina, 1; Australia, 9; Austria, 2; Belgium, 4; Brazil, 4; Canada, 8; China, 6; Colombia, 2; Croatia, 1; Denmark, 3; France, 11; Germany, 12; Greece, 1; Hong Kong, 1; Hungary, 1; India, 1; Indonesia, 1; Israel, 1; Italy, 3; Japan, 1; Korea, 3; Lebanon, 1; Mexico, 1; Netherlands, 11; Norway, 1; Serbia, 1; Singapore, 1; Spain, 5; Sweden, 2; Switzerland, 11; Taiwan, 3; Thailand, 4; Turkey, 1; UK, 22; USA, 89.

Number of hospitals

A total of 178 (79%) studies were conducted in one hospital, 9 studies in 2 hospitals, 18 studies in 3 to 9 hospitals, and 16 studies in 10 or more hospitals.

Deliverer of intervention

Of the 221 interventions, 112 (51%) were designed and delivered by a multidisciplinary team, 54 (24%) by specialist physicians (infectious diseases or microbiology), 35 (16%) by department physicians (e.g. emergency department or critical care), and 20 (9%) by pharmacists.

Funding

Five studies received some funding from manufacturers of drugs or laboratory tests. The remaining 216 studies were funded by government agencies or the participating hospitals. Details are provided in the Characteristics of included studies table.

Power calculations

Details of power calculations are provided in Appendix 3

Excluded studies

We excluded 32 unique studies from the review because they did not contain relevant or interpretable data (Selection of studies). For details of each study, see Characteristics of excluded studies.

Risk of bias in included studies

All 14 CBAs and NRTs were at high risk of bias (Figure 2). High risk of bias was more common in RCTs (36/58, 62%) than in ITS studies (20/138, 14%) (Figure 2). All 51 studies at low risk of bias were ITS studies (Figure 2). Among RCTs, high risk of bias was much more likely in studies with two or fewer hospitals (31/36, 86%) versus three or more hospitals (11/22, 50%). Of the 11 RCTs with two or fewer hospitals with medium risk of bias, nine interventions were circumstantial reminders targeted at doctors who were managing specific patients (Christ‐Crain 2004; Christ‐Crain 2006; Esposito 2011; Kerremans 2009; Lacroix 2014; Lesprit 2013; Long 2014; Senn 2004; Stocker 2010; Strom 2010), so the risks of allocation or contamination bias were relatively low compared with the other RCTs of interventions in one or two hospitals. However, the remaining two RCTs at low risk of bias show that these risks can be minimised for RCTs of review and recommend change interventions in single hospitals (Lesprit 2013; Palmay 2014).


Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.Blank sections in this graph are due to use of different ROB criteria for CBA, NRT and RCT versus ITS studies

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Blank sections in this graph are due to use of different ROB criteria for CBA, NRT and RCT versus ITS studies

We have presented 'Risk of bias' criteria for the case control and cohort studies of unintended consequences in the Notes section in Characteristics of included studies.. For the nine studies, we assessed the risk of bias as high in two (Calfee 2003; Friedberg 2009), medium in two (Linkin 2007; Welker 2008), and low in five (Connor 2007; Duvoisin 2014; Kanwar 2007; LaRosa 2007; Winters 2010).

Allocation

Most of the RCTs had high risk of selection bias because of problems with concealment of allocation (Figure 2). The RCTs with low risk of selection bias were either cluster RCTs or interventions with circumstantial reminders, for which concealment of allocation is relatively straightforward.

Blinding

Most of the RCTs also had high risk of performance and detection bias because RCTs in single hospitals were often single‐blind and it was difficult to conceal the allocation of participants in these trials (Figure 2).

Incomplete outcome data

The RCTs used data collected specifically for the trial, and all provided convincing evidence about lack of attrition bias. Most of the ITS studies used data from routine systems for prescribing (pharmacy) and microbial (microbiology) outcomes; we assessed these sources as having low risk of attrition bias (Figure 2). Examples of high risk of attrition bias in routine data are changes in the number of participants who did not have serum creatinine measure preoperatively during the study period, which may have biased ascertainment of postoperative kidney injury (Bell 2014), and use of surveillance data about surgical‐site infection that did not include information about infections arising after discharge from hospital (Dua 2014).

Selective reporting

We also assessed routine data systems as being at low risk of reporting bias (Figure 2). Most of the ITS studies used computerised pharmacy systems to measure drug consumption.

Other potential sources of bias

Less than 25% of RCTs provided clear information about baseline outcome; most of these were cluster RCTs (Figure 2). The most common single risk of bias for ITS studies was that the intervention was not independent of other changes (Figure 2). For ITS studies, the main risks of bias were that there were insufficient data to account for seasonal variation or that one or more of the microbial 'Risk of bias' criteria were present (Figure 2).

Effects of interventions

See: Summary of findings for the main comparison Effects of interventions to improve use of antibiotics on prescribing, clinical outcomes, adverse events, and effect modifiers (heterogeneity)

Studies included in evidence synthesis and 'Summary of findings' tables

Outcomes from 49 (84%) of the 58 RCTs and110 (80%) of the 138 ITS studies were used in at least one meta‐analysis or meta‐regression or are summarised in text or Additional tables. The contribution that each RCT made can be found in Appendix 4. One ITS study contributed data about unintended consequences (Bell 2014). The contribution of 109 ITS studies to meta‐regression of prescribing outcomes is summarised in Appendix 5. Reasons for exclusion of 10 RCTs and 28 ITS studies from evidence synthesis can be found in Appendix 6.

The 10 case control, cohort, or qualitative studies of unintended consequences all contributed evidence about adverse effects.

None of the 6 CBAs or 8 NRTs included evidence about adverse effects of interventions, and there were not enough studies for evidence synthesis.

Intended prescribing outcomes for RCTs and ITS studies included in evidence synthesis

Interventions were targeted at antibiotic treatment for 46 (94%) of 49 RCTs and 101 (92%) of 110 ITS studies. The remaining 11 studies targeted surgical antibiotic prophylaxis (Bell 2014; Dull 2008; Gulmezoglu 2007; Kritchevsky 2008; Meyer 2010; Perez 2003; Schwann 2011; Sun 2011; Van Kasteren 2005; Wax 2007; Weinberg 2001).

For the 148 interventions targeted at antibiotic treatment, the intended outcome of 137 (93%) interventions was to decrease excessive use of antibiotics: 45/46 (98%) RCTs and 93/102 (91%) ITS studies. The only RCT that was primarily intended to increase effective treatment targeted dosing of gentamicin (Burton 1991). Two RCTs with antibiotic choice as the primary outcome did include time to first antibiotic dose for participants with community‐acquired pneumonia as a secondary outcome (Schouten 2007; Yealy 2005). The only other evidence about increasing effective treatment of sepsis came from six ITS studies that aimed to reduce time to first antibiotic dose (Barlow 2007; Hitti 2012; Jobson 2015; Marwick 2013; Volpe 2012; Weiner 2009).

In contrast, reduction in excessive use of antibiotics was the intended outcome of only 3 (25%) of the 12 interventions targeted at surgical antibiotic prophylaxis (Bell 2014; Sun 2011; Van Kasteren 2005). The remaining nine interventions were all intended to increase effective use of antibiotics by increasing the number of participants who received prophylaxis or reducing the time to first antibiotic dose.

Effectiveness and adverse effects of interventions

Effectiveness of interventions in RCTs

Interventions were associated with an increase in compliance with desired practice by 15% (95% confidence interval (CI) 14% to 16%) in 29 RCTs (Analysis 1.1; Figure 3). We obtained similar results in sensitivity analyses for unit of analysis errors (Analysis 1.2) or risk of bias (Analysis 1.3). Interventions were associated with a reduction in duration of total antibiotic treatment by ‐1.95 days (95% CI ‐2.22 to ‐1.67) in 14 RCTs (Analysis 1.4; Figure 4). We obtained similar results in sensitivity analyses for unit of analysis errors (Analysis 1.5) or risk of bias (Analysis 1.6).


Forest plot of comparison: 1 Prescribing: RCTs of all interventions to reduce unnecessary prescribing, outcome: 1.1 Dichotomous outcomes, increase in desired practice.

Forest plot of comparison: 1 Prescribing: RCTs of all interventions to reduce unnecessary prescribing, outcome: 1.1 Dichotomous outcomes, increase in desired practice.


Forest plot of comparison: 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 1.4 Continuous outcomes, duration of all antibiotic treatment (days).

Forest plot of comparison: 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 1.4 Continuous outcomes, duration of all antibiotic treatment (days).

In four RCTs the prescribing outcome was the consumption of targeted antibiotics measured in different units (cost, days, or defined daily dose), so results were expressed as standardised mean reduction (Analysis 1.7.).

Adverse effects of interventions
Evidence from RCTs

Interventions were not associated with any increase in mortality (95% CI 1 to 0 fewer deaths per 100 participants) in 28 RCTs (Analysis 2.1; Figure 5). We obtained similar results in sensitivity analyses for unit of analysis errors (Analysis 2.2) or risk of bias (Analysis 2.3). Interventions were associated with reduction in length of stay by ‐1.12 days (95% CI ‐1.54 to ‐0.70) in 15 RCTs Analysis 2.4; Figure 6). We obtained similar results in sensitivity analyses for unit of analysis errors (Analysis 2.5) or risk of bias (Analysis 2.6). We found no evidence of a difference in results for interventions that targeted antibiotic exposure (decision to treat or duration of all antibiotic treatment) versus the choice of antibiotic prescribed (Analysis 3.1; Analysis 3.2; Analysis 4.1; Analysis 4.2).


Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.1 Mortality, all RCTs.

Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.1 Mortality, all RCTs.


Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.4 Length of stay, all RCTs.

Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.4 Length of stay, all RCTs.

One RCT measured clinical outcome as potentially harmful delay in essential treatment (Strom 2010). The outcome was ascertained by the Trial Monitoring Committee, who stopped the trial prematurely when four participants were found to have potentially harmful delay in treatment with trimethoprim‐sulphamethoxazole or warfarin. This was a restrictive intervention intended to prevent interactions between these drugs.

Evidence from NRS

ITS studies

Clinical outcome data were measured as mortality in four ITS studies (Table 2) and length of stay in one ITS study (Table 3). However, we could only calculate 95% CI for three of these studies (Lee 2014; Popovski 2015; Skaer 1993), and the outcome data came from all participants in the hospital rather than just the participants who were the targets of the interventions.

Open in table viewer
Table 2. Unintended consequences of ITS studies: mortality*

Study

Prescribing target

Restriction

Design of analysis

Effect estimate

95% CI

Lee 2014

Choice of drug

No

Cohort

Incidence rate ratio 1.1

0.9 to 1.5

Popovski 2015

Choice of drug

No

Cohort

Increase by 1.4%

‐1.2% to 4.1%

Wang 2014

Choice of drug

Yes

ITS, segmented regression

Change in slope ‐0.0172

No data

Yoon 2014

Choice of drug

Yes

Cohort

+0.43 per 1000 OBD

No data

*Mortality was measured in all patients in the hospital rather than just those patients who were the targets of the interventions.

CI: confidence interval
ITS: interrupted time series
OBD: occupied bed day

Open in table viewer
Table 3. Unintended consequences of ITS studies: length of stay*

Study

Prescribing target

Restrictive

Design of analysis

Effect estimate

95% CI

Mittal 2014

Exposure, % treated

No

Cohort

‐0.5 days

No data

Skaer 1993

Choice of drug

No

Cohort

‐0.1 days

‐0.49 to +0.29

*Length of stay was measured in all patients in the hospital rather than just those patients who were the targets of the interventions.

CI: confidence interval
ITS: interrupted time series

Three ITS studies reported other clinical outcomes that provided more direct evidence about unintended consequences of the interventions (Table 4). An intervention to promote gentamicin for prophylaxis was intended to reduce risk of CDI but was associated with a large increase in acute kidney injury in the participants undergoing target operations, and as a consequence the antibiotic policy change was reversed (Bell 2014). An intervention designed to shorten time to first antibiotic dose for people with sepsis was not associated with any increase in the time left without being seen for all other participants in the emergency department (Volpe 2012). An intervention to reduce the duration of surgical antibiotic prophylaxis was not associated with increased surgical‐site infection (Van Kasteren 2005).

Open in table viewer
Table 4. Unintended consequences of ITS studies: other

Study

Prescribing target

Design of analysis

Effect measure

Effect estimate

95% CI

Bell 2014

Antibiotic choice

ITS, segmented regression

Risk of postoperative acute kidney injury

Increase 98%

93.8% to 94.2%

Van Kasteren 2005

Exposure, duration

Cohort

Surgical‐site infection

Decrease 0.8%

‐2.2% to 0.6%

Volpe 2012

Time to first antibiotic dose

Cohort

Left without being seen rate

Decrease 0.4%

No data

CI: confidence interval
ITS: interrupted time series

Case control, cohort and qualitative studies

Ten studies investigated unintended consequences of interventions to change antibiotic choice with cohort (n = 8), case control (n = 1), or qualitative case study (n = 1) designs (Table 5).

Open in table viewer
Table 5. Unintended consequences studies (case control, cohort, or qualitative)

Study

Design

Patients

Intended target

Unintended consequence

Effect estimate

95% CI

Interventions with a restrictive component

Baysari 2013

Qualitative

36 physicians

Reduce unnecessary use of restricted antibiotics

Inaccurate feedback

Not quantified; qualitative study

Calfee 2003

Case control

Not clear

Increase in physician‐based diagnosis of nosocomial infection

No denominator data

Connor 2007

Cohort

120

Failure to warn prescribers about discontinuation

Duvoisin 2014

Cohort

222

Reduce unnecessary laboratory tests

Delay in TFAD (HR > 1 shows delay less likely in intervention period)

Multivariate HR 1.56

1.17 to 2.07

LaRosa 2007

Cross‐sectional

15,440

Reduce unnecessary use of restricted antibiotics

Orders for restricted antibiotics (% all orders) from 10 to 11 pm vs all other hours

Cohort

360

% appropriate orders 10 to 11 pm vs 9 to 10 pm

‐23.7%

‐31.8% to ‐15.5%

Linkin 2007

Cohort

200

Risk of inaccurate information in orders judged inappropriate vs appropriate

OR 2.2

1.0 to 4.4

Winters 2010

Cohort

3251

Risk of 1‐hour delay in TFAD

OR 1.5

1.2 to 1.8

Risk of 2‐hour delay in TFAD

OR 1.8

1.4 to 2.2

Interventions with no restrictive component

Friedberg 2009

Cohort

13,042

Reduce time to first antibiotic dose for patients with community‐acquired pneumonia

% CAP diagnoses

1% increase

No denominator data

Kanwar 2007

Cohort

518

% correct CAP diagnoses

‐7.9% decrease

‐15.4% to ‐0.4%

Welker 2008

Cohort

548

% correct CAP diagnoses

‐16.0% decrease

‐7.6% to ‐24.4%

CAP: community‐acquired pneumonia
CI: confidence interval
HR: hazard ratio
OR: odds ratio
TFAD: time to the first antibiotic dose

There was a restrictive component to the intervention in seven studies. One study showed that restriction of laboratory tests of inflammation (C‐reactive protein and white blood cell count) was not associated with an increase in time to first antibiotic dose (Duvoisin 2014). The remaining six studies all revealed unintended consequences of interventions that restricted antibiotic choice by requiring prior approval, as follows.

In three studies (Friedberg 2009; Kanwar 2007; Welker 2008), the intervention was a national financial incentive in the USA that was intended to reduce time to first antibiotic dose for people admitted to hospital with community‐acquired pneumonia (CAP). In all three studies, the unintended consequence was misdiagnosis of pneumonia, which could lead to an increase in unnecessary antibiotic treatment. In two single‐centre studies, there was a decrease in the percentage of participants with correct diagnosis of CAP based on prespecified criteria (Kanwar 2007; Welker 2008). In contrast, a large, multicentre study reported no evidence of an overall increase in the diagnosis of CAP (Friedberg 2009); however, this study was at high risk of bias.

Explaining heterogeneity in the intended effect of interventions

Meta‐regresson of RCTs

We performed meta‐regression on 29 RCTs with dichotomous prescribing outcomes (Analysis 1.1; Figure 3). Outcomes for all of these trials could be expressed as number of participants where treatment was compliant with policy divided by total participants. We did not perform meta‐regression on 15 RCTs with continuous prescribing outcomes because the outcomes were heterogeneous (Analysis 1.4; Analysis 1.7) and because none of the interventions included restriction or feedback, and only two did not include enablement (Danaher 2009; Kerremans 2008).

Meta‐regression results for 29 RCTs with dichotomous outcomes

In the meta‐regression, enablement, restriction, targeting antibiotic choice versus exposure and high risk of bias were significantly associated with greater intervention effect in univariate analysis, and they all remained significant in multivariate analysis (Figure 7).


Meta‐regression by effect modifier for 29 RCTs. A positive value for Beta indicates enhanced intervention effect. One RCT had both enabling and restrictive components in the intervention (Strom 2010).

Meta‐regression by effect modifier for 29 RCTs. A positive value for Beta indicates enhanced intervention effect. One RCT had both enabling and restrictive components in the intervention (Strom 2010).

Of the 23 RCTs of enabling interventions, four also included feedback (Camins 2009; Schnoor 2010; Schouten 2007; Yealy 2005). All four of these RCTs targeted antibiotic choice, so we have compared their effects with seven RCTs of enabling interventions without feedback that also targeted antibiotic choice. The mean risk difference for interventions with feedback was 19% (95% CI 16% to 22%) (Figure 8) compared with 13% (95% CI 9% to 17%) (Figure 9) for interventions with no feedback. Only two of the feedback RCTs also included action planning (Schouten 2007; Yealy 2005).


Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.1 Enablement plus feedback.

Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.1 Enablement plus feedback.


Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.2 Enablement without feedback.

Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.2 Enablement without feedback.

Meta‐regression of ITS studies
Do interventions that involve enablement have greater initial effect?

There were 107 ITS studies with data that could be used for meta‐regression of prescribing outcomes at one, six, or 12 months' postintervention. We used multivariable meta‐regression to identify effect modifiers in 91 ITS studies including data about prescribing at six months' postintervention. As with the RCTs (Figure 7), both enablement and restriction were independently associated with increased effect in ITS studies (Figure 10). Of 29 ITS studies with restrictive interventions, 13 (45%) also had enablement, and this independently enhanced intervention effect (Figure 11). In comparison with interventions targeting antibiotic exposure, those targeting choice were associated with greater effect in RCTs (Figure 7), but not in ITS studies (Figure 10). The number of studies in each category only allowed analysis of the effects of setting in ITS studies (Figure 10), and intention could only be included in meta‐regression of ITS studies of enabling intervention (Figure 12). The limited evidence suggests that intention and setting were not effect modifiers (Figure 7; Figure 10).


Meta‐regression by effect modifiers of intervention for 91 ITS studies. Outcome is effect on prescribing six months' postintervention. There are 16 studies with both enabling and restricting intervention components ().

Meta‐regression by effect modifiers of intervention for 91 ITS studies. Outcome is effect on prescribing six months' postintervention. There are 16 studies with both enabling and restricting intervention components (Figure 11).


Meta‐regression of prescribing outcome by effect modifiers for 29 ITS studies of interventions that included restriction.

Meta‐regression of prescribing outcome by effect modifiers for 29 ITS studies of interventions that included restriction.


Meta‐regression by effect modifier for 43 ITS studies of interventions that included enablement but not restriction. Outcome is effect on prescribing six months' postintervention. Note that four studies with feedback were not included in this analysis because they also included restriction.

Meta‐regression by effect modifier for 43 ITS studies of interventions that included enablement but not restriction. Outcome is effect on prescribing six months' postintervention. Note that four studies with feedback were not included in this analysis because they also included restriction.

Are interventions that include feedback more effective than those that do not?

Feedback was included in 4 (17%) of 23 RCTs (Figure 8) and 20 (47%) of 43 ITS studies (Figure 12) of enabling interventions that did not include restriction. The intervention was audit and feedback alone in three RCTs and 10 ITS studies. In one RCT and 11 ITS studies, audit and feedback was combined with review and recommend change or circumstantial reminders. Interventions that included feedback were more effective than those that did not. However, there were too few studies with goal setting or action planning to assess their effect in addition to feedback.

There were only two ITS studies with enough data to analyse the effect of adding an additional component to an effective intervention. However, the second intervention component did not include goal setting, feedback, or action planning in either study (Mol 2005; Po 2012)

Summary of interventions for the studies included in meta‐regression

In comparison with RCTs, the ITS studies were more likely to have multiple intervention components: 35 (38%) of 91 ITS studies versus 5 (17%) of 29 RCTs, odds ratio 3.00 (95% CI 1.05 to 8.59) (Table 6). There were also differences in the components for enabling interventions (review and recommend change was included in 53% of ITS studies versus 25% of RCTs) and restrictive interventions (removal of target drugs from clinical areas was included in 34% of ITS studies but in no RCTs) (Table 6). Educational meetings or distribution of educational materials was the most common intervention in studies that did include enablement or restriction (75% of RCTs and 89% of ITS studies) (Table 6).

Open in table viewer
Table 6. Summary of intervention components for 29 RCTs (Analysis 1.1; Figures 3 and 7) and 91 ITS studies (Figure 10)

Intervention function and components

RCT

ITS

Enablement

24

studies

59

studies

Number of enabling or restrictive intervention components

27

76

Studies with > 1 Enabling intervention component

2

8%*

19

32%*

Audit and feedback

4

17%

24

41%

Computerised decision support

1

4%

3

5%

Circumstantial reminders

16

67%

18

31%

Review and recommend change

6

25%

31

53%

Restriction

2

studies

29

studies

Number of Restrictive intervention components

3

41

Studies with > 1 Restrictive intervention component

1

50%

10

34%

Expert approval

1

50%

18

62%

Compulsory order form

1

50%

7

24%

Removal

0

10

34%

Review and make change

1

50%

6

21%

No Enablement or Restriction

4

studies

18

studies

Number of intervention components

6

25

Studies with > 1 intervention component

2

50%

6

33%

Educational materials or meetings

3

75%

16

89%

Educational outreach (academic detailing)

1

25%

6

33%

Physical reminders

1

25%

2

11%

Structural intervention

1

25%

1

6%

*The denominator for all percentages is the number of studies for each intervention function. One RCT, Strom 2010, and 16 ITS studies (Figure 11) included both enabling and restrictive intervention components.

ITS: interrupted time series
RCT: randomised controlled trial

Sustainability of intervention effect

Sustainability was assessed in 64 of 91 ITS studies, with prescribing outcome data at both 6 and 12 months' postintervention. Intervention effect was sustained at 12 months' postintervention in 55 (86%) of these studies (95% CI 77% to 94%). There were 13 interventions with neither enablement nor restriction; intervention effect was sustained in 11 (85%) (95% CI 65% to 100%). Consequently, it was unlikely that either enablement or restriction would be associated with greater sustainability. However, the results suggest that restrictive interventions were less likely to have sustained effect if they did not include enablement: 5/8 (62%) versus 12/13 (92%) with enablement, risk difference 30% (95% CI ‐7% to 66%).

Five ITS studies with data about removal of interventions provided additional information about sustainability of interventions (Table 7). Three of these studies also provided data about the effect of the intervention. The intended effect of all interventions was decrease in the use of target antibiotics. Removal of the intervention was associated with increase in the use of target antibiotics in all five studies and, with one exception (Kim 2008), the 95% CI for effect size did not include decrease in use of target antibiotics. Kim 2008 was the only one of these five interventions including enablement by audit and feedback.

Open in table viewer
Table 7. Data from 5 studies about the effect of removal of interventions. The intended effect of all interventions was reduction in unnecessary antibiotic use

Study

Intervention function

Intervention effect (95% CI)

Time intervention was in place

Effect of removal (95% CI)

Kallen 2009

Restriction

‐87.5%

‐115.4 to ‐59.7

6 months

398.9%

238.2 to 559.5

Kim 2008

Restriction

‐23.1%

‐53.7 to +7.4

9 months

6.0%

‐23.4 to 35.4

Standiford 2012

Enablement

‐28.6%

‐46.5 to ‐10.6

7 years

31.0%

6.8 to 55.3

Himmelberg 1991

Restriction

No data

“long‐standing”

301.2%

230.9 to 371.5

Skrlin 2011

Restriction

2 years

255.8%

194.7 to 316.9

CI: confidence interval

Microbial outcomes (antibiotic resistance and CDI)

There were 1 CBA and 5 RCTs with microbial outcome data, and these were too heterogeneous for data synthesis (Table 8).

Open in table viewer
Table 8. Randomised controlled trials with microbial outcomes

Study

Design

Microbial outcome

Reason not in meta‐analysis

Annane 2013

RCT

Colonisation with MRSA (nasal swab) and GNRB (rectal swabs)

Not comparable with any other RCT

Bouza 2007

RCT

Number of cases of Clostridium difficile

Not in prescribing meta‐analysis

Lesprit 2013

RCT

Secondary infection and/or colonisation with multidrug‐resistant bacteria in the 6 months following randomisation

Not in prescribing meta‐analysis. It is impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics.

Palmay 2014

RCT

CDI and infection with antibiotic resistant organisms cases/1000 OBD

Not in prescribing meta‐analysis

Singh 2000

RCT

Number of participants with "antimicrobial resistance and/or superinfections" from randomisation until discharge from hospital

It is impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics.

CDI: Clostridium difficile infection
GNRB: gram‐negative resistant bacteria
MRSA: methicillin‐resistant Staphylococcus aureus
OBD: occupied bed day
RCT: randomised controlled trial

We performed meta‐regression on 26 ITS studies including reliable data about prescribing outcomes at 6 months and microbial outcomes at 12 months after the intervention (Table 9). Six unplanned interventions (in response to outbreaks) were associated with markedly greater effect on microbial outcomes (Figure 13). When studies were ranked in descending order of effect size for microbial outcome at 12 months, the top five studies were all unplanned interventions (Kim 2008; May 2000; McNulty 1997; Tangdén 2011; Valiquette 2007), with the remaining unplanned intervention ranking 9th (Lautenbach 2003).


Meta‐regression by effect modifiers for 34 microbial outcomes 12 months' postintervention from 26 ITS studies. The bars show the results for unadjusted versus adjusted analyses, the comparison for unplanned interventions is with planned interventions in both the unadjusted and adjusted analysis.CDI: Clostridium difficile infection
 GPC: infection with antibiotic‐resistant gram‐positive cocci
 GNB: infection with antibiotic‐resistant gram‐negative bacteriaOther infection control: 'Yes' means there were changes to infection control processes during the study period.

Meta‐regression by effect modifiers for 34 microbial outcomes 12 months' postintervention from 26 ITS studies. The bars show the results for unadjusted versus adjusted analyses, the comparison for unplanned interventions is with planned interventions in both the unadjusted and adjusted analysis.

CDI: Clostridium difficile infection
GPC: infection with antibiotic‐resistant gram‐positive cocci
GNB: infection with antibiotic‐resistant gram‐negative bacteria

Other infection control: 'Yes' means there were changes to infection control processes during the study period.

Open in table viewer
Table 9. Microbial outcomes from 26 ITS studies from the prescribing meta‐analysis that include reliable data about prescribing outcomes at 6 months and microbial outcomes at 12 months postintervention

Prescribing target

Microbial outcome

N

Study ID

Cephalosporins

GNRB

8

Grohs 2014; Kim 2008; Knudsen 2014; Lee 2007; McNulty 1997; Meyer 2009; Petrikkos 2007; Tangdén 2011

MRSA

1

May 2000

Carbapenems

GNRB

1

Goldstein 2009

Fluoroquinolones

GNRB

3

Cook 2011b; Lafaurie 2012; Willemsen 2010

MRSA

1

Lafaurie 2012

High‐risk antibiotics

CDI

6

Aldeyab 2012; Chan 2011; Dancer 2013; Fowler 2007; Talpaert 2011; Valiquette 2007

GNRB

4

Buising 2008a; Chan 2011; Dancer 2013; Liebowitz 2008

MRSA

6

Aldeyab 2014; Ananda‐Rajah 2010; Chan 2011; Dancer 2013; Fowler 2007; Liebowitz 2008

Total antibiotic use

CDI

2

Cook 2011a; Jump 2012

MRSA

1

Cook 2011a

Vancomycin

VRE

1

Lautenbach 2003

Total microbial

34*

*Some studies had more than one microbial outcome, so the total is 34 microbial outcomes from 26 studies.

CDI: Clostridium difficile infection
GNRB: gram‐negative resistant bacteria
ITS: interrupted time series
MRSA: methicillin‐resistant Staphylococcus aureus
VRE: vancomycin‐resistant enterococci

In the 20 studies of planned intervention, there were six studies with unclear information about other infection control interventions or changes during the study period (Chan 2011; Grohs 2014; Jump 2012; Liebowitz 2008; Meyer 2009; Petrikkos 2007). We performed meta‐regression on the remaining 14 studies from Table 9 (Figure 14). In contrast with the meta‐regression of all 27 studies (Figure 13), the effects of setting, other infection control interventions, and microbial outcome type were all reversed so that each of these variables was associated with increase in effect size in the 14 studies with planned interventions and details of other infection control interventions (Figure 14).


Meta‐regression by effect modifiers for 20 microbial outcomes 12 months' postintervention from 14 ITS studies of planned interventions that provided details about other infection control changes or interventions.CDI: Clostridium difficile infection
 GPC: infection with antibiotic‐resistant gram‐positive cocci
 GNB: infection with antibiotic‐resistant gram‐negative bacteriaOther infection control: 'Yes' means there were changes to infection control processes during the study period.

Meta‐regression by effect modifiers for 20 microbial outcomes 12 months' postintervention from 14 ITS studies of planned interventions that provided details about other infection control changes or interventions.

CDI: Clostridium difficile infection
GPC: infection with antibiotic‐resistant gram‐positive cocci
GNB: infection with antibiotic‐resistant gram‐negative bacteria

Other infection control: 'Yes' means there were changes to infection control processes during the study period.

The antibiotic targets for the 20 studies of planned interventions were single antibiotic classes in nine studies (Cook 2011b; Grohs 2014; Knudsen 2014; Lafaurie 2012; Lee 2007; Meyer 2009; Petrikkos 2007; Willemsen 2010; Yoon 2014), high‐risk antibiotics in nine studies (Aldeyab 2012; Aldeyab 2014; Ananda‐Rajah 2010; Buising 2008a; Chan 2011; Dancer 2013; Fowler 2007; Liebowitz 2008; Talpaert 2011), and all antibiotics in the remaining two studies (Cook 2011a; Jump 2012). High‐risk antibiotics were a combination of drugs from more than one class of antibiotic, which were all considered to be high risk for the microbial outcome. The prescribing outcome data reported in these nine studies varied from just one of the high‐risk antibiotics, in Dancer 2013, through individual results for all of the high‐risk antibiotics, in Buising 2008a, Chan 2011, Fowler 2007, and Talpaert 2011, to combined results for all of the high‐risk antibiotics (Aldeyab 2012; Aldeyab 2014; Ananda‐Rajah 2010; Liebowitz 2008).

One study can be used to demonstrate the technical challenges of estimation of intervention effect on microbial outcomes (Dancer 2013). The intervention was addition of complete restriction of ceftriaxone and ciprofloxacin to a pre‐existing multifaceted intervention introduced seven months before restriction and remaining in place throughout the restrictive period (Dancer 2013). We could not analyse the effect of the initial multifaceted intervention because there were no pre‐intervention data about prescribing or microbial outcomes. However, the available data showed CDI was lower by ‐0.143 cases per 1000 occupied bed days per month in the nine months prior to the addition of the restrictive intervention. At the start of the restrictive intervention, CDI rates were already low (1.5 cases per 1000 occupied bed days). After the introduction of restriction, CDI rates continued to decline for five months, and then stabilised at around 0.5 cases per 1000 occupied bed days. These data suggest that the restrictive intervention had no additional effect on the rate of CDI. However, the segmented regression analysis estimated that there was a relative increase of 35.8% in CDI rate 12 months after the restrictive intervention with very wide confidence intervals (from 81.0% decrease to 152.7% increase).

Our review did include one multicentre controlled ITS study comparing CDI rates in six hospitals with antimicrobial stewardship programmes versus four control hospitals (Ostrowsky 2014). We did not include this study in evidence synthesis because neither the interventions nor the prescribing outcomes were standardised across the six hospitals with stewardship programmes. Baseline rates of CDI were only 0.8 cases per 1000 occupied bed days in the intervention and control hospitals before the intervention, and the authors did not report a decrease in aggregate CDI rates either between intervention and non‐intervention groups or within the intervention groups over time (Ostrowsky 2014).

We have not attempted to synthesise microbial outcome data because of the small number of studies, the heterogeneity of intervention targets and prescribing outcomes, and the wide confidence intervals for estimated relative effect. We have focused on the 20 ITS studies of planned interventions and separated the results by microbial outcome type. Interventions were associated with consistent reduction in CDI (median ‐48.6%, interquartile range ‐80.7% to ‐19.2%) but inconsistent effect on resistant gram‐negative bacteria (median ‐12.9%, interquartile range ‐35.3% to 25.2%) and resistant gram‐positive bacteria (median ‐19.3%, interquartile range ‐50.1% to 23.1%). There were too few studies with too much variance in microbial outcomes to reliably assess the relationship between change in antibiotic use and each of the microbial outcomes.

Discussion

disponible en

Summary of main results

The RCTs provide high‐certainty evidence that interventions are effective in increasing compliance with antibiotic policies and in reducing duration of antibiotic treatment safely, without an increase in mortality. Furthermore, interventions were associated with a reduction in length of stay. The mechanism is not clear, and further investigation is required. However, reducing length of stay is a key organisational objective for most hospitals, so this evidence should be used to prioritise antimicrobial stewardship in hospitals.

Analysis of effect modifiers in RCTs and ITS studies consistently supported the theory that involving enablement increases intervention effect, including those with restrictive components. However, feedback was only used in a minority of enablement interventions, and very few included goal setting or action planning.

Overall completeness and applicability of evidence

The RCTs show that interventions increase compliance with policies or guidelines by 15%, which is a clinically important effect size. However, the result is less impressive when one considers that health professionals’ adherence to prescribing recommendations increased from 43% to 58%, because 58% compliance is probably still far too low. Three studies did achieve 90% compliance with guidelines by making this an explicit goal for the intervention and using action planning to revise interventions until the goal was achieved (Jobson 2015; Volpe 2012; Weinberg 2001).

The ITS studies provided important additional evidence that the results of RCTs regarding effectiveness of interventions can be reproduced in routine practice: 70% of ITS studies reported on hospital‐wide interventions compared with only 31% of RCTs.

Only two ITS studies included data that enabled assessment of the effect of adding an intervention component to an existing intervention (Mol 2005; Po 2012). This is a strong study design that should be more widely used to evaluate these types of interventions.

Safety and unintended consequences of interventions

The main limitation of the RCT evidence regarding safety of reducing unnecessary use was that only two interventions included restriction, and one was stopped early because of delay in the start of treatment (Strom 2010). Two NRS also raised concerns about delay in time to first antibiotic dose associated with restrictive interventions (LaRosa 2007; Winters 2010). Furthermore, four NRS described negative effects of restrictive interventions on professional culture through breakdown in trust and communication (Baysari 2013; Calfee 2003; Connor 2007; Linkin 2007). These NRS used either case control, cohort, or qualitative designs because they required collection of data that were not available in routine clinical systems (Table 5).

The ITS studies provided very little evidence about the safety of interventions because they rely on routine clinical systems for outcome measures, which are currently largely incapable of providing information about specific patients, for example those with infection. Moreover, the range of clinical measures should be extended beyond infection outcomes to include safety indicators such as acute kidney injury (AKI). (Bell 2014). Scotland’s Infection Intelligence Platform was established to improve linkage and availability of routine data (ISD 2016), but research is required to improve timeliness, quality, and relevance of clinical outcome measures and to provide a richer understanding of the unintended consequences of improvement interventions (SISCC 2016). We found only one example of a qualitative study of unintended consequences (Baysari 2013). This is an important study design for investigation of unanticipated consequences of interventions and should be more widely used (Rogers 1995).

Studying the effect of removal of an intervention can be used to provide additional evidence about the outcomes of the original intervention (Walker 2016). This study was from the same group that reported that an intervention that was intended to reduce risk of CDI in people undergoing orthopaedic surgery was associated with an increased risk of postoperative AKI (Bell 2014) (Table 4). The increase in AKI was attributed to change in antibiotic surgical prophylaxis policy from cefuroxime to flucloxacillin and gentamicin. This second study showed reduction in postoperative AKI associated with a change away from flucloxacillin and gentamicin, which provides persuasive additional evidence that gentamicin was responsible for the original increase in postoperative AKI (Walker 2016).

Interventions were consistently associated with reduced length of stay (Analysis 2.4), and the results were similar when analysis was restricted to RCTs at low or medium risk of bias (Analysis 2.6). Measurement of length of stay was intended to provide reassurance about safety of the intervention so that reduction in length of stay is an example of an unanticipated beneficial outcome (Ash 2007; Rogers 1995). We found similar results for interventions that targeted antibiotic choice (Analysis 3.2) or antibiotic exposure (Analysis 4.2). One possible mechanism for reduction in length of stay is that interventions reduced the duration of intravenous antibiotic therapy (Carratala 2012). However, further research is required.

Microbial outcomes

Interventions were consistently associated with reduction in CDI, but less consistently associated with reduction in infection by resistant bacteria. However, intervention effects on microbial outcomes could only be analysed reliably in planned interventions (Figure 13), and our meta‐analysis was limited by four technical challenges.

  1. Each study had considerable variance because of the small number of microbial events in each time point.

  2. Studies rarely had stable pre‐intervention data, so that extrapolation of the pre‐intervention trend throughout the postintervention phase was probably unreliable.

  3. We analysed a single prescribing outcome for each study (even if more were reported). The criteria for selection of the prescribing outcome were determined by the analysis plan for the effect of interventions on prescribing behaviour. However, these criteria may not have been correct for analysis of the relationship between changes in prescribing and microbial outcomes.

  4. We could only analyse the relationship between prescribing and microbial outcomes at fixed time points. We chose six and 12 months, respectively, imposing a six‐month time lag for all interventions. However, the time lag will likely vary by prescribing and microbial outcomes, and by intervention context (Vernaz 2008).

Quality of the evidence

We found high‐certainty evidence that interventions increase appropriate use of antibiotics, reduce duration of antibiotic treatment, and shorten hospital stay without increasing the risk of mortality. There was low‐certainty evidence that these interventions can delay treatment and create a negative professional culture (summary of findings Table for the main comparison). High risk of bias was associated with greater intervention effect in RCTs (Figure 7) for the outcome of compliance with desired practice. However, we have presented separate analysis of effects for RCTs at low or medium risk of bias (Analysis 1.3; Analysis 1.6; Analysis 2.3; Analysis 2.6). These analyses provide evidence supporting our decision not to downgrade for risk of bias, since excluding studies at high risk of bias did not substantively change the direction of effect. We did not downgrade for inconsistency since the direction of effect across the studies was consistent, and our meta‐regression provides some explanation for the high levels of statistical heterogeneity between the results of the studies. The certainty of evidence about adverse effects was more variable, with particular concerns about the unintended consequences of restrictive interventions, namely delays in treatment and negative professional culture, for which we have low‐certainty evidence.

The quality of reporting of interventions was poor, which makes it difficult for professionals and clinical teams to reliably implement interventions that have been shown to be useful and for other researchers to replicate or build on research findings (Hoffmann 2014). We found high‐certainty evidence that enablement and restriction both enhanced the effectiveness of interventions. However, we found only moderate‐certainty evidence for the effectiveness of feedback, and there were too few studies with action planning and goal setting to provide any reliable information about the combined effects of these behaviour change techniques.

In the analysis of risk of bias equal weight is given to all criteria (Figure 2). Our results for microbial outcomes clearly showed that the risk of bias from unplanned interventions is much greater than the risk from other infection control interventions (Figure 13; Figure 14).

We found that some NRS study designs provided important additional evidence about intervention effects and sustainability in routine clinical practice (ITS studies) and about unintended consequences (case control, cohort, and qualitative studies). However, we found no useful evidence from CBAs or NRTs and suggest that these study designs should not be included in updates to this review.

Heterogeneity of intervention effect

We found that two intervention functions, enablement and restriction, explained some of the variation in targeted prescribing behaviour. However, we found little evidence that behaviour change theory had been used to design interventions (Charani 2011). There were too few interventions with explicit goals or action planning to include these variables in meta‐regression.

There was no consistent evidence that intervention setting or target explained variation in the effect of interventions (Figure 7; Figure 10)

Potential biases in the review process

Our decision not to use adjusted data for cluster RCTs for the primary analysis could be contested. The consequences of using unadjusted data would be to assign too much weight to cluster studies in the analysis, potentially biasing the effect from our analyses to their results (Higgins 2011). We believe that taking clustering into account is unlikely to impact on the strength of the results in such a way as to change the conclusions of the review. Our sensitivity analyses provide some indirect support for the approach we have undertaken. In comparison to unadjusted results, analyses based on the effective sample sizes calculated from assumed ICCs consistently gave a larger average intervention effect (Analysis 1.1 versus Analysis 1.2; Analysis 2.1 versus Analysis 2.2; Analysis 2.4 versus Analysis 2.5).The increased effect size could be explained by the lower weight assigned to the cluster studies, which tended to have smaller effects than the individually randomised studies.

The electronic literature search did not identify 42 (19%) of the 221 included studies, highlighting some of the challenges in constructing sensitive search terms for reviews of behavioural interventions and the identification of non‐randomised studies. It is possible that additional eligible studies have not been retrieved by the search process we undertook for this review.

We did not find evidence of publication bias in the RCTs, however publication bias is more likely in the ITS studies because the decision to publish may have been made after the analysis of intervention effect.

Agreements and disagreements with other studies or reviews

Agreements

Ivers 2012 included and analysed 140 RCTs that compared any intervention in which audit and feedback was a core, essential component to usual care and evaluated effects on professional practice. The review concluded that interventions were more effective if they also included goal setting and action planning. We were unable to reproduce their analysis because only four of our RCTs included feedback (Figure 8). Although 20 ITS studies included feedback (Figure 12), there were not enough studies with goal setting or action planning for reliable analysis.

Our findings are similar to a previous review that found that behavioural determinants and social norms were not given due consideration in the design and evaluation of interventions to change antibiotic prescribing (Charani 2011).

Sustainability of intervention effect

We found evidence that removal of restriction, in Himmelberg 1991, Kallen 2009, Kim 2008, and Skrlin 2011, or of review and recommend change (enablement, Standiford 2012) was associated with reversal of intervention effect (Table 7). Three previous studies have shown that removal of financial incentives is associated with reversal of intervention effects in primary care (Avery 2012; Dreischulte 2016; Lester 2010). This is an important issue because the attractiveness of interventions will be reduced if improvement resources cannot be moved on to new priorities. Restriction is a relatively low‐cost intervention, but it is worrying that an enabling intervention (review and recommend change) apparently had no sustained effect on clinical teams after being in place for seven years (Standiford 2012). Review and recommend change is a time‐intensive process that was included in 36 (54%) of 67 of the enabling interventions in ITS studies.

Disagreements

A systematic review on current evidence about antimicrobial stewardship objectives reported that "guideline‐adherent empirical therapy was associated with a reduction for mortality (odds ratio 0.65, 95% CI 0.54‐0.80)" (Schuts 2016). Only two of the 39 studies in this review reported an intervention: one was invalid because it was an uncontrolled before‐after study (Garcia 2007), and the other was a CBA (Dean 2006). The remaining 37 studies used case control study or cohort designs to compare the outcomes of participants with and without guideline‐adherent antibiotic treatment, and did not include an intervention to change professional practice. The results of this review are in marked contrast to our analysis of mortality in 11 RCTs targeting antibiotic choice (Analysis 3.1). The most likely explanation for the discrepancy between our results and Schuts 2016 is confounding by indication. It is likely that participants with less complex or severe illness were more likely to receive guideline‐adherent antibiotic treatment and that there was residual confounding after adjustment for available clinical information.

A systematic review on the effect of antibiotic stewardship programmes on CDI reported that interventions were associated with a consistent, significant protective effect (pooled risk ratio for CDI 0.48, 95% CI 0.38 to 0.62) (Feazel 2014). Of the 16 studies included in this systematic review, four were ITS studies that were also included in our review (Elligsen 2012; Fowler 2007; Price 2010; Talpaert 2011), and the remaining 12 studies were either uncontrolled before‐after or inadequate ITS studies. The statistical analysis in this review was not appropriate (Feazel 2014). Calculation of risk ratios for the post‐ versus pre‐intervention periods is an uncontrolled before‐after analysis, which does not provide a reliable estimate of intervention effect.

Additional details about the disagreements with Feazel 2014 and Schuts 2016 can be found in Appendix 7.

Limitations

There are five weaknesses in the current evidence.

  1. Evidence of intended effects is unbalanced towards reducing unnecessary treatment (compliance with guidelines that are intended to reduce use of broad‐spectrum antibiotics or shorten duration of treatment). More evidence is required about finessing effective treatment of sepsis without also causing excessive use of antibiotics.

  2. The limited evidence regarding adverse effects of restrictive interventions suggests that they can be associated with delay in essential treatment. There is a need for better patient safety outcome measures that can be used in studies of interventions in clinical practice.

  3. The majority of the interventions do not use effective behaviour change techniques such as action planning or feedback.

  4. Given the critical role of junior doctors in antimicrobial stewardship in hospitals, it is surprising that there is only a single example of an intervention that involved junior doctors in self monitoring and reflection on feedback about their prescribing (Price 2010).

  5. Analysis of the impact of interventions on microbial outcomes requires large, multihospital RCTs.

Figure 1 Study flow diagram.
Figuras y tablas -
Figure 1

Figure 1 Study flow diagram.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.Blank sections in this graph are due to use of different ROB criteria for CBA, NRT and RCT versus ITS studies
Figuras y tablas -
Figure 2

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Blank sections in this graph are due to use of different ROB criteria for CBA, NRT and RCT versus ITS studies

Forest plot of comparison: 1 Prescribing: RCTs of all interventions to reduce unnecessary prescribing, outcome: 1.1 Dichotomous outcomes, increase in desired practice.
Figuras y tablas -
Figure 3

Forest plot of comparison: 1 Prescribing: RCTs of all interventions to reduce unnecessary prescribing, outcome: 1.1 Dichotomous outcomes, increase in desired practice.

Forest plot of comparison: 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 1.4 Continuous outcomes, duration of all antibiotic treatment (days).
Figuras y tablas -
Figure 4

Forest plot of comparison: 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 1.4 Continuous outcomes, duration of all antibiotic treatment (days).

Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.1 Mortality, all RCTs.
Figuras y tablas -
Figure 5

Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.1 Mortality, all RCTs.

Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.4 Length of stay, all RCTs.
Figuras y tablas -
Figure 6

Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.4 Length of stay, all RCTs.

Meta‐regression by effect modifier for 29 RCTs. A positive value for Beta indicates enhanced intervention effect. One RCT had both enabling and restrictive components in the intervention (Strom 2010).
Figuras y tablas -
Figure 7

Meta‐regression by effect modifier for 29 RCTs. A positive value for Beta indicates enhanced intervention effect. One RCT had both enabling and restrictive components in the intervention (Strom 2010).

Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.1 Enablement plus feedback.
Figuras y tablas -
Figure 8

Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.1 Enablement plus feedback.

Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.2 Enablement without feedback.
Figuras y tablas -
Figure 9

Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.2 Enablement without feedback.

Meta‐regression by effect modifiers of intervention for 91 ITS studies. Outcome is effect on prescribing six months' postintervention. There are 16 studies with both enabling and restricting intervention components ().
Figuras y tablas -
Figure 10

Meta‐regression by effect modifiers of intervention for 91 ITS studies. Outcome is effect on prescribing six months' postintervention. There are 16 studies with both enabling and restricting intervention components (Figure 11).

Meta‐regression of prescribing outcome by effect modifiers for 29 ITS studies of interventions that included restriction.
Figuras y tablas -
Figure 11

Meta‐regression of prescribing outcome by effect modifiers for 29 ITS studies of interventions that included restriction.

Meta‐regression by effect modifier for 43 ITS studies of interventions that included enablement but not restriction. Outcome is effect on prescribing six months' postintervention. Note that four studies with feedback were not included in this analysis because they also included restriction.
Figuras y tablas -
Figure 12

Meta‐regression by effect modifier for 43 ITS studies of interventions that included enablement but not restriction. Outcome is effect on prescribing six months' postintervention. Note that four studies with feedback were not included in this analysis because they also included restriction.

Meta‐regression by effect modifiers for 34 microbial outcomes 12 months' postintervention from 26 ITS studies. The bars show the results for unadjusted versus adjusted analyses, the comparison for unplanned interventions is with planned interventions in both the unadjusted and adjusted analysis.CDI: Clostridium difficile infection
 GPC: infection with antibiotic‐resistant gram‐positive cocci
 GNB: infection with antibiotic‐resistant gram‐negative bacteriaOther infection control: 'Yes' means there were changes to infection control processes during the study period.
Figuras y tablas -
Figure 13

Meta‐regression by effect modifiers for 34 microbial outcomes 12 months' postintervention from 26 ITS studies. The bars show the results for unadjusted versus adjusted analyses, the comparison for unplanned interventions is with planned interventions in both the unadjusted and adjusted analysis.

CDI: Clostridium difficile infection
GPC: infection with antibiotic‐resistant gram‐positive cocci
GNB: infection with antibiotic‐resistant gram‐negative bacteria

Other infection control: 'Yes' means there were changes to infection control processes during the study period.

Meta‐regression by effect modifiers for 20 microbial outcomes 12 months' postintervention from 14 ITS studies of planned interventions that provided details about other infection control changes or interventions.CDI: Clostridium difficile infection
 GPC: infection with antibiotic‐resistant gram‐positive cocci
 GNB: infection with antibiotic‐resistant gram‐negative bacteriaOther infection control: 'Yes' means there were changes to infection control processes during the study period.
Figuras y tablas -
Figure 14

Meta‐regression by effect modifiers for 20 microbial outcomes 12 months' postintervention from 14 ITS studies of planned interventions that provided details about other infection control changes or interventions.

CDI: Clostridium difficile infection
GPC: infection with antibiotic‐resistant gram‐positive cocci
GNB: infection with antibiotic‐resistant gram‐negative bacteria

Other infection control: 'Yes' means there were changes to infection control processes during the study period.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Dichotomous outcomes, increase in desired practice.
Figuras y tablas -
Analysis 1.1

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Dichotomous outcomes, increase in desired practice.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Dichotomous outcomes, all RCTs with results of cluster RCTs adjusted by inflation factor.
Figuras y tablas -
Analysis 1.2

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Dichotomous outcomes, all RCTs with results of cluster RCTs adjusted by inflation factor.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Dichotomous outcomes, low or medium 'Risk of bias' studies only.
Figuras y tablas -
Analysis 1.3

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Dichotomous outcomes, low or medium 'Risk of bias' studies only.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Continuous outcomes, duration of all antibiotic treatment (days).
Figuras y tablas -
Analysis 1.4

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Continuous outcomes, duration of all antibiotic treatment (days).

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Continuous outcomes, duration of all antibiotic treatment with results of cluster RCTs adjusted by inflation factor.
Figuras y tablas -
Analysis 1.5

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Continuous outcomes, duration of all antibiotic treatment with results of cluster RCTs adjusted by inflation factor.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Continuous outcomes, low or medium 'Risk of bias' studies only.
Figuras y tablas -
Analysis 1.6

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Continuous outcomes, low or medium 'Risk of bias' studies only.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 7 Continuous outcome, consumption of targeted antibiotic only, standardised mean reduction (original outcome cost, days or DDD).
Figuras y tablas -
Analysis 1.7

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 7 Continuous outcome, consumption of targeted antibiotic only, standardised mean reduction (original outcome cost, days or DDD).

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Mortality, all RCTs.
Figuras y tablas -
Analysis 2.1

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Mortality, all RCTs.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Mortality, all RCTs with results of cluster RCTs adjusted by inflation factor.
Figuras y tablas -
Analysis 2.2

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Mortality, all RCTs with results of cluster RCTs adjusted by inflation factor.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Mortality, low or medium 'Risk of bias' RCTs.
Figuras y tablas -
Analysis 2.3

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Mortality, low or medium 'Risk of bias' RCTs.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Length of stay, all RCTs.
Figuras y tablas -
Analysis 2.4

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Length of stay, all RCTs.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Length of stay, all RCTs with results of cluster RCTs adjusted by inflation factor.
Figuras y tablas -
Analysis 2.5

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Length of stay, all RCTs with results of cluster RCTs adjusted by inflation factor.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Length of stay, low or medium 'Risk of bias' RCTs only.
Figuras y tablas -
Analysis 2.6

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Length of stay, low or medium 'Risk of bias' RCTs only.

Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 1 Mortality for trial patients.
Figuras y tablas -
Analysis 3.1

Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 1 Mortality for trial patients.

Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 2 Length of stay for trial patients.
Figuras y tablas -
Analysis 3.2

Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 2 Length of stay for trial patients.

Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 1 Mortality for trial patients.
Figuras y tablas -
Analysis 4.1

Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 1 Mortality for trial patients.

Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 2 Length of stay for trial patients.
Figuras y tablas -
Analysis 4.2

Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 2 Length of stay for trial patients.

Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 1 Enablement with feedback.
Figuras y tablas -
Analysis 5.1

Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 1 Enablement with feedback.

Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 2 Enablement without feedback.
Figuras y tablas -
Analysis 5.2

Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 2 Enablement without feedback.

Summary of findings for the main comparison. Effects of interventions to improve use of antibiotics on prescribing, clinical outcomes, adverse events, and effect modifiers (heterogeneity)

Patient or population: adults or children undergoing inpatient antibiotic prophylaxis or treatment

Settings: mainly high‐income countries (North America or Western Europe)

Intervention: any intervention targeting healthcare professionals that aimed to improve antibiotic prescribing to hospital inpatients

Comparison: usual care (varied across studies)

Effectiveness: prescribing outcomes from RCTs

Outcomes

Absolute effect*

No of participants

(No of studies)

Certainty of the evidence (GRADE)

Comments

Without intervention

With intervention

Proportion of participants who were treated according to antibiotic prescribing guidelines

Follow‐up to end of study

43 per 100

58 per 100

23,394 participants

(29 RCTs)

⊕⊕⊕⊕
High

We have graded the certainty of evidence as high because heterogeneity was explained by prespecified effect modifiers (see below). The intervention effect varied between the studies, but the direction of effect was consistent. Restricting the analysis to studies at low risk of bias gave a similar result (RD 11%, 95% CI 10% to 12%).

Difference: 15 more participants per 100 (95% CI 15 to 23) received appropriate treatment following intervention.

Duration of all antibiotic treatment

11.0 days

9.1 days

3318 participants

(14 RCTs)

⊕⊕⊕⊕
High

Difference: 1.95 fewer days per participant (95% CI 2.22 to 1.67)

Mortality

Follow‐up to end of study

11 per 100

11 per 100

15,827 participants

28 (RCTs)

⊕⊕⊕⊝1
Moderate

Mortality and length of stay were measured to determine the impact of reduced antibiotic use on clinical outcomes. The results were similar for studies that targeted antibiotic choice or exposure.

Only 1 of the interventions in the RCTs with mortality or length‐of‐stay outcomes had a restrictive component (Singh 2000). This evidence is therefore at high risk of indirectness because 7 studies in the next section of the table (see below) raise concerns about the safety of restrictive interventions. Moreover, the ITS studies showed that restrictive components were included in 42 (34%) of 123 hospital interventions.

Difference: 0 more deaths per 100 participants (95% CI 1 to 0 fewer)

Mean length of hospital stay per participant

12.9 days

11.8 days

3834 participants

15 (RCTs)

⊕⊕⊕⊝1
Moderate

Difference: 1.1 fewer days per participant (95% CI 1.5 to 0.7 fewer)

Delay in treatment

Restrictive interventions increased the risk of delay in all 3 studies. The risk to patients resulted in termination of the RCT by the Trial Monitoring Committee.

1 RCT, 2 cohort

⊕⊕⊝⊝2
Low

The evidence from these 7 studies of unintended consequences raises concerns about the directness of the evidence of safety from the 29 RCTs in the previous section of the table (see above).

Negative professional culture

Loss of trust in infection specialists because of failure to record approvals for restricted drugs or provide warning about stopping treatment

Misleading or inaccurate information from prescribers in order to meet criteria for restricted drugs. In 1 hospital, misdiagnosis of hospital‐acquired infection was large enough to trigger an outbreak investigation.

1 case control, 2 cohort, 1 qualitative

⊕⊕⊖⊖3
Low

Effect modifiers (heterogeneity) for immediate effect of intervention on prescribing outcomes:
impact of behaviour change functions (enablementor restriction) and additional impact of feedback, RCTs and ITS studies. A positive value for Beta means the modifier is associated with increased effect

Effect modifier

Adjusted effect in meta‐regression
Beta
(95% CI)

Number of studies

Certainty of the evidence (GRADE)

Comments

Enablement

15.12

(8.45 to 21.8)

29 RCTs

⊕⊕⊕⊕
High

The effect of enablement and restriction is similar in the RCTs and ITS studies. Of the 29 RCTs, only 8 (31%) of interventions were hospital‐wide, the majority being in single units. In contrast, 64 (70%) of the interventions in ITS studies were hospital‐wide.

12.86

(4.11 to 21.6)

91 ITS

Restriction

34.91

(13.52 to 56.29)

29 RCTs

⊕⊕⊕⊕
High

24.69

(13.74 to 35.64)

91 ITS

Addition of feedback to enablement

10.88

(7.16 to 19.32)

23 RCTs

⊕⊕⊕⊝2
Moderate

Feedback was included in 4 (17%) of 23 RCTs and 20 (47%) of 43 ITS studies with interventions that included enablement. There were not enough interventions with goal setting and action planning to analyse as effect modifiers.

15.63

(0.56 to 30.70)

43 ITS

Addition of enablement to restriction

38.36

(18.94 to 57.78)

29 ITS

⊕⊕⊖⊖3
Low

Enablement was included in 13 (45%) of 29 ITS studies with restrictive interventions.

*The risk WITHOUT the intervention is based on the median control group risk across studies. The corresponding risk WITH the intervention (and the 95% confidence interval for the difference) is based on the overall relative effect (and its 95% confidence interval).
CI: confidence interval; ITS: interrupted time series; RCT: randomised controlled trial; RD: risk difference

GRADE Working Group grades of evidence
High certainty: Further research is very unlikely to change our confidence in the estimate of effect.
Moderate certainty: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.
Low certainty: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.
Very low certainty: We are very uncertain about the estimate.

Details of five GRADE criteria for all outcomes from RCTs are in Appendix 2.

1We downgraded the evidence to moderate because of indirectness.
2We downgraded the evidence because most studies are non‐randomised studies.
3We graded the evidence as low because it is all from non‐randomised studies.
4We graded the evidence as very low because it is all from non‐randomised studies and there was too much heterogeneity for reliable evidence synthesis.

Figuras y tablas -
Summary of findings for the main comparison. Effects of interventions to improve use of antibiotics on prescribing, clinical outcomes, adverse events, and effect modifiers (heterogeneity)
Table 1. Definition of behaviour change techniques and intervention functions

Intervention Function

Definition

Intervention components

Education

Increasing knowledge or understanding

Educational meetings;

Dissemination of educational materials;

Educational outreach

Persuasion

Using communication to induce positive or negative feelings or to stimulate action

Educational outreach by academic detailing or review and recommend change

Restriction

Using rules to reduce the opportunity to engage in the target behaviour (or increase the target behaviour by reducing the opportunity to engage in competing behaviours)

Restrictive

Environmental restructuring

Changing the physical context

Reminders (physical) such as posters, pocket‐size or credit card‐size summaries or on laboratory test reports;

Structural (e.g. new laboratory tests or rapid reporting of results)

Enablement

Increasing means/reducing barriers to increase capability or opportunity

Audit and feedback;

Decision support through computerised systems or through circumstantial reminders that were triggered by actions or events related to the targeted behaviour;

Educational outreach by review and recommend change

Figuras y tablas -
Table 1. Definition of behaviour change techniques and intervention functions
Table 2. Unintended consequences of ITS studies: mortality*

Study

Prescribing target

Restriction

Design of analysis

Effect estimate

95% CI

Lee 2014

Choice of drug

No

Cohort

Incidence rate ratio 1.1

0.9 to 1.5

Popovski 2015

Choice of drug

No

Cohort

Increase by 1.4%

‐1.2% to 4.1%

Wang 2014

Choice of drug

Yes

ITS, segmented regression

Change in slope ‐0.0172

No data

Yoon 2014

Choice of drug

Yes

Cohort

+0.43 per 1000 OBD

No data

*Mortality was measured in all patients in the hospital rather than just those patients who were the targets of the interventions.

CI: confidence interval
ITS: interrupted time series
OBD: occupied bed day

Figuras y tablas -
Table 2. Unintended consequences of ITS studies: mortality*
Table 3. Unintended consequences of ITS studies: length of stay*

Study

Prescribing target

Restrictive

Design of analysis

Effect estimate

95% CI

Mittal 2014

Exposure, % treated

No

Cohort

‐0.5 days

No data

Skaer 1993

Choice of drug

No

Cohort

‐0.1 days

‐0.49 to +0.29

*Length of stay was measured in all patients in the hospital rather than just those patients who were the targets of the interventions.

CI: confidence interval
ITS: interrupted time series

Figuras y tablas -
Table 3. Unintended consequences of ITS studies: length of stay*
Table 4. Unintended consequences of ITS studies: other

Study

Prescribing target

Design of analysis

Effect measure

Effect estimate

95% CI

Bell 2014

Antibiotic choice

ITS, segmented regression

Risk of postoperative acute kidney injury

Increase 98%

93.8% to 94.2%

Van Kasteren 2005

Exposure, duration

Cohort

Surgical‐site infection

Decrease 0.8%

‐2.2% to 0.6%

Volpe 2012

Time to first antibiotic dose

Cohort

Left without being seen rate

Decrease 0.4%

No data

CI: confidence interval
ITS: interrupted time series

Figuras y tablas -
Table 4. Unintended consequences of ITS studies: other
Table 5. Unintended consequences studies (case control, cohort, or qualitative)

Study

Design

Patients

Intended target

Unintended consequence

Effect estimate

95% CI

Interventions with a restrictive component

Baysari 2013

Qualitative

36 physicians

Reduce unnecessary use of restricted antibiotics

Inaccurate feedback

Not quantified; qualitative study

Calfee 2003

Case control

Not clear

Increase in physician‐based diagnosis of nosocomial infection

No denominator data

Connor 2007

Cohort

120

Failure to warn prescribers about discontinuation

Duvoisin 2014

Cohort

222

Reduce unnecessary laboratory tests

Delay in TFAD (HR > 1 shows delay less likely in intervention period)

Multivariate HR 1.56

1.17 to 2.07

LaRosa 2007

Cross‐sectional

15,440

Reduce unnecessary use of restricted antibiotics

Orders for restricted antibiotics (% all orders) from 10 to 11 pm vs all other hours

Cohort

360

% appropriate orders 10 to 11 pm vs 9 to 10 pm

‐23.7%

‐31.8% to ‐15.5%

Linkin 2007

Cohort

200

Risk of inaccurate information in orders judged inappropriate vs appropriate

OR 2.2

1.0 to 4.4

Winters 2010

Cohort

3251

Risk of 1‐hour delay in TFAD

OR 1.5

1.2 to 1.8

Risk of 2‐hour delay in TFAD

OR 1.8

1.4 to 2.2

Interventions with no restrictive component

Friedberg 2009

Cohort

13,042

Reduce time to first antibiotic dose for patients with community‐acquired pneumonia

% CAP diagnoses

1% increase

No denominator data

Kanwar 2007

Cohort

518

% correct CAP diagnoses

‐7.9% decrease

‐15.4% to ‐0.4%

Welker 2008

Cohort

548

% correct CAP diagnoses

‐16.0% decrease

‐7.6% to ‐24.4%

CAP: community‐acquired pneumonia
CI: confidence interval
HR: hazard ratio
OR: odds ratio
TFAD: time to the first antibiotic dose

Figuras y tablas -
Table 5. Unintended consequences studies (case control, cohort, or qualitative)
Table 6. Summary of intervention components for 29 RCTs (Analysis 1.1; Figures 3 and 7) and 91 ITS studies (Figure 10)

Intervention function and components

RCT

ITS

Enablement

24

studies

59

studies

Number of enabling or restrictive intervention components

27

76

Studies with > 1 Enabling intervention component

2

8%*

19

32%*

Audit and feedback

4

17%

24

41%

Computerised decision support

1

4%

3

5%

Circumstantial reminders

16

67%

18

31%

Review and recommend change

6

25%

31

53%

Restriction

2

studies

29

studies

Number of Restrictive intervention components

3

41

Studies with > 1 Restrictive intervention component

1

50%

10

34%

Expert approval

1

50%

18

62%

Compulsory order form

1

50%

7

24%

Removal

0

10

34%

Review and make change

1

50%

6

21%

No Enablement or Restriction

4

studies

18

studies

Number of intervention components

6

25

Studies with > 1 intervention component

2

50%

6

33%

Educational materials or meetings

3

75%

16

89%

Educational outreach (academic detailing)

1

25%

6

33%

Physical reminders

1

25%

2

11%

Structural intervention

1

25%

1

6%

*The denominator for all percentages is the number of studies for each intervention function. One RCT, Strom 2010, and 16 ITS studies (Figure 11) included both enabling and restrictive intervention components.

ITS: interrupted time series
RCT: randomised controlled trial

Figuras y tablas -
Table 6. Summary of intervention components for 29 RCTs (Analysis 1.1; Figures 3 and 7) and 91 ITS studies (Figure 10)
Table 7. Data from 5 studies about the effect of removal of interventions. The intended effect of all interventions was reduction in unnecessary antibiotic use

Study

Intervention function

Intervention effect (95% CI)

Time intervention was in place

Effect of removal (95% CI)

Kallen 2009

Restriction

‐87.5%

‐115.4 to ‐59.7

6 months

398.9%

238.2 to 559.5

Kim 2008

Restriction

‐23.1%

‐53.7 to +7.4

9 months

6.0%

‐23.4 to 35.4

Standiford 2012

Enablement

‐28.6%

‐46.5 to ‐10.6

7 years

31.0%

6.8 to 55.3

Himmelberg 1991

Restriction

No data

“long‐standing”

301.2%

230.9 to 371.5

Skrlin 2011

Restriction

2 years

255.8%

194.7 to 316.9

CI: confidence interval

Figuras y tablas -
Table 7. Data from 5 studies about the effect of removal of interventions. The intended effect of all interventions was reduction in unnecessary antibiotic use
Table 8. Randomised controlled trials with microbial outcomes

Study

Design

Microbial outcome

Reason not in meta‐analysis

Annane 2013

RCT

Colonisation with MRSA (nasal swab) and GNRB (rectal swabs)

Not comparable with any other RCT

Bouza 2007

RCT

Number of cases of Clostridium difficile

Not in prescribing meta‐analysis

Lesprit 2013

RCT

Secondary infection and/or colonisation with multidrug‐resistant bacteria in the 6 months following randomisation

Not in prescribing meta‐analysis. It is impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics.

Palmay 2014

RCT

CDI and infection with antibiotic resistant organisms cases/1000 OBD

Not in prescribing meta‐analysis

Singh 2000

RCT

Number of participants with "antimicrobial resistance and/or superinfections" from randomisation until discharge from hospital

It is impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics.

CDI: Clostridium difficile infection
GNRB: gram‐negative resistant bacteria
MRSA: methicillin‐resistant Staphylococcus aureus
OBD: occupied bed day
RCT: randomised controlled trial

Figuras y tablas -
Table 8. Randomised controlled trials with microbial outcomes
Table 9. Microbial outcomes from 26 ITS studies from the prescribing meta‐analysis that include reliable data about prescribing outcomes at 6 months and microbial outcomes at 12 months postintervention

Prescribing target

Microbial outcome

N

Study ID

Cephalosporins

GNRB

8

Grohs 2014; Kim 2008; Knudsen 2014; Lee 2007; McNulty 1997; Meyer 2009; Petrikkos 2007; Tangdén 2011

MRSA

1

May 2000

Carbapenems

GNRB

1

Goldstein 2009

Fluoroquinolones

GNRB

3

Cook 2011b; Lafaurie 2012; Willemsen 2010

MRSA

1

Lafaurie 2012

High‐risk antibiotics

CDI

6

Aldeyab 2012; Chan 2011; Dancer 2013; Fowler 2007; Talpaert 2011; Valiquette 2007

GNRB

4

Buising 2008a; Chan 2011; Dancer 2013; Liebowitz 2008

MRSA

6

Aldeyab 2014; Ananda‐Rajah 2010; Chan 2011; Dancer 2013; Fowler 2007; Liebowitz 2008

Total antibiotic use

CDI

2

Cook 2011a; Jump 2012

MRSA

1

Cook 2011a

Vancomycin

VRE

1

Lautenbach 2003

Total microbial

34*

*Some studies had more than one microbial outcome, so the total is 34 microbial outcomes from 26 studies.

CDI: Clostridium difficile infection
GNRB: gram‐negative resistant bacteria
ITS: interrupted time series
MRSA: methicillin‐resistant Staphylococcus aureus
VRE: vancomycin‐resistant enterococci

Figuras y tablas -
Table 9. Microbial outcomes from 26 ITS studies from the prescribing meta‐analysis that include reliable data about prescribing outcomes at 6 months and microbial outcomes at 12 months postintervention
Comparison 1. Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Dichotomous outcomes, increase in desired practice Show forest plot

29

23394

Risk Difference (M‐H, Fixed, 95% CI)

0.15 [0.14, 0.16]

2 Dichotomous outcomes, all RCTs with results of cluster RCTs adjusted by inflation factor Show forest plot

29

5802

Risk Difference (M‐H, Fixed, 95% CI)

0.17 [0.15, 0.19]

3 Dichotomous outcomes, low or medium 'Risk of bias' studies only Show forest plot

15

13086

Risk Difference (M‐H, Fixed, 95% CI)

0.11 [0.10, 0.12]

4 Continuous outcomes, duration of all antibiotic treatment (days) Show forest plot

14

3318

Mean Difference (IV, Fixed, 95% CI)

‐1.95 [‐2.22, ‐1.67]

5 Continuous outcomes, duration of all antibiotic treatment with results of cluster RCTs adjusted by inflation factor Show forest plot

14

3318

Mean Difference (IV, Fixed, 95% CI)

‐1.95 [‐2.23, ‐1.67]

6 Continuous outcomes, low or medium 'Risk of bias' studies only Show forest plot

3

755

Mean Difference (IV, Fixed, 95% CI)

‐3.06 [‐3.76, ‐2.37]

7 Continuous outcome, consumption of targeted antibiotic only, standardised mean reduction (original outcome cost, days or DDD) Show forest plot

4

1053

Std. Mean Difference (IV, Fixed, 95% CI)

‐0.25 [‐0.37, ‐0.13]

Figuras y tablas -
Comparison 1. Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use
Comparison 2. Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mortality, all RCTs Show forest plot

28

15827

Risk Difference (M‐H, Fixed, 95% CI)

‐0.00 [‐0.01, 0.00]

2 Mortality, all RCTs with results of cluster RCTs adjusted by inflation factor Show forest plot

28

8332

Risk Difference (M‐H, Fixed, 95% CI)

‐0.01 [‐0.02, 0.01]

3 Mortality, low or medium 'Risk of bias' RCTs Show forest plot

8

6249

Risk Difference (M‐H, Fixed, 95% CI)

‐0.00 [‐0.02, 0.01]

4 Length of stay, all RCTs Show forest plot

15

3834

Mean Difference (IV, Fixed, 95% CI)

‐1.12 [‐1.54, ‐0.70]

5 Length of stay, all RCTs with results of cluster RCTs adjusted by inflation factor Show forest plot

15

3834

Mean Difference (IV, Fixed, 95% CI)

‐1.22 [‐1.68, ‐0.76]

6 Length of stay, low or medium 'Risk of bias' RCTs only Show forest plot

6

1731

Mean Difference (IV, Fixed, 95% CI)

‐0.85 [‐1.38, ‐0.32]

Figuras y tablas -
Comparison 2. Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use
Comparison 3. Adverse effects: Clinical outcomes of interventions targeting antibiotic choice

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mortality for trial patients Show forest plot

11

7658

Risk Difference (M‐H, Fixed, 95% CI)

‐0.00 [‐0.02, 0.01]

2 Length of stay for trial patients Show forest plot

7

2276

Mean Difference (IV, Fixed, 95% CI)

‐1.50 [‐2.16, ‐0.83]

Figuras y tablas -
Comparison 3. Adverse effects: Clinical outcomes of interventions targeting antibiotic choice
Comparison 4. Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mortality for trial patients Show forest plot

18

9173

Risk Difference (M‐H, Fixed, 95% CI)

‐0.00 [‐0.01, 0.01]

2 Length of stay for trial patients Show forest plot

8

1558

Mean Difference (IV, Fixed, 95% CI)

‐0.87 [‐1.42, ‐0.33]

Figuras y tablas -
Comparison 4. Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure
Comparison 5. Modifiers of intended effect: Comparison of enabling interventions with and without feedback

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Enablement with feedback Show forest plot

4

3747

Risk Difference (M‐H, Fixed, 95% CI)

0.19 [0.16, 0.22]

2 Enablement without feedback Show forest plot

7

1827

Risk Difference (M‐H, Fixed, 95% CI)

0.13 [0.09, 0.17]

Figuras y tablas -
Comparison 5. Modifiers of intended effect: Comparison of enabling interventions with and without feedback