Interventions visant à améliorer les pratiques de prescription d'antibiotiques aux patients hospitalisés
Referencias
References to studies included in this review
References to studies excluded from this review
Additional references
References to other published versions of this review
Characteristics of studies
Characteristics of included studies [ordered by study ID]
Ir a:
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in hospital CLINICAL PROBLEM: receiving treatment with target antibiotics SETTING: single university hospital in the USA | |
| Interventions | FORMAT, Interventions: educational meetings with dissemination of materials; audit and feedback; educational outreach by review and recommend change Intervention Functions: education; enablement; persuasion DELIVERER: pharmacist COMPARISON: 9 months' pre‐intervention. Usual care DESIRED CHANGE: reduce inappropriate | |
| Outcomes | PRESCRIBING: Choice: decrease in use of cefoxitin and cefamandole COST: total cost of 6 target antibiotics (calculated from data in Tables 1 and 2) | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Not stated. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper (comparison of means, uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine pharmacy systems database. |
| Free of other bias (ITS) ? | Low risk | Price of target antibiotics constant over the study period. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: dissemination of educational materials; educational outreach by review and recommend change; reminders (physical ‐ newsletter) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: pharmacist | |
| Outcomes | PRESCRIBING: Choice: reduce vancomycin prescribing and increase appropriate use of vancomycin COST: valid financial savings | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper (comparison of means, uncontrolled before and after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Unclear risk | Not clear, no information about changes in price of vancomycin over the study period. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all paediatricians and nurses in the ED | |
| Interventions | FORMAT, Interventions: audit and feedback; dissemination of educational materials; educational outreach by review and recommend change; reminders (physical ‐ posters and email) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: departmental physicians, nurses, and managers | |
| Outcomes | PRESCRIBING: exposure, % children treated with antibiotics CLINICAL: balancing, % admission rate, % return ED visit rate, ED length of stay (minutes) FINANCIAL: total cost per patient. No data about the intervention cost. | |
| Notes | FINANCIAL SUPPORT: Funding: Boston Children’s Hospital Department of Medicine Quality Improvement Publication (QIPub) grant. Competing Interest: none declared ADDITIONAL INFORMATION: care pathway is in a supplementary online file | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Electronic outcome data |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Electronic outcome data |
| Incomplete outcome data addressed (ITS) ? | Low risk | Electronic outcome data |
| Free of selected reporting (ITS) ? | Low risk | Electronic outcome data |
| Free of other bias (ITS) ? | Low risk | > 1 year of data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: audit and feedback; restrictive ‐ expert approval Intervention Functions: enablement, restriction DELIVERER: AMT | |
| Outcomes | PRESCRIBING: Choice: use of target antibiotics in DDD/100 OBD MICROBIAL: Clostridium difficile infections/100 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: Deanship of Scientific Research (DSR), King Abdulaziz University, Jeddah grant no. 7‐968‐D1432. Competing interest: none declared ADDITIONAL DATA: restriction policy is described in detail in an additional online file for this paper and in Conlon 2011. Microbial Risk of Bias: LOW, case definition Low, planned intervention Low, other infection control Low | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Changes in CDI screening policy and cleaning policy occurred between Phases 1 and 2 (Figure 1). |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Electronic data from pharmacy and microbiology |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Electronic data from pharmacy and microbiology |
| Incomplete outcome data addressed (ITS) ? | Low risk | Electronic data from pharmacy and microbiology |
| Free of selected reporting (ITS) ? | Low risk | Electronic data from pharmacy and microbiology |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: same as in Aldeyab 2012; this article provides additional microbial outcome data for impact on MRSA infections | |
| Outcomes | PRESCRIBING: same as in Aldeyab 2012 MICROBIAL: MRSA infections/100 OBD | |
| Notes | FINANCIAL SUPPORT: same as in Aldeyab 2012 ADDITIONAL DATA: restriction policy is described in detail in an additional online file for Aldeyab 2012 and in Conlon 2011 (additional studies) Microbial Risk of Bias: LOW, case definition Low, planned intervention Low, other infection control Low | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Data and segmented regression model of alcohol‐based hand rub included as a proxy measure for infection control practices. |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Electronic data from microbiology |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Electronic data from microbiology |
| Incomplete outcome data addressed (ITS) ? | Low risk | Electronic data from microbiology |
| Free of selected reporting (ITS) ? | Low risk | Electronic data from microbiology |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the medical‐surgical ICU | |
| Interventions | FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT | |
| Outcomes | PRESCRIBING: Choice: use of broad‐spectrum antibiotics in DDD/1000 OBD MICROBIAL: MRSA bacteraemia rate | |
| Notes | FINANCIAL SUPPORT: none declared. Competing Interest: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH, case definition Low, planned intervention Low, other infection control High. Infection control interventions close to antibiotic stewardship interventions clearly documented in Figure 1. | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Other changes are clearly documented in Figure 1. This includes an outbreak of Acinetobacter infection co‐incident with the stewardship intervention, which resulted in appointment of 2 infection control practitioners and associated interventions. The additional staff could have influenced prescribing outcome. |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention is point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Pharmacy and microbiology routine data |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Pharmacy and microbiology routine data |
| Incomplete outcome data addressed (ITS) ? | Low risk | Pharmacy and microbiology routine data |
| Free of selected reporting (ITS) ? | Low risk | Pharmacy and microbiology routine data |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in participating ICUs | |
| Interventions | FORMAT, Interventions: structural ‐ rapid testing of PCT with decision support algorithm Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physician POWER CALCULATION: yes, 140 participants in total (70 in each arm) would be needed (details in Appendix 3) | |
| Outcomes | PRESCRIBING: exposure, % receiving antibiotics at day 5 CLINICAL: mortality, length of ICU stay, length of hospital stay MICROBIAL: colonisation with MRSA (nasal swab) and GNRB (rectal swabs) | |
| Notes | FINANCIAL SUPPORT: Funding: commercial, Thermo Fisher B.R.A.H.M.S. France, a subsidiary of the maker of the PCT assay used in this study. Competing interests: none declared ADDITIONAL INFORMATION: supplementary online file has PCT algorithm, authors provided full study protocol (in French) Microbial Risk of Bias: MEDIUM (no data about infection control) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer generated |
| Allocation concealment (selection bias) | Low risk | Central allocation |
| Blinding (performance bias and detection bias) | High risk | PCT levels not reported on control participants. |
| Incomplete outcome data (attrition bias) | Low risk | No participants lost to follow‐up. |
| Selective reporting (reporting bias) | Low risk | No participants lost to follow‐up. |
| Other bias | High risk | Study stopped prematurely because of low recruitment. |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | PCT levels not reported on control participants. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: antibiotics dispensed to hospital wards for administration for therapy or prophylaxis SETTING: 1 university hospital in the UK | |
| Interventions | FORMAT, Interventions: educational meetings; dissemination of educational materials; educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: total use of Alert Antibiotics in DDD/1000 OBD FINANCIAL: cost of antibiotics adjusted for changes in price over the 4‐year study period. Cost of the Alert Antibiotic Monitoring intervention and of the setup and analysis of the ward antimicrobial supply database (Table 3) | |
| Notes | FINANCIAL SUPPORT: no financial support. Competing Interests: none declared ADDITIONAL DATA: email response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | "In 2000, the Antibiotic Subcommittee of Tayside University Hospitals Trust devised an Alert Antibiotic Policy to reduce inappropriate use of key antibiotics, targeted because they should be reserved for infections caused by organisms that are resistant to first line antimicrobials." There were no other changes in local or national policy likely to influence use of Alert Antibiotics. |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: segmented regression analysis with adjustment for autocorrelation and seasonality. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | "The aim of this study was to use routine data from the pharmacy stock control computer to evaluate this intervention". Sources and methods of data collection were the same before and after the intervention. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | "After evaluation of the intervention according to patient records and its shortcomings, we decided to use the pharmacy stock data. During the 4 year period of analysis no restriction policy for dispensing the Alert Antibiotics was implemented by the hospital pharmacy, therefore the pharmacy data about dispensed Alert Antibiotics would provide us with the best available independent indicator for evaluation of the intervention." |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | "Correcting for autocorrelation avoids underestimating standard errors Data about cost of antibiotics adjusted for price changes during study period. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians at 1 teaching hospital | |
| Interventions | FORMAT, Interventions: educational meetings; dissemination of educational materials; reminders ‐ circumstantial (order form triggered by receiving target antibiotic) and physical (posters) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT | |
| Outcomes | PRESCRIBING: Choice: inappropriate dosing intervals of cefazolin, clindamycin, and metronidazole | |
| Notes | FINANCIAL SUPPORT: Fund for Cooperative Innovation of Blue Cross of Massachusetts and the Massachusetts Hospital Association. Competing interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | No price changes in the target antibiotics during the study period. |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: segmented regression analysis. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: RCT stratified by type of infection Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians at 2 teaching hospitals, excluding ICUs PARTICIPANTS: a total of 102 inpatients, 51 intervention and 51 control CLINICAL PROBLEM: patients receiving IV ABs for at least 3 days, but excluded if in ICU or with uncontrolled infection or close to discharge SETTING: 2 tertiary‐care teaching hospitals in USA | |
| Interventions | FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: none reported | |
| Outcomes | PRESCRIBING: patients switched from parenteral to oral antibiotics or discontinuation of 1 or more antibiotics and mean IV antibiotic days COST: mean antibiotic costs CLINICAL: 30‐day re‐admission (total and infection‐related) and in‐hospital mortality | |
| Notes | FINANCIAL SUPPORT: Funding: Department of Pharmacy. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "Physicians of patients considered candidates for intervention were randomised to be either contacted by the clinical pharmacist ... or to be observed" |
| Allocation concealment (selection bias) | Unclear risk | Not stated |
| Blinding (performance bias and detection bias) | Unclear risk | Not stated |
| Incomplete outcome data (attrition bias) | Low risk | No problems found. |
| Selective reporting (reporting bias) | Low risk | No problems found. |
| Other bias | High risk | No power calculation. Prices of antibiotics unlikely to change over the 6‐month study period. |
| Baseline Outcomes similar? | Unclear risk | Not stated |
| Free of contamination? | Unclear risk | Not stated |
| Baseline characteristics similar? | Low risk | See Table 1 in study. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: IV antibiotics, restriction applied to carbapenems SETTING: a single university hospital in Argentina. Total use was compared for > 2 years before and after the intervention | |
| Interventions | FORMAT, Intervention 1: educational outreach by review and recommend change; restrictive ‐ compulsory order form Intervention Functions: education, enablement, persuasion, restriction Intervention 2: unavailability of antibiotics during a national financial crisis DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive (choice) | |
| Outcomes | PRESCRIBING: use of all IV antibiotics and carbapenems in DDD/1000 OBD CLINICAL: all‐cause inpatient mortality | |
| Notes | FUNDING: none. Competing Interests: 2 authors declared conflicts of interest for speaker and advisory board fees ADDITIONAL INFORMATION: no response from authors | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Intervention 1 was independent of other changes. The "crisis" (following the intervention) was a national economic crisis and will be reported separately in the review. |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data were obtained from pharmacy systems. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | Prescribing data were processed by the investigators to convert grams to DDD and identify only IV antibiotics. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine pharmacy data |
| Free of selected reporting (ITS) ? | Unclear risk | Processing of data has potential for selective outcome reporting. |
| Free of other bias (ITS) ? | Low risk | 3 years' data pre‐ and 2 years' data postintervention |
| Methods | STUDY DESIGN: Controlled ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in hospital CLINICAL PROBLEM: adults with community‐acquired pneumonia SETTING: 2 acute university hospitals in Scotland | |
| Interventions | FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials; reminders ‐ physical by posters and email Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: control hospital with no intervention DESIRED CHANGE: increase effective | |
| Outcomes | PRESCRIBING: Choice: % appropriate antibiotics within 4 h of admission COST: cost‐effectiveness, intervention cost, and estimated impact on mortality | |
| Notes | FINANCIAL SUPPORT: Funding: NHS Education Scotland and Chief Scientist Office, Scotland. Competing Interests: none declared ADDITIONAL DATA: email response from authors with additional information about intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | |
| Shape of effect pre‐specified (ITS) ? | Low risk | |
| Unlikely to affect data collection (ITS) ? | Low risk | |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | |
| Incomplete outcome data addressed (ITS) ? | Low risk | |
| Free of selected reporting (ITS) ? | Low risk | |
| Free of other bias (ITS) ? | High risk | |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the ICU (mixed medical/surgical) | |
| Interventions | FORMAT, Interventions: educational outreach by review and recommend change; restrictive ‐ compulsory order form Intervention Functions: education, enablement, persuasion, restriction DELIVERER: specialist physicians (ID) | |
| Outcomes | PRESCRIBING: Choice: use of cephalosporins in DDD/1000 OBD MICROBIAL: MRSA | |
| Notes | FINANCIAL SUPPORT: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: LOW | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine pharmacy data |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine pharmacy data |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine pharmacy data |
| Free of selected reporting (ITS) ? | Low risk | Routine pharmacy data |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention. Microbial Risk of Bias: case defintion done, planned intervention done, other infection control measures done. |
| Methods | STUDY DESIGN: unintended consequences, qualitative Risk of Bias: not assessed (qualitative study) | |
| Participants | PROVIDERS: 36 physicians | |
| Interventions | FORMAT, Intervention: audit and feedback; restriction by prior approval Intervention Functions: enablement, persuasion, restriction DELIVERER: AMT | |
| Outcomes | UNINTENDED CONSEQUENCES: problems with antibiotic policy and approval process identified through semi‐structured interviews with prescribers who had received feedback letters | |
| Notes | FINANCIAL SUPPORT: Funding: St Vincent’s Clinic Foundation Research Grant, annual Grant #3 and National Health and Medical Research Council program grant #568612. Competing Interests: none declared ADDITIONAL DATA: email from authors with additional data about the antibiotic policy and feedback | |
| Methods | STUDY DESIGN: unintended consequences, ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in general, gynaecological, orthopaedic, urological, and vascular surgery wards | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings; dissemination of antibiotic policy; reminders (physical ‐ posters in operating theatres) | |
| Outcomes | UNINTENDED CONSEQUENCES: % postoperative AKI before and after antibiotic policy change | |
| Notes | FINANCIAL SUPPORT: Funding: Scottish Government Healthcare Associated Infection Task Force. Competing Interests: none declared ADDITIONAL DATA: email response from authors but no additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from laboratory computer system (serum creatinine) |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from laboratory computer system |
| Incomplete outcome data addressed (ITS) ? | High risk | Completeness of pre‐ and postoperative creatinine data presented in full for all services (Table 2). There was a significant increase in testing after policy change in gynaecology. |
| Free of selected reporting (ITS) ? | Low risk | Data from laboratory computer system |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in hospital PARTICIPANTS: all patients in hospital CLINICAL PROBLEM: patients receiving vancomycin therapy SETTING: 1 university hospital in the USA | |
| Interventions | FORMAT, Interventions: educational meetings; dissemination of educational materials; educational outreach by academic detailing; reminders (physical ‐ posters and newsletter); restrictive ‐ expert approval Intervention Functions: education, environmental restructuring, persuasion, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: vancomycin doses/1000 OBD | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | > 12 months' pre‐ and postrestriction data |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper (comparison of means with t‐test, uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. Outcome data were collected from all participants. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: audit and feedback; educational outreach by academic detailing Intervention Functions: education, enablement, persuasion DELIVERER: AMT | |
| Outcomes | PRESCRIBING: Choice: antibiotic cost per patient day | |
| Notes | FINANCIAL SUPPORT: none. Competing Interests; none declared ADDITIONAL INFORMATION: no response from author | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Electronic data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Electronic data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Electronic data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Electronic data from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: physicians (paediatricians) in the hospital PARTICIPANTS: all paediatric patients in the hospital CLINICAL PROBLEM: children with infections requiring antibiotic therapy SETTING: 1 paediatric university hospital in Norway | |
| Interventions | FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials Intervention Functions: education, enablement DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: total antibiotic usage and usage of 5 specific groups of antibiotics in DDD/100 OBD | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Done, 3 years' pre‐intervention and 2 years' postintervention data |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper (run charts, Figure 1, with no statistical analysis). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | Changes in antibiotic price were documented with their contribution to reduction in cost over the study period (Table 1 in study). |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians and pharmacists in the Medical Service CLINICAL PROBLEM: patients receiving antibiotics | |
| Interventions | FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials; educational outreach by review and recommend change; reminders ‐ circumstantial, on rounds Intervention Functions: education, enablement, persuasion DELIVERER: AMT | |
| Outcomes | PRESCRIBING: Choice: drug use measured in RDD/100 OBD FINANCIAL: cost of intervention and impact on prescribing cost | |
| Notes | FINANCIAL SUPPORT: Funding: internal funds from the Department of Medicine and Federal Ministry of Health (BMG grant IIA5‐2011‐2511FSB340). Competing Interests: none declared | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Prescribing data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Prescribing data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Prescribing data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Prescribing data from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | > 24 months' data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: dissemination of educational materials; reminders ‐ circumstantial, on microbiology reports for positive blood cultures Intervention Functions: education, enablement, environmental restructuring DELIVERER: ID physician | |
| Outcomes | PRESCRIBING: Choice: average score per participant, with 0.5 points for each of 4 prescribing indicators, maximum score 2.0 per participant CLINICAL: not valid (mean mortality in pre‐ and postintervention phases) | |
| Notes | FINANCIAL SUPPORT: Funding: internal funds from the Department of Medicine and Federal Ministry of Health (BMG grant IIA5‐2011‐2511FSB340). Competing Interests: none declared ADDITIONAL DATA: the original paper reports average scores per participant for compliance, with 5 bundle elements of which only 2 were about antibiotic prescribing (Figure 2). The authors provided us with additional data about scores for the 2 prescribing elements in the bundle. | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | High risk | Prescribing outcomes were collected by the investigators. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | Prescribing outcomes were collected by the investigators. |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | Data are presented as % compliance per quarter, but it is not clear whether complete data were collected from all participants. |
| Free of selected reporting (ITS) ? | Unclear risk | Data are presented as % compliance per quarter, but it is not clear whether complete data were collected from all participants. |
| Free of other bias (ITS) ? | High risk | Only 9 months' data postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians caring for medical emergency patients | |
| Interventions | FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials; educational outreach by review and recommend change; reminders ‐ circumstantial, on rounds Intervention Functions: education, enablement, persuasion DELIVERER: AMT | |
| Outcomes | PRESCRIBING: Choice: drug use measured in RDD/100 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: internal funds from the Department of Medicine and Federal Ministry of Health (BMG grant IIA5‐2011‐2511FSB340). Competing Interests: none declared | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | > 24 months' data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians and pharmacists in the Medical Service CLINICAL PROBLEM: patients receiving antibiotics | |
| Interventions | FORMAT, Interventions: educational meetings; dissemination of educational materials; educational outreach by review and recommend change in ICU and for bacteraemic patients in other wards; reminders ‐ circumstantial, on rounds Intervention Functions: education, enablement, persuasion DELIVERER: AMT | |
| Outcomes | PRESCRIBING: Choice: target drug use measured in RDD/100 OBD. Exposure: impact on total anti‐infective use was measured | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | > 12 months' data pre‐ and postintervention |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the ICU | |
| Interventions | FORMAT, Interventions: reminders ‐ circumstantial; structural ‐ procalcitonin testing with decision support by treatment algorithm Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physicians (Anaesthesiology and Intensive Care) POWER CALCULATION: yes, 133 participants per study group (details in Appendix 3) | |
| Outcomes | PRESCRIBING: Exposure: days of antibiotic exposure per 1000 patient days CLINICAL: primary outcome measure 28‐day mortality, also 60‐day mortality, length of ICU stay, and length of hospital stay | |
| Notes | FINANCIAL SUPPORT: Funding: Assistance Publique‐Hopitaux de Paris, France and B.R.A.H.M.S, Germany. Competing Interests: 4 authors declared conflicts of interest from several pharmaceutical companies ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer‐generated randomisation sequence |
| Allocation concealment (selection bias) | Low risk | Assignment concealed before allocation. |
| Blinding (performance bias and detection bias) | High risk | Assignment not concealed postallocation. |
| Incomplete outcome data (attrition bias) | Low risk | Outcome data reported on 98% of participants in control and intervention groups. |
| Selective reporting (reporting bias) | Low risk | Outcome data reported fully on all included participants. |
| Other bias | High risk | Patients assigned to the trial were < 50% of all patients receiving antibiotics (630/1315). |
| Baseline Outcomes similar? | High risk | No data |
| Free of contamination? | Low risk | Procalcitonin only reported on intervention participants. |
| Baseline characteristics similar? | Low risk | ITS |
| Methods | STUDY DESIGN: RCT Risk of bias: HIGH | |
| Participants | PROVIDERS: ICU staff PARTICIPANTS: 297 patients with bloodstream infection in hospital, 109 control and 188 intervention CLINICAL PROBLEM: bacteraemia/fungaemia (bloodstream infection) SETTING: 1 university hospital in Spain | |
| Interventions | FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: microbiologists (specialist physicians) COMPARISON: usual care DESIRED CHANGE: reduce inappropriate POWER CALCULATION: no information about sample size | |
| Outcomes | PRESCRIBING: Choice: proportion of days on which adequate treatment received CLINICAL: Intended: length of stay, mortality | |
| Notes | FINANCIAL SUPPORT: Funding: Red Española de Investigación de Patología Infecciosa (REIPI C03‐14) and Fondo de Investigaciones Sanitarias of Spain (FIS 02‐1049). Competing Interest: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "We randomly classified the patients ... into 3 different group by means of a computer assisted random list" |
| Allocation concealment (selection bias) | Unclear risk | No information |
| Blinding (performance bias and detection bias) | High risk | Not possible with this study design |
| Incomplete outcome data (attrition bias) | Unclear risk | Not stated |
| Selective reporting (reporting bias) | Unclear risk | Not stated |
| Other bias | High risk | Not done, adequate prescription was defined by 7 criteria, some of which required clinical judgement. The reliability of the primary outcome measure was not assessed. |
| Baseline Outcomes similar? | Unclear risk | Not stated |
| Free of contamination? | High risk | All doctors in the hospital were distributed across all 3 study groups. |
| Baseline characteristics similar? | Unclear risk | Not stated |
| Methods | STUDY DESIGN: RCT Risk of bias: HIGH | |
| Participants | PROVIDERS: ICU staff PARTICIPANTS: 250 patients in the adult ICU, 167 intervention and 83 control CLINICAL PROBLEM: ventilator‐associated pneumonia with bacteria identified on gram stain of first tracheal aspirate SETTING: single general, teaching, and referral hospital in Spain | |
| Interventions | FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: microbiologists (specialist physicians) COMPARISON: usual care DESIRED CHANGE: reduce inappropriate POWER CALCULATION: no information about sample size | |
| Outcomes | PRESCRIBING: Exposure: mean days of therapy MICROBIAL: Clostridium difficile infection CLINICAL:Balancing: median days of fever and mechanical ventilation FINANCIAL: cost of antibiotics | |
| Notes | Microbial Risk of Bias HIGH: no case definition, no details of other infection control measures ADDITIONAL DATA: no response from authors to request for additional data FINANCIAL SUPPORT: Red Española de Investigación de Patología Infecciosas (REIPI) and Fondo de Investigación Sanitaria (FIS). The Spanish Ministry of Health (BEFI BF03/00237, to M.V.T.). Competing Interest: none declared | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer generated |
| Allocation concealment (selection bias) | Low risk | Computer generated |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all participants. |
| Selective reporting (reporting bias) | Unclear risk | No primary outcome measure identified. Defined daily dose of antibiotic therapy free from selective reporting, but other outcomes (e.g. % adequate days of antibiotic therapy) were not. |
| Other bias | High risk | High microbial risk of bias |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | ETEST results only available for intervention group. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: physicians in an adult haematology unit PARTICIPANTS: all patients with clinical problem CLINICAL PROBLEM: adult patients receiving treatment for haematological malignancy SETTING: adult haematology unit in a university hospital in the UK | |
| Interventions | FORMAT, Interventions: restrictive Intervention Functions: restriction by removal DELIVERER: specialist physician (microbiologist) COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: use of 4 principal IV antibiotics in patient days per month MICROBIAL: probability of remaining free of colonisation by GRE by weeks of exposure on the ward from date of first admission | |
| Notes | FINANCIAL SUPPORT: Funding: commercial, Wyeth Pharmaceuticals. Competing Interest: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Only 4 months' pre‐intervention data, so secular changes possible. |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: Kaplan‐Meier plot and log rank test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, screening protocol was the same throughout the study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, screening protocol was the same throughout the study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, screening protocol was the same throughout the study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, screening protocol was the same throughout the study period. |
| Free of other bias (ITS) ? | Low risk | Microbiology Risk of Bias Criteria: Case definition: DONE, colonisation by screening; Planned intervention: DONE; Other infection control, isolation, and IC practices: DONE, same throughout study. |
| Methods | STUDY DESIGN: NRT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in hospital CLINICAL PROBLEM: inappropriate antibiotic therapy SETTING: 1 university hospital in the Netherlands | |
| Interventions | FORMAT, Interventions: structural ‐ rapid microbiology laboratory testing Intervention Functions: environmental restructuring DELIVERER: specialist physician (microbiologist) COMPARISON: usual care DESIRED CHANGE: reduce inappropriate POWER CALCULATION: yes, 296 participants in each study arm (details in Appendix 3) | |
| Outcomes | PRESCRIBING: % of participants who receive appropriate treatment in first 48 h. Turnaround times for microbiology tests and results CLINICAL: intended clinical outcomes, total hospital mortality rate and length of hospital stay COST: valid financial savings | |
| Notes | FINANCIAL SUPPORT: Funding: commercial, bioMerieux and Stichting Zorg op Regionale ´ Grondslag (ZORG). Competing Interest: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Quasi‐randomised. "Patients were randomised on the basis of the sum of the day and month of their date of birth ... even numbers assigned to the control group ... odd number to the intervention group" |
| Allocation concealment (selection bias) | High risk | Allocation not concealed. |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | |
| Selective reporting (reporting bias) | Low risk | |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | Rapid reports only received by intervention group. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: audit and feedback; educational outreach by review and recommend change; restrictive ‐ expert approval and removal Intervention Functions: education, enablement, environmental, persuasion, restriction DELIVERER: AMT | |
| Outcomes | PRESCRIBING: Choice: use of restricted antibiotics in DDD/1000 OBD MICROBIAL: ARGNB (Escherichia coli,Pseudomonas aeruginosa); ARGPB (MRSA) | |
| Notes | FINANCIAL SUPPORT: Funding: National Health and Medical Research Council of Australia; Biotechnology Innovation Fund from the Commonwealth Government of Australia; Melbourne Health. Competing Interest: none declared Microbial Risk of Bias: LOW (case definition, planned intervention, and other infection control measures all low risk) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis is point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology systems |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology systems |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology systems |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology systems |
| Free of other bias (ITS) ? | Low risk | 5 years' pre‐ and 2 years' postintervention data |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the ED | |
| Interventions | FORMAT, Interventions 1: educational outreach by academic detailing; reminders ‐ physical, posters Intervention Functions: education, environmental restructuring, persuasion Intervention 2: structural ‐ computerised decision support Intervention Functions: enablement, environmental restructuring | |
| Outcomes | PRESCRIBING: Choice: % prescribing concordant with recommendation | |
| Notes | FINANCIAL SUPPORT: Funding: National Health and Medical Research Council of Australia. Competing Interest: none declared ADDITIONAL DATA: email response from authors with further details about intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Junior staff who were targets of the intervention rotated every 3 months. |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data collection was identical in all 3 phases. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | The nurse and physicians who collected data were not blinded to allocation. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data were collected from all eligible participants. |
| Free of selected reporting (ITS) ? | Low risk | Data were collected from all eligible participants. The accuracy of data collection was checked in a 5% sample of participants. |
| Free of other bias (ITS) ? | High risk | 1 year of data in pre‐ and Intervention 1 time series, but only 6 months' data for Intervention 2 |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in hospital PARTICIPANTS: all patients in hospital CLINICAL PROBLEM: receiving metronidazole SETTING: single university hospital in Canada | |
| Interventions | FORMAT, Interventions: educational meetings; dissemination of educational materials; reminders ‐ circumstantial, on rounds; restrictive ‐ review and make change Intervention Functions: education, enablement, restriction DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: % doses of metronidazole prescribed 12‐hourly | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Although the pre‐ and postintervention phases were only a 6‐month period, data from 1 year prior to the intervention were used to control for any seasonal variation in prescribing patterns. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: run charts with no statistical analysis. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, the analysis included all prescriptions for metronidazole. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in hospital PARTICIPANTS: 147 receiving aminoglycosides CLINICAL PROBLEM: patients receiving IV aminoglycosides SETTING: 1 hospital in the USA | |
| Interventions | FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: reduce inappropriate POWER CALCULATION: no information about sample size | |
| Outcomes | PRESCRIBING: Choice: aminoglycoside dosing and serum concentration CLINICAL: Intended: length of stay | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "Random numbers table used to assign 9 of 17 house staff teams to the intervention group. Patients allocated to intervention or control groups based on house staff team to which they were admitted. The other 8 teams were assigned as control groups" |
| Allocation concealment (selection bias) | Unclear risk | Not stated but unlikely: 9 house staff teams were in the intervention group, 8 control, groups swapped over after 4 months. |
| Blinding (performance bias and detection bias) | High risk | "Blinding as to patient status was not performed" |
| Incomplete outcome data (attrition bias) | Low risk | No problems found. |
| Selective reporting (reporting bias) | Low risk | No problems found. |
| Other bias | High risk | Unit of analysis error for length of stay. This was a cluster RCT, but length of stay was analysed at participant level. |
| Baseline Outcomes similar? | Unclear risk | Not measured before interventions. |
| Free of contamination? | High risk | Not done, 9 house staff teams were in the intervention group, 8 control, groups swapped over after 4 months. |
| Baseline characteristics similar? | Low risk | See Table 2 in paper. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in hospital PARTICIPANTS: all patients receiving IV fluoroquinolones CLINICAL PROBLEM: switch from IV fluoroquinolones to oral SETTING: 1 hospital in Belgium | |
| Interventions | FORMAT, Interventions: educational meetings; dissemination of guideline; reminders ‐ circumstantial and physical (pre‐printed note placed in patient notes when the patient fulfilled criteria for IV to oral switch). NB: the circumstantial reminder was only implemented on some wards (abdominal surgery, gastro‐enterology, and plastic surgery) over 2 months, and there are no reliable data to estimate the effect of this component. Intervention Functions: education, enablement (only for the circumstantial reminder), environmental restructuring (only for the circumstantial reminder) DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce inappropriate | |
| Outcomes | PRESCRIBING: Choice: % IV/(IV + oral) fluoroquinolone usage | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interest: none declared ADDITIONAL DATA: email from authors with further information about the intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data for ITS from pharmacy computer (Figure 1). Other data in Tables 2 and 3 not valid, UBA. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data for ITS from pharmacy computer (Figure 1). Other data in Tables 2 and 3 not valid, UBA. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data for ITS from pharmacy computer (Figure 1). Other data in Tables 2 and 3 not valid, UBA. |
| Free of selected reporting (ITS) ? | Low risk | Data for ITS from pharmacy computer (Figure 1). Other data in Tables 2 and 3 not valid, UBA. |
| Free of other bias (ITS) ? | Low risk | 21 months' pre‐ and 24 months' postintervention |
| Methods | STUDY DESIGN: unintended consequences, case control Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in adult medical and surgical units | |
| Interventions | FORMAT, Interventions: restrictive by review and make change, automatic stop order for prescriptions not meeting policy indications | |
| Outcomes | UNINTENDED CONSEQUENCES: proportion of nosocomial infections reported solely on the basis of a treating physician’s diagnosis during the endemic and epidemic periods (Table 1) | |
| Notes | ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding unlikely 2. Selection of participants into the study: Unclear, insufficient detail about selection of cases for the endemic and epidemic period 3. Measurement classification of interventions: Low, intervention status well defined, recorded at the time of intervention, and unaffected by knowledge of the outcome or risk of the outcome 4. Deviation from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Unclear, outcomes are reported as % with no numerator or denominator data 6. Measurement of outcome: High, outcome measure objective, but outcome assessors were aware of the intervention status, and the study does not report the actual number of cases 7. Selection of the reported result: High, reported effect selected from multiple measurements within the outcome domain FINANCIAL SUPPORT:Funding: no information. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: staff in a neonatal unit CLINICAL PROBLEM: requiring neonatal care SETTING: 1 neonatal care unit in a university hospital in Brazil | |
| Interventions | FORMAT: no valid prescribing data. Restrictive DELIVERER: specialist physician COMPARISON: usual carer DESIRED CHANGE: decrease exessive | |
| Outcomes | MICROBIAL: monthly incidence of Enterobacter cloacae infections | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | More than 1 year of data before and after intervention |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with logistic regression analysis of relationship between antibiotic prescribing and resistance. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Unclear risk | Not clear, no information about changes in sampling or testing protocol over study period. |
| Free of other bias (ITS) ? | High risk | Not done. Microbial Risk of Bias Criteria: Case definition: infection, monthly infections with E cloacae; Unplanned intervention: other infection control measures: barrier precautions, isolation of participants, and personal IC procedures fully described and same in both phases. |
| Methods | STUDY DESIGN: cluster RCT, service level Risk of Bias: HIGH | |
| Participants | PROVIDERS: all internal medicine teams in the hospital CLINICAL PROBLEM: patients receiving therapeutic piperacillin‐tazobactam, levofloxacin, or vancomycin SETTING: 1 hospital in the USA | |
| Interventions | Interventions: audit and feedback; dissemination of guidelines; educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease in excessive treatment POWER CALCULATION: assuming a baseline proportion of inappropriate use for target antimicrobials of 35% (with inappropriate‐use data based on preliminary‐usage data from Grady Memorial Hospital), review of at least 330 antimicrobial prescriptions in each arm would allow for detection of a 10% reduction in inappropriate antimicrobial use. | |
| Outcomes | PRESCRIBING: Choice: % appropriate | |
| Notes | FINANCIAL SUPPORT: Emory Medical Care Foundation; National Institutes of Health (UL1RR024992 to BCC, K12 RR017643 to MDK and HMB, K23 AI054371 to MDK, and UL1 RR025008 to HMB). Competing Interests: BCC reports was on the speakers’ bureau for Wyeth Pharmaceuticals. All other authors report no conflicts of interest. ADDITIONAL DATA: no additional data requested | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Each month, 6 internal medicine teams were randomly assigned to the intervention arm, and 6 teams were randomly assigned to the control group by means of a random number list. |
| Allocation concealment (selection bias) | High risk | Not concealed |
| Blinding (performance bias and detection bias) | High risk | Not blinded |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all participants. |
| Selective reporting (reporting bias) | Low risk | Outcomes reported on all participants. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | High risk | Doctors randomised to intervention were in the same hospital as control doctors. |
| Baseline characteristics similar? | Low risk | |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: no valid prescribing data. Educational outreach ‐ review and recommend change; educational meetings with dissemination of educational materials | |
| Outcomes | MICROBIAL: prevalence of Clostridium difficile, ceftazidime‐resistant Enterobacteriaceae, and MRSA FINANCIAL: cost of intervention | |
| Notes | FINANCIAL SUPPORT: Funding: institutional support. Competing Interests: none declared ADDITIONAL DATA: no additional data requested | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 3 years' pre‐intervention data |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: regression analysis with adjustment for autocorrelation. Analysis repeated by review team because of incomplete reporting of results. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Unclear risk | Not clear, no information about changes in sampling or testing protocol over study period. |
| Free of other bias (ITS) ? | Low risk | VRE isolation unlikely to have influenced C difficile or resistant gram‐negative bacteria. Microbial Risk of Bias Criteria: Planned intervention: DONE Implementation of antimicrobial management team in response to increase in use of target drugs. Case definition: DONE for C difficile infection (diarrhoea and toxin positive) or infection with clinical isolates of gram‐negative bacteria resistant to ceftazidime, or MRSA (CDC definition of nosocomial infection). Other infection control measures: DONE For C difficile contact precautions and procedures for cleansing equipment and patient care areas remained unchanged. Other infection control processes are not described in detail but may have changed during the study period (e.g. VRE isolation introduced after intervention). Data about VRE infections NOT RELIABLE: There were no cases in the pre‐intervention phase and none in the first 3 years postintervention, but there was an outbreak in the 4th and 5th postintervention years caused by admission of patients from other hospitals who were colonised with VRE. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: educational outreach by review and recommend change; restrictive ‐ expert approval required plus review and make change Intervention Functions: education, enablement, persuasion, restriction | |
| Outcomes | PRESCRIBING: Choice: DDD/1000 OBD of restricted antibiotics MICROBIAL: isolation rates Clostridium difficile, MRSA, and multidrug‐resistant gram‐negative bacteria | |
| Notes | ADDITIONAL DATA: no response from authors to request for additional data FINANCIAL SUPPORT: Funding: Chang Gung Memorial Hospital (Taoyuan, Taiwan) (grant CMRPG340236). Competing Interests: none declared Microbial Risk of Bias: HIGH (case definition clear, planned intervention but no data about infection control) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | States in discussion that biggest limitation was lack of external controls, but that is common to all ITS studies. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Unclear risk | DDD data from pharmacy computer, the same pre‐ and postintervention |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | DDD data from pharmacy computer, the same pre‐ and postintervention |
| Incomplete outcome data addressed (ITS) ? | Low risk | DDD data from pharmacy computer, the same pre‐ and postintervention |
| Free of selected reporting (ITS) ? | Low risk | DDD data from pharmacy computer, the same pre‐ and postintervention |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT Interventions: restrictive ‐ expert approval Intervention Functions: restriction | |
| Outcomes | PRESCRIBING: Choice: use of vancomycin in DDD/1000 OBD | |
| Notes | FINANCIAL SUPPORT: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Routine data from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | 21 months' pre‐ and 51 months' postintervention data |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT Interventions: dissemination of educational materials (guidelines) Intervention Functions: education | |
| Outcomes | PRESCRIBING: Exposure: total antibiotic use in DDD/100 OBD | |
| Notes | FINANCIAL SUPPORT: none. Competing Interests: none declared ADDITIONAL INFORMATION: authors provided additional detail about the antibiotic policy and confirmed that feedback was not used in this intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Routine data from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | > 18 months' data pre‐ and postintervention |
| Methods | STUDY DESIGN: Controlled ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in hospital PARTICIPANTS: all patients who qualified for fluoroquinolone therapy CLINICAL PROBLEM: infection with MRSA SETTING: 1 university hospital in France | |
| Interventions | FORMAT: no valid prescribing data. Restriction, educational meetings, and dissemination of educational materials DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | MICROBIAL: reduction of MRSA infections | |
| Notes | FINANCIAL SUPPORT: Funding: Programme Hospitalier de Recherche Clinique. Competing Interests: none declared ADDITIONAL DATA: no request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 1 year post‐ and 2 years' pre‐intervention data, so secular changes unlikely. Infection control protocols were unchanged pre‐ and postintervention. |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: the study is analysed as a CBA adjusting for confounders and slope and level. The ITS analyses are correct, but the results are not well reported. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Unclear risk | Not clear, no information about changes in sampling or testing protocol over study period. |
| Free of other bias (ITS) ? | Low risk | Microbial Risk of Bias Criteria: Planned intervention: DONE Case definition: DONE clear case definition of clinical infection: "A new case was defined as a case in a patient with no previous history of MRSA or ESBL‐EB colonization or infection who was infected with MRSA or ESBL‐EB no less than 48 h after hospital admission." Other infection control measures: DONE "The measures recommended by French national guidelines for the prevention of nosocomial infections were implemented in the 4 study hospitals several years before the study began" |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: educational meetings with dissemination of guidelines; educational outreach by review and recommend change | |
| Outcomes | PRESCRIBING: Choice: use of targeted antibiotics in DDD/1000 OBD | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Does not mention other changes apart from preceding Antimicrobial Stewardship Programme. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine pharmacy data used for outcome, so assume complete. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine pharmacy data used for outcome, so assume complete. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine pharmacy data used for outcome, so assume complete. |
| Free of selected reporting (ITS) ? | Low risk | |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: cluster RCT Risk of bias: MEDIUM | |
| Participants | PROVIDERS: physicians in hospital CLINICAL PROBLEM: suspected lower respiratory tract infection SETTING: 1 university hospital in Switzerland | |
| Interventions | FORMAT, Interventions: dissemination of educational materials; reminders ‐ circumstantial and physical (procalcitonin algorithm) triggered by prescribing antibiotics; structural Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 105 participants in each group | |
| Outcomes | PRESCRIBING: Choice: relative risk of antibiotic exposure measured in percentage and patient‐days CLINICAL: Balancing: length of stay; mortality | |
| Notes | FINANCIAL SUPPORT: Funding: B.R.A.H.M.S (Hennigsdorf, Germany) and Orgenium Laboratories (Turku, Finland) provided assay material and partial support of this investigator‐initiated project. Freiwillige Akademische Gesellschaft Basel, Switzerland; internal from the Department of Internal Medicine and the Divisions of Endocrinology and Pneumology. Competing Interests: BM served as consultant and received payments from B.R.A.H.M.S (the manufacturer of procalcitonin assays) to attend meetings related to the trial and for travel expenses, speaking engagements, or research ADDITIONAL DATA: email response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "We randomly assigned eligible patients ... according to a computer generated week wise randomisation scheme" |
| Allocation concealment (selection bias) | Unclear risk | "We randomly assigned eligible patients either standard antimicrobial therapy (standard group) or procalcitonin‐guided antimicrobial treatment (procalcitonin group) according to a computer‐generated week wise randomisation scheme". No information about concealment of allocation |
| Blinding (performance bias and detection bias) | Low risk | "Single blinded intervention trial" |
| Incomplete outcome data (attrition bias) | Low risk | Antibiotic data from all treated participants |
| Selective reporting (reporting bias) | Low risk | Objective outcome measure in all participants |
| Other bias | Low risk | No other apparent biases found. |
| Baseline Outcomes similar? | Unclear risk | Not stated |
| Free of contamination? | Low risk | Although same doctors treated participants in non‐intervention weeks, they did not have data about procalcitonin results. |
| Baseline characteristics similar? | Low risk | Done, Tables 1 and 2 in the original paper |
| Methods | STUDY DESIGN: RCT Risk of bias: MEDIUM | |
| Participants | PROVIDERS: physicians in hospital CLINICAL PROBLEM: suspected community‐acquired pneumonia SETTING: 1 university hospital in Switzerland | |
| Interventions | FORMAT, Interventions: dissemination of educational materials; reminders ‐ circumstantial and physical (procalcitonin algorithm) triggered by prescribing antibiotics; structural Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 150 participants in each group | |
| Outcomes | PRESCRIBING: Choice: relative risk of antibiotic exposure, total antibiotic use. Duration of antibiotic course CLINICAL: Balancing: mortality and length of hospital stay FINANCIAL: total antibiotic cost and cost per patient | |
| Notes | FINANCIAL SUPPORT: Funding: B.R.A.H.M.S (Hennigsdorf, Germany), Pfizer (Schweiz AG), and Mepha (Schweiz AG) was used for assay material and salaries of technical personnel; internal from Departments of Internal Medicine and Emergency Medicine, the Stiftung Forschung Infektionskrankheiten (SFI), and Departments of Endocrinology and Pulmonary Medicine, University Hospital Basel, Switzerland. Competing Interests: 2 authors received payments from B.R.A.H.M.S AG, the manufacturer of the procalcitonin assay. ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "Patients were randomly assigned to one of the two groups by sealed opaque envelopes", no information about generation of randomisation sequence |
| Allocation concealment (selection bias) | Low risk | "Sealed opaque envelopes" |
| Blinding (performance bias and detection bias) | Low risk | |
| Incomplete outcome data (attrition bias) | Low risk | |
| Selective reporting (reporting bias) | Low risk | |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | Same doctors in the intervention and control weeks, but they did not have access to procalcitonin results in the control weeks. |
| Baseline characteristics similar? | Low risk | Done, Table 1 in the original paper |
| Methods | STUDY DESIGN: CBA Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in hospital | |
| Interventions | FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials ‐ pack including guideline and literature review Intervention Functions: education, enablement | |
| Outcomes | PRESCRIBING: Choice: process measures sputum and blood cultures within 4 hours, antibiotics within 4 hours, first antibiotic in emergency room | |
| Notes | FINANCIAL SUPPORT: Funding: contract 500‐99‐P619 "Utilization and Quality Control Peer Review Organization for the State of Oklahoma" from the Centers for Medicare and Medicaid Services. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Control cohort study (CBA) |
| Allocation concealment (selection bias) | High risk | Control cohort study (CBA) |
| Blinding (performance bias and detection bias) | High risk | Control cohort study (CBA) |
| Incomplete outcome data (attrition bias) | Low risk | Objective primary outcome collected on all participants. |
| Selective reporting (reporting bias) | Low risk | Objective primary outcome collected on all participants. |
| Other bias | Low risk | No other apparent biases found. |
| Baseline Outcomes similar? | Low risk | Tables 1 and 2 |
| Free of contamination? | Low risk | Intervention and control were at different sites. |
| Baseline characteristics similar? | Low risk | Tables 3 and 4 |
| Methods | STUDY DESIGN: NRT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: structural ‐ rapid laboratory testing for meticillin resistance | |
| Outcomes | PRESCRIBING: Choice: % compliance with guideline recommended use of vancomycin | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL INFORMATION: the authors confirmed that this intervention did not include feedback to participants | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Odd versus even hospital number |
| Allocation concealment (selection bias) | Unclear risk | "Mode of allocation was concealed from the clinicians", but unclear how this was achieved. |
| Blinding (performance bias and detection bias) | High risk | Open study |
| Incomplete outcome data (attrition bias) | Low risk | Primary outcome reported on all participants. |
| Selective reporting (reporting bias) | Unclear risk | Primary outcome reported on all participants. Authors did a secondary analysis excluding participants with penicillin allergy, but this was not prespecified. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Unclear risk | Clinicians received results verbally and electronically, so it is likely that they were aware of the intervention, which may have influenced their management of other participants. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: no reliable prescribing data. Restriction by expert approval DELIVERER: specialist physician | |
| Outcomes | MICROBIAL: cases of Clostridium difficile‐associated diarrhoea per quarter (ITS data). Prevalence of clindamycin‐resistant C difficile | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Done, infection control measures fully described and same in both phases. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with t‐test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Unclear risk | Not clear, no information about changes in sampling or testing protocol over study period |
| Free of other bias (ITS) ? | High risk | NOT DONE Microbial Risk of Bias Criteria: Planned intervention: NOT DONE; Case definition: DONE infection, diarrhoea, and toxin positive Other infection control measures: DONE barrier precautions, isolation of participants with C difficile‐associated diarrhoea, and personal IC procedures fully described and same in both phases. |
| Methods | STUDY DESIGN: unintended consequences, cohort study Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians prescribing vancomycin | |
| Interventions | FORMAT, Interventions: reminders (circumstantial and physical) stickers in medical records on day 3 warning of impending stop order; restrictive: stop order if approval not obtained | |
| Outcomes | UNINTENDED CONSEQUENCES: interruption of vancomycin treatment | |
| Notes | ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding unlikely 2. Selection of participants into the study: Low, selection into the study unrelated to intervention (sticker in notes) or outcome 3. Measurement classification of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome 4. Deviations from intended interventions: Low, the study was designed to detect intervention failure (no warning sticker) 5. Missing data: Low, outcome data and intervention status complete on all 120 participants 6. Measurement of outcome: Low, outcome measure objective and unaffected by intervention status 7. Selection of the reported result: Low, reported effect predefined FINANCIAL SUPPORT: Funding: none. Competing Interests: EL received research support from Merck Pharmaceuticals and Ortho‐McNeil Pharmaceuticals. All other authors reported no conflicts of interest. | |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: educational outreach by review and recommend change | |
| Outcomes | PRESCRIBING: Exposure: total use of all antibiotics in DDD/1000 OBD MICROBIAL: Clostridium difficile and MRSA infections/1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: PPC is a member of the speakers’ bureau of Pfizer, Astellas, and Merck. PPC has received research funding from GlaxoSmithKline, Merck, Gilead, Pfizer, and Bristol‐Myers Squibb. All other authors have none to declare. ADDITIONAL DATA: email response from authors with additional data about intervention Microbial Risk of Bias HIGH: case definition low; planned intervention, other infection control high ‐ new policy for screening and isolation of MRSA introduced just before prescribing intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Routine data from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | 2 years' pre‐ and postintervention data |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Intervention 1 component: educational outreach by review and recommend change Intervention 2 component: restrictive by expert approval Intervention 2 function: restriction | |
| Outcomes | PRESCRIBING: Choice: use of ciprofloxacin in DDD/1000 OBD MICROBIAL: infections with ARGNB ‐ % carbapenem resistant Pseudomonas aeruginosa | |
| Notes | FINANCIAL SUPPORT: Funding: commercial, grant from Merck & Co., Inc. Competing Interests: PC is a member of the speakers’ bureau of Merck and Astellas. He has received research support from Merck, Gilead, and Pfizer. ADDITIONAL DATA: email response from authors with additional data about intervention Microbial Risk of Bias HIGH case definition low; planned intervention, other infection control high ‐ change in screening and isolation for MRSA just before prescribing intervention may have impacted on transmission of P aeruginosa | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Routine data from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | 5 years' pre‐ and 4 years' postintervention data |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Intervention 1: dissemination of educational materials Intervention 1 function: education Intervention 2: reminders ‐ physical, questionnaire about guideline compliance, distributed once | |
| Outcomes | PRESCRIBING: Choice: % guideline compliance | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL INFORMATION: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | This and all other ROB criteria are for interventions 1 and 2 only. Intervention 3 and 4 could not be evaluated because they are too close together and also coincided with an influenza epidemic. Neither intervention 3 nor intervention 4 meets the EPOC minimum criteria for ITS. There are insufficient data to adjust for seasonal effects, and the target condition (pneumonia) has large seasonal variation. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | High risk | Data collection was different in the postintervention phase (see below). |
| Knowledge of the allocation adequately prevented(ITS)? | Unclear risk | Compliance to therapy was assessed with a "computerised algorithm". However, the criteria for guideline adherence presented in the supplementary materials (Table S2) would require chart review, unless the hospitals had very sophisticated electronic patient records, which is not stated. The fact that patients were excluded because of "incomplete files" suggests that chart review was required, so knowledge of the allocated interventions could not be adequately prevented. |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | The 477 included participants had complete data for assessment of outcomes. 5 patients were excluded because of incomplete patient records. |
| Free of selected reporting (ITS) ? | Low risk | |
| Free of other bias (ITS) ? | Unclear risk | Insufficient data to account for seasonal effects. Although information about guideline compliance is reported for 2 hospitals, the ITS in Figure 1 only has data from 1 hospital (UZL). The data for the second hospital (ZOL) are actually a UBA and have been excluded. |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: excessive prescribing of antibiotics SETTING: 1 military teaching hospital in USA | |
| Interventions | FORMAT, Interventions: educational outreach by review and recommend change FORMAT: Persuasive: educational outreach ‐ review and recommend change DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: "Since this was an explanatory study, no a priori estimates of effect size were available to perform power and sample size calculations." The goal was to have 180 participants in the trial. | |
| Outcomes | PRESCRIBING: Choice: antibiotic use (DDDs and days of treatment) CLINICAL: Balancing: clinical outcomes, mortality, and re‐admission | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer generated |
| Allocation concealment (selection bias) | Unclear risk | 21 of 73 patients considered for enrolment were excluded, but it is not clear if this was pre‐ or postrandomisation. The number of participants in the study group was 14, versus 38 in the control group, with no justification for the unequal numbers. |
| Blinding (performance bias and detection bias) | Low risk | Primary outcome data from pharmacy computer |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all participants. |
| Selective reporting (reporting bias) | Low risk | Outcomes reported on all participants. |
| Other bias | High risk | Aim was to enrol 180 patients, but only 72 patients were identified, and 21 of them were excluded. |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | High risk | Education intervention with study and control in same hospital |
| Baseline characteristics similar? | Low risk | |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: restrictive | |
| Outcomes | PRESCRIBING: Choice: use of cephalosporins and ciprofloxacin in DDD/1000 OBD MICROBIAL: CDI, MRSA, and resistant gram‐negative bacteria | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: SD received financial support for attending conferences by Janssen‐Cilag, Pfizer, and Novartis ADDITIONAL DATA: authors provided additional details about the intervention, including information about regular feedback to participants that was not in the original paper Microbial Risk of Bias LOW case definition Low, planned intervention Low, other infection control practices Low | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | 9 months' data pre‐restriction includes an additional persuasive intervention 7 months' pre‐restriction; effect cannot be assessed because of insufficient pre‐intervention data. |
| Analysed appropriately (ITS) ? | Low risk | Analysed by correlation and time‐lag modelling, but re‐analysed as segmented regression analysis. |
| Shape of effect pre‐specified (ITS) ? | Low risk | |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine data from microbiology and pharmacy computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine data from microbiology and pharmacy computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine data from microbiology and pharmacy computers |
| Free of selected reporting (ITS) ? | Low risk | Routine data from microbiology and pharmacy computers |
| Free of other bias (ITS) ? | High risk | Only 9 months' pre‐intervention data |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: physicians on a paediatric ICU | |
| Interventions | FORMAT: No valid prescribing data. Restrictive: change in antibiotic policy from gentamicin to amikacin DELIVERER: specialist physician | |
| Outcomes | MICROBIAL: monthly incidence of infection with multiresistant Enterobacter cloacae | |
| Notes | FINANCIAL SUPPORT: Funding: grant from D.R.E.D. (Direction de la Recherche et des Etudes Doctorales). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Only 7 months' pre‐intervention data, so secular/seasonal changes possible. Very complex case definition with no information about how this was applied reliably across the pre‐ and postintervention periods. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with t‐test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Unclear risk | Case definition included clinical interpretation. |
| Knowledge of the allocation adequately prevented(ITS)? | Unclear risk | NOT CLEAR because of case definition |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | Availability of all data required for the case definition not documented. |
| Free of selected reporting (ITS) ? | Unclear risk | Not clear, no information about changes in sampling or testing protocol over study period. |
| Free of other bias (ITS) ? | High risk | Microbial outcome risk of bias: Unplanned intervention: implementation of change in response to emergence of gentamicin‐resistant E cloacae; Case definition:infection from clinical or screening isolates combined with 7 clinical criteria and 5 additional laboratory criteria assessed by a resident paediatrician and a consultant microbiologist and verified by a consultant paediatrician. Reliability of this outcome measure not clear. Other infection control measures: well documented, no changes during the study period |
| Methods | STUDY DESIGN: CBA Risk of Bias: HIGH | |
| Participants | PROVIDERS: all inpatient and outpatient services in the state of Utah | |
| Interventions | FORMAT: no valid prescribing data. Reminder; educational outreach ‐ academic detailing; and educational meetings or dissemination of educational material DELIVERER: AMT | |
| Outcomes | CLINICAL: Intended: 30‐day mortality and length of stay | |
| Notes | FINANCIAL SUPPORT: supported by HealthInsight and Intermountain Healthcare. The analyses upon which this publication is based were performed under contract number 500 –96‐P604, entitled “Utilization and Quality Control Peer Review Organization for the State of Utah”, sponsored by the Health Care Financing Administration (HCFA), Department of Health and Human Services. This article is a direct result of the Health Care Quality Improvement Program initiated by HCFA, which has encouraged identification of quality improvement projects derived from analysis of patterns of care, and therefore required no special funding on the part of the contractor. Conflict of interest: no information ADDITIONAL DATA: authors provided additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | CBA |
| Allocation concealment (selection bias) | High risk | CBA |
| Blinding (performance bias and detection bias) | High risk | CBA |
| Incomplete outcome data (attrition bias) | Low risk | Objective outcome measure collected on all participants. |
| Selective reporting (reporting bias) | Low risk | Objective outcome measure collected on all participants. |
| Other bias | Low risk | No other apparent biases found. |
| Baseline Outcomes similar? | Low risk | Table 1 |
| Free of contamination? | Unclear risk | NOT CLEAR, some hospitals had both intervention and control physicians. Intermountain Healthcare provides 50% of regional health care delivery in Utah. In rural IHC hospitals, 90% of pneumonia admissions were cared for by IHC‐affiliated physicians, whereas in urban IHC hospitals only 44% of pneumonia admissions were cared for by IHC‐affiliated physicians. Non‐affiliated physicians caring for patients at IHC hospitals may have been influenced by guideline implementation at these hospitals. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: CBA Risk of bias: HIGH | |
| Participants | PROVIDERS: all physicians in hospital CLINICAL PROBLEM: admitted with community‐acquired pneumonia SETTING: 35 hospitals in Utah, USA (16 from Intermountain Healthcare and 19 from other systems) | |
| Interventions | FORMAT: no valid prescribing data. Reminder; educational outreach by academic detailing; and educational meetings with dissemination of educational materials DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: increase effective | |
| Outcomes | CLINICAL: Intended: 30‐day mortality, LOS, and 30‐day re‐admission | |
| Notes | FINANCIAL SUPPORT: this study was funded by the Deseret Foundation and HealthInsight, Salt Lake City. The authors have no relevant conflicts of interest to report. ADDITIONAL DATA: authors provided additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | CBA |
| Allocation concealment (selection bias) | High risk | CBA |
| Blinding (performance bias and detection bias) | High risk | CBA |
| Incomplete outcome data (attrition bias) | Low risk | Electronic record linkage used. |
| Selective reporting (reporting bias) | Low risk | 30‐day mortality was primary outcome. |
| Other bias | Low risk | Objective primary outcome measure |
| Baseline Outcomes similar? | Low risk | Table 3 |
| Free of contamination? | Low risk | NOT CLEAR, some hospitals had both intervention and control physicians. The 100,000 annual inpatient admissions of Intermountain Healthcare represent almost one‐half of Utah hospital admissions. Intermountain Healthcare has an employed physician group and several non‐Medicare health maintenance organisation insurance plans, but many non‐employed physicians and non‐health maintenance organisation patients also utilise its facilities. Non‐affiliated physicians caring for patients at Intermountain Healthcare hospitals may have been influenced by guideline implementation at these hospitals. |
| Baseline characteristics similar? | Unclear risk | Not stated |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: no valid prescribing data. Audit and feedback; reminders; and educational meetings with dissemination of educational materials DELIVERER: AMT | |
| Outcomes | CLINICAL: Intended: length of stay FINANCIAL: charge per case of nursing home‐acquired pneumonia | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | < 1 year data pre‐ and postintervention, so seasonal trends cannot be excluded. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Patient administration system |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Patient administration system |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | No explicit statement about complete follow‐up |
| Free of selected reporting (ITS) ? | Low risk | |
| Free of other bias (ITS) ? | Low risk | |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT, Interventions: structural ‐ introduction of procalcitonin testing with decision support algorithm DELIVERER: respiratory physicians POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Exposure: % participants treated and duration of antibiotic treatment CLINICAL: Balancing: mortality, length of stay, duration of mechanical ventilation | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response to request from authors | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer‐generated numbers |
| Allocation concealment (selection bias) | Low risk | Computer‐generated numbers |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all participants. |
| Selective reporting (reporting bias) | Low risk | Outcomes reported on all participants. |
| Other bias | High risk | No power calculation |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | Procalcitonin results only available for intervention participants. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: physicians assigned to the 7 services | |
| Interventions | FORMAT: Interventions: educational outreach ‐ review and recommend change Intervention Functions: enablement, persuasion DELIVERER: pharmacist POWER CALCULATION: yes, 330 participants, 165 in each group. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Choice: percentage of cefotaxime prescriptions that were consistent with guideline for both indication and dosage | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "Randomised on a one to one basis via a computer generated list" |
| Allocation concealment (selection bias) | Low risk | Randomisations carried out in central pharmacy and "telephone on a consecutive basis". |
| Blinding (performance bias and detection bias) | High risk | Not done, acknowledged as a limitation by authors on page 179. |
| Incomplete outcome data (attrition bias) | Low risk | See Table 3; all participants included |
| Selective reporting (reporting bias) | Low risk | See Table 3; all participants included |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | High risk | Control participants were managed by the same physicians as intervention participants. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians involved in vascular surgery | |
| Interventions | FORMAT: no valid prescribing data. Surgical Care Improvement Project core measures with financial incentives implemented in 2006 | |
| Outcomes | PRESCRIBING: no data CLINICAL: inpatient surgical‐site infection | |
| Notes | FINANCIAL SUPPORT: no funding. Competing Interests: none declared ADDIDIONAL DATA: authors responded to request but had no additional relevant data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | No data about antibiotic prescribing |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Unclear risk | Outcome relied on ICD discharge coding to identify surgical‐site infection, may have been influenced by financial incentives to meet SCIP targets. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Electronic outcome data |
| Incomplete outcome data addressed (ITS) ? | High risk | Outcome data were restricted to inpatient coding, but most surgical‐site infections likely to present postdischarge. |
| Free of selected reporting (ITS) ? | Low risk | Electronic outcome data |
| Free of other bias (ITS) ? | Low risk | > 1 year of pre‐ and postintervention data |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians, pharmacists, and nurses in surgical department | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings with dissemination of educational materials; educational outreach by academic detailing; reminders (physical, posters, intranet, and faxes to physicians) | |
| Outcomes | PRESCRIBING: Exposure: % participants with prophylaxis discontinued within 24 h of surgery | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Electronic outcome data |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Electronic outcome data |
| Incomplete outcome data addressed (ITS) ? | Low risk | Electronic outcome data |
| Free of selected reporting (ITS) ? | Low risk | Electronic outcome data |
| Free of other bias (ITS) ? | High risk | 10 months' pre‐ and 12 months' postintervention data |
| Methods | STUDY DESIGN: unintended consequences, cohort study Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT, Interventions: restrictive by review and make change targeted at ordering of CBC and CRP tests | |
| Outcomes | UNINTENDED CONSEQUENCES: time to first antibiotic dose and complications (requirement for catecholamine treatment and/or mechanical ventilation, meningitis, or death) | |
| Notes | ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding unlikely 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement classification of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome 4. Deviations from intended interventions: Low, the study demonstrated large reduction in CBC (30%) and CRP (91%) 5. Missing data: Low, outcome data and intervention status complete on all 222 participants 6. Measurement of outcome: Low, outcome measure objective and unaffected by intervention status 7. Selection of the reported result: Low, reported outcomes predefined and measured from routine data FINANCIAL SUPPORT: Funding: SICPA Foundation and the Société Académique Vaudoise. Competing Interests: none declared ADDITIONAL DATA: email from authors with additional data about intervention | |
| Methods | STUDY DESIGN: CITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the critical care team CLINICAL PROBLEM: decrease use of broad‐spectrum antibiotics in critical care patients SETTING: 1 tertiary‐care centre in Ontario, Canada | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: use of targeted broad‐spectrum antibiotics (days of therapy/1000 OBD) | |
| Notes | FINANCIAL SUPPORT: Funding: Canadian Institutes of Health Research, Ontario Ministry of Health, and Long Term Care Academic Health Services Centre Innovation Award. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Done. October to August both pre‐ and postintervention |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | No, the intervention was open to all participants and prescribers, difficult to conceal. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, Figures 1 and 2 |
| Free of other bias (ITS) ? | Low risk | Done, no other apparent biases |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all paediatric physicians | |
| Interventions | FORMAT: Interventions: structural ‐ rapid testing for procalcitonin and decision support algorithm POWER CALCULATION: yes, 76 participants in each group. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: % started on antibiotics and % children treated for > 10 days CLINICAL: length of stay, duration of fever, antibiotic adverse effects | |
| Notes | FINANCIAL SUPPORT: Funding: Italian Ministry of Health (Bando Giovani Ricercatori 2007). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer generated |
| Allocation concealment (selection bias) | Low risk | Sealed envelopes |
| Blinding (performance bias and detection bias) | Low risk | PCT levels only reported on intervention participants. |
| Incomplete outcome data (attrition bias) | Low risk | 319 randomised, consent was withdrawn during the study in 9 cases (3% participants, 5 in the PCT group and 4 in the control group). Outcomes reported on all participants (Tables 2‐3). All 310 children came to the planned follow‐up visits. |
| Selective reporting (reporting bias) | Low risk | Outcomes reported on all 310 participants (Tables 2‐3). All 310 children came to the planned follow‐up visits. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | PCT levels only reported on intervention participants. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: physicians in Obstetrics & Gynaecology | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines; reminders (circumstantial, on the structured order form for intravenous antibiotics, which was triggered for every order for IV antibiotics); restriction by expert approval and by removal DELIVERER: department physician COMPARISON: usual care | |
| Outcomes | PRESCRIBING: relative use of cefazolin or cefoxitin in Caesarean sections that received < 5 g of either drug perioperatively | |
| Notes | FINANCIAL SUPPORT: Funding: Beth Israel Hospital, Boston, Massachusetts and Fund for Cooperative Innovation of Blue Cross of Massachusetts and the Massachusetts Hospital Association. Competing Interests: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Only 9 months pre‐intervention data, so secular/seasonal changes possible. |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper, segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | Antibiotic costs adjusted to 1986 prices over the whole study period. |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: no valid prescribing outcome data. Educational outreach (review and recommend change) SAMPLE SIZE: 571 intervention, 614 control POWER CALCULATION: no power calculation. No adjustment for intracluster correlation | |
| Outcomes | PRESCRIBING: Choice but no valid outcome data (% adherence with recommendations, but no data about antibiotic use in terms of choice, route, or duration of treatment) CLINICAL: Balancing: length of stay and % treatment failure | |
| Notes | FINANCIAL SUPPORT: Funding: Fondo de Investigaciones Sanitarias (FIS PI06/90094), and Instituto de Formación e Investigación Marqués de Valdecilla (IFIMAV) (API 06/03). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer |
| Allocation concealment (selection bias) | High risk | Randomisation stratified by clinical units, not blinded. Participants were randomised by groups (stratified randomisation by clinical units) to intervention or non‐intervention using the EPIDAT 3.1 programme (Dirección Xeral de Saúde Pública, Xunta de Galicia & Organización Panamericana de la Salud. Santiago de Compostela, Coruña, Spain, 2003). |
| Blinding (performance bias and detection bias) | High risk | Randomisation not blinded |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all participants. |
| Selective reporting (reporting bias) | Unclear risk | The primary outcome (clinical failure) was complex and not entirely objective. |
| Other bias | High risk | Unit of analysis error, no adjustment for intracluster correlation |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | 19 participants in the control group were excluded because they had ID consultation. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: cluster RCT, service level Risk of bias: HIGH | |
| Participants | PROVIDERS: all physicians in hospitals PARTICIPANTS: 608 patients with community‐acquired pneumonia (263 intervention, 325 control), 7 clusters (sites) CLINICAL PROBLEM: duration of IV antibiotic therapy and LOS SETTING: 7 nonprofit hospitals in Pittsburgh, Pennsylvania, USA | |
| Interventions | FORMAT: Interventions: dissemination of educational materials, educational outreach by review and recommend change; reminders (circumstantial, physical, detail sheets in physician notes for patients with community‐acquired pneumonia and verbal, telephone calls); restrictive; structural DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 600 participants in total. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Choice: duration of IV antibiotic therapy CLINICAL: intended clinical outcomes, mortality, re‐admission | |
| Notes | FINANCIAL SUPPORT: Funding: Agency for Healthcare Research and Quality and the National Institute of Allergy and Infectious Diseases (HS08282), Robert Wood Johnson Foundation. Competing Interests: no statement ADDITIONAL DATA: authors provided additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Physician groups were randomly assigned after stratification for practice type, group size, and patient volume, but details not clear. |
| Allocation concealment (selection bias) | High risk | |
| Blinding (performance bias and detection bias) | Low risk | |
| Incomplete outcome data (attrition bias) | Low risk | |
| Selective reporting (reporting bias) | Low risk | |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data about LOS prior to intervention |
| Free of contamination? | Low risk | |
| Baseline characteristics similar? | Low risk | |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: prescribing of cefuroxime and quinolones SETTING: 1 hospital in the UK | |
| Interventions | FORMAT: Interventions: dissemination of guideline DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: use of cefuroxime and ciprofloxacin (DDD/Finished Consultant Episode ratio) | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | No mention of any other changes, although little information given. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done. Intervention point was clear. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done. Outcomes are objective. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done. Figures 1 and 2 |
| Free of other bias (ITS) ? | Low risk | Done. No other bias apparent. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: physicians in the hospital CLINICAL PROBLEM: Clostridium difficile infection in the elderly SETTING: 3 acute medical wards for the elderly in 1 university hospital in the UK | |
| Interventions | FORMAT: Interventions: audit and feedback, dissemination of guideline; reminders (physical, laminated pocket version of guideline) Intervention Functions: education, enablement, environmental restructuring DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce inappropriate | |
| Outcomes | PRESCRIBING: Choice: monthly use of target antibiotics MICROBIAL: monthly count of cases of CDI | |
| Notes | FINANCIAL SUPPORT: no funding. Competing Interests: none declared ADDITIONAL DATA: email response from authors but no additional data Microbial Risk of Bias LOW: Planned intervention: Low Case definition: Low, National definition. Other infection control measures: Low | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Ongoing audit and feedback |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done. Point of analysis is point of the intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | No, not possible |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems so unlikely to be incomplete. |
| Free of selected reporting (ITS) ? | Low risk | Done, Figures 3 and 4 |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: physicians in neonatal units CLINICAL PROBLEM: suspected bacterial infection SETTING: 8 centres in 5 countries (Australia, Austria, Belgium, Germany, Sweden) | |
| Interventions | FORMAT, Interventions: dissemination of guideline; structural, introduction of testing for C‐reactive protein and interleukin‐8 with decision support algorithm DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, total of 1150 participants. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: number of newborn infants who received antibiotic therapy | |
| Notes | FINANCIAL SUPPORT: Funding: grant P.575 from the Center for Applied Clinical Studies of the University of Ulm and Swedish Research Council. DPC (Los Angeles, CA) provided the Immulite automated analysers and the kits for determination of IL‐8 and sponsored the initial meeting of the investigators. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "Randomly assigned to 1 or 2 diagnostic algorithms using sealed opaque envelopes" |
| Allocation concealment (selection bias) | Low risk | |
| Blinding (performance bias and detection bias) | Low risk | Done, IL‐8 results were only provided to physicians in the intervention group. |
| Incomplete outcome data (attrition bias) | Low risk | |
| Selective reporting (reporting bias) | Low risk | |
| Other bias | Low risk | |
| Baseline Outcomes similar? | High risk | No data |
| Free of contamination? | Low risk | |
| Baseline characteristics similar? | Low risk | |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: medical, surgery, intensive care, haematology, and oncology | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change; reminders (circumstantial, physical, written suggestions placed in the notes of participants receiving IV antibiotics) COMPARISON: usual care DESIRED CHANGE: decrease excessive SAMPLE SIZE CALCULATION: no information | |
| Outcomes | PRESCRIBING: Choice: days receiving IV antibiotic therapy per participant, DDDs of IV antibiotics per participant. Antibiotic charges (USD) per participant CLINICAL: Balancing: clinical response at 3 days after completion of antibiotics; retreatment with antibiotics within 7 days; inpatient mortality; re‐admission within 30 days of discharge FINANCIAL: savings on drug costs in USD | |
| Notes | FINANCIAL SUPPORT: Funding: commercial (Bayer Pharmaceuticals) and the Maine Medical Center Research Committee. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "Patients randomised ... using an unblocked computer generated random number table" |
| Allocation concealment (selection bias) | High risk | Not possible; "The patient population was assigned to 1 of 4 medical service groups based on where they were treated at randomizations" |
| Blinding (performance bias and detection bias) | High risk | Not possible |
| Incomplete outcome data (attrition bias) | Low risk | For primary outcomes, not secondary |
| Selective reporting (reporting bias) | Low risk | Based on microbial response and other clinical parameters |
| Other bias | Low risk | No problems noted. |
| Baseline Outcomes similar? | Unclear risk | No information about baseline outcomes pretrial in the allocated groups. |
| Free of contamination? | High risk | Doctors likely to have cared for participants in all groups. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: CBA Risk of Bias: HIGH | |
| Participants | PROVIDERS: a total of 50 ICUs located in 20 hospitals PARTICIPANTS: patients in the ICU CLINICAL PROBLEM: vancomycin use, prevalence of VRE SETTING: hospitals in the USA participating in the ICU surveillance component of National Nosocomial Infection Surveillance | |
| Interventions | FORMAT: 5 interventions were used by 3 to 19 hospitals (some hospitals used more than 1). 3 interventions were hospital‐wide and 2 were unit‐specific. Hospital‐wide interventions (22 ICUs) Intervention 1: educational meetings with dissemination of educational materials, 9 ICUs. Intervention function: education. Intervention 2: audit and feedback, 19 ICUs. Intervention function: enablement. Intervention 3: restriction, 3 ICUs. Intervention function: restriction. Unit‐specific interventions (11 ICUs) Intervention 4: educational meetings with dissemination of educational materials. Intervention function: education. Intervention 5: restriction, 3 ICUs. Intervention function: restriction. DELIVERER: AMT COMPARISON: national benchmark data DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: DDDs of vancomycin MICROBIAL: percentages of VRE and MRSA | |
| Notes | FINANCIAL SUPPORT: Funding: CDC Emerging Infections Program. Competing Interests: no information ADDITIONAL DATA: email response from authors but no additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | CBA ‐ not randomised |
| Allocation concealment (selection bias) | High risk | CBA ‐ not randomised |
| Blinding (performance bias and detection bias) | High risk | CBA, allocation not blinded |
| Incomplete outcome data (attrition bias) | Low risk | Not clear: “Susceptibility reports from isolates obtained as part of infection‐control surveillance were excluded.” Criteria for exclusion of isolates are not described and may not have been consistent across all hospitals. |
| Selective reporting (reporting bias) | Low risk | Not clear: “Susceptibility reports from isolates obtained as part of infection‐control surveillance were excluded.” Criteria for exclusion of isolates are not described and could have led to reporting bias. |
| Other bias | Unclear risk | NOT CLEAR Microbial Risk of Bias Criteria: Case definition: percentage VRE or percentage MRSA in clinical isolates; Planned intervention: DONE; Other infection control isolation: NOT CLEAR; IC practices: NOT CLEAR Data were collected about infection control changes in response to feedback of data, but the paper does not report any results. |
| Baseline Outcomes similar? | Unclear risk | Not stated |
| Free of contamination? | Low risk | Interventions were at different hospitals from control sites. |
| Baseline characteristics similar? | Unclear risk | Not stated |
| Methods | STUDY DESIGN: unintended consequences, cohort study Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in EDs | |
| Interventions | FORMAT, Interventions: audit and feedback, public reporting of antibiotic timing measure as 1 of 10 national quality indicators; financial, institution incentive | |
| Outcomes | UNINTENDED CONSEQUENCES: rates of pneumonia diagnosis, antibiotic use, and waiting times to see a physician | |
| Notes | ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Unclear, analysis says it was adjusted for confounding of the effect of intervention but insufficient detail 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement classification of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Deviations from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Unclear, outcome data reported as % with no numerator/denominator (Table 2) 6. Measurement of outcome: High, the effect estimate is based on regression analysis of annual data for 3 years pre‐ and 2 years postintervention (i.e. only 2 postintervention time points). The authors say that "based on the NHAMCS sample, there were an estimated 40 million (95% confidence interval, 39 to 42 million) ED visits to hospitals by adults with respiratory symptoms between 2001 and 2005." In Table 1, around 10% of these patients had a diagnosis of CAP, so they were not short of data! They should surely have split their data into more time points. 7. Selection of the reported result: Low, single analysis of the intervention‐outcome relationship FINANCIAL SUPPORT:Funding: Primary Care Teaching and Education Fund (internal), Health Resources and Services Administration, and Agency for Healthcare Research and Quality. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change DELIVERER: AMT | |
| Outcomes | PRESCRIBING: Choice: cost of target antibiotics (USD/1000 OBD) | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from pharmacy computer |
| Free of other bias (ITS) ? | High risk | Only 6 month pre‐intervention data, so cannot adjust for seasonal effects. |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all prescribers in the hospital | |
| Interventions | FORMAT: no valid prescribing data. Restrictive. DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | MICROBIAL: resistance to gentamicin and aminoglycoside use | |
| Notes | FINANCIAL SUPPORT: Funding: commercial, Bristol Laboratories and the Veterans Administration. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: MEDIUM, case definition Low, planned intervention Low, other infection control Unclear, no information | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Only 4 months' pre‐intervention data, so secular/seasonal changes possible. No information about infection control measures. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analyses was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine data |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine data |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine data |
| Free of selected reporting (ITS) ? | Low risk | Routine data |
| Free of other bias (ITS) ? | Unclear risk | NOT CLEARMicrobial Outcome Risk of Bias: Planned intervention: DONE Implementation in response to emergence of gentamicin resistance over the previous 5 years; Case definition: DONE Infection from clinical isolates; Other infection control measures: NOT CLEAR, no information provided. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT: Interventions: dissemination of formulary Intervention Function: education After 9 months there was an additional restrictive intervention (autosubstitution of ampicillin sulbactam by ertapenem), but this was not targeted at imipenem use, and no data are provided about prescribing or microbial outcomes for ampicillin sulbactam. | |
| Outcomes | PRESCRIBING: Choice: imipenem use in DDD MICROBIAL: % susceptibility to imipenem in clinical isolates of Pseudomonas aeruginosa | |
| Notes | FINANCIAL SUPPORT: Funding: commercial Merck (manufacturers of ertapenem). Competing Interests: Ellie JC Goldstein is on the advisory boards of Merck, is in the speakers' bureau of Merck, and received research support from Merck; Shuang Lu is employed by Merck Research Laboratories and may own stock or stock options. Anne R Meibohm was formerly employed by Merck Research Laboratories and may own stock or stock options. ADDITIONAL DATA: email response from authors to request for additional data Microbial Risk of Bias LOW: case definition low risk, planned intervention low risk, other infection control measures low risk, no change | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | High risk | Only 6 months' pre‐intervention data for intervention 1 and 9 months' for intervention 2 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT: Intervention: distribution of antibiotic policy Intervention Function: education | |
| Outcomes | PRESCRIBING: Choice: ceftriaxone use in DDD/1000 OBD MICROBIAL: number of participants carrying high level AmpC beta‐lactamase | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH case definition low, planned intervention low, other infection control measures unclear (no data) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from microbiology and pharmacy computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from microbiology and pharmacy computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from microbiology and pharmacy computers |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from microbiology and pharmacy computers |
| Free of other bias (ITS) ? | High risk | Short time series, annual data with only 5 pre‐ and 7 postintervention data points |
| Methods | STUDY DESIGN: cluster RCT, hospital level Risk of Bias: HIGH | |
| Participants | PROVIDERS: obstetric teams | |
| Interventions | FORMAT: Interventions: educational meetings and dissemination of brochures; reminders (physical, posters and brochures) POWER CALCULATION: yes, 40 hospitals. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: % women receiving antibiotic prophylaxis for Caesarean section | |
| Notes | FINANCIAL SUPPORT: Funding: UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP). Competing Interests: 4 authors were editors of The WHO Reproductive Health Library since its inception in 1997 to date of publication. ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Random number generator used (detailed in other article). |
| Allocation concealment (selection bias) | Low risk | Allocation by hospital |
| Blinding (performance bias and detection bias) | Unclear risk | No mention of this |
| Incomplete outcome data (attrition bias) | Low risk | Field workers collected from hospital data and were able to consult mothers for any missing data. |
| Selective reporting (reporting bias) | Low risk | Field workers collected from hospital data and were able to consult mothers for any missing data. |
| Other bias | High risk | End of study in Thai control hospital was conducted at a later stage due to other healthcare‐related activities going on. |
| Baseline Outcomes similar? | Unclear risk | Appear to be different but unclear |
| Free of contamination? | Low risk | Allocation by hospital |
| Baseline characteristics similar? | Unclear risk | No data |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change; reminders (circumstantial and physical, placed in notes of patients who were receiving antibiotics) DELIVERER: AMT POWER CALCULATION: no justification provided for the sample size | |
| Outcomes | PRESCRIBING: Choice: cost of antibiotic therapy CLINICAL: Balancing: length of stay FINANCIAL: charges for antibiotics, laboratory and radiology services, total patient charges. Implementation cost based on days per week required for Pharmacy and Infectious Diseases staff. | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL INFORMATION: no response from authors | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Method of sequence generation not clear; "eligible patients were blindly randomised to the intervention or control group" |
| Allocation concealment (selection bias) | High risk | Not possible to conceal allocation because all intervention participants had a consultation, whereas no control participants did. |
| Blinding (performance bias and detection bias) | Unclear risk | Unclear, despite objective primary outcome measure (LOS), it is not clearly stated that record linkage was without knowledge of allocation. |
| Incomplete outcome data (attrition bias) | Low risk | No problems found, data were analysed from 93% of randomised participants. |
| Selective reporting (reporting bias) | Low risk | No problems found. |
| Other bias | Low risk | No other apparent biases found. |
| Baseline Outcomes similar? | Low risk | Done for primary outcome |
| Free of contamination? | Low risk | Participants were randomised to receive a consultation from an ID specialist (intervention) or no consultation (control), so no contamination likely. |
| Baseline characteristics similar? | Low risk | Done, Table 1 of the original paper |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: dissemination of memo; reminders (physical, newsletter); restrictive by review and make change DELIVERER: pharmacist COMPARISON: usual care | |
| Outcomes | PRESCRIBING: Choice: % of cefazolin doses prescribed at < 8‐hour intervals | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Only 3 months' pre‐intervention data, so secular/seasonal changes possible. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper, Х2 test on mean before‐after. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: residents in internal medicine department CLINICAL PROBLEM: antibiotics use in patients with a fever SETTING: 5 wards in internal medicine department of teaching hospital in Indonesia | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by academic detailing; reminders (physical, pocket book version of guideline) COMPARISON: usual care DESIRED CHANGE: reduce inappropriate | |
| Outcomes | PRESCRIBING: Exposure: % patients treated and total antibiotic consumption (DDD/100 patient days) | |
| Notes | FINANCIAL SUPPORT: Funding: Royal Netherlands Academy of Arts and Sciences, Scientific Programme Indonesia‐Netherlands (SPIN). Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | No, seasonal variation |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, point of analysis is point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data was collected by trained data collectors. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | No, blinding was not possible. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, states they assured completeness of data by collecting while patients were still in the department. |
| Free of selected reporting (ITS) ? | Low risk | Done, Figure 2 |
| Free of other bias (ITS) ? | Low risk | Done, all biases addressed. |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospitals CLINICAL PROBLEM: adults with community‐acquired pneumonia SETTING: 4 university hospitals, New York, USA | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines; reminders (circumstantial and physical, on computer order system for antibiotics and pocket version of guideline) DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: percentage of patients treated with guideline‐recommended antibiotics | |
| Notes | FINANCIAL SUPPORT: Funding: Mount Sinai‐New York University Health System, the North Shore‐Long Island Jewish Health System, and the Robert Wood Johnson Foundation Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | NOT DONE, subjective outcome measure, not blinded |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with χ2 test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data collection same pre‐ and postintervention. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | NOT DONE, subjective outcome measure, not blinded |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | Not stated whether outcome data collected on all participants. |
| Free of selected reporting (ITS) ? | Unclear risk | Not stated whether outcome data collected on all participants. |
| Free of other bias (ITS) ? | High risk | NOT DONE, the only reliable data for analysis are about compliance with the antibiotic policy, which was 80% at baseline. Serious risk of ceiling effect. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: dissemination of guideline; educational outreach by review and recommend change COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: cefazolin expenditure per patient day FINANCIAL: savings in drug costs | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 12 months' data pre‐ and postintervention |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper, no statistical analysis, and only comparison was between mean (uncontrolled) before and after. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Unclear risk | On page 588 the authors state that "a proportion of these savings can be attributed to a decrease in acquisition cost", but they do not say how much. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: physicians in the hospital | |
| Interventions | FORMAT: Interventions: restrictive, removal of restriction Intervention Functions: restriction DELIVERER: specialist physician | |
| Outcomes | PRESCRIBING: Choice: number of courses and cost of restricted drugs FINANCIAL: cost of drugs | |
| Notes | FINANCIAL SUPPORT:Funding: commercial, Pfizer Roerig and the Upjohn companies. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Data collected in same months in 2 consecutive years. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with t‐test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the ED | |
| Interventions | FORMAT, Interventions: structural | |
| Outcomes | PRESCRIBING: Choice: time to first antibiotic dose in minutes measured both from arrival in the ED and from ordering the antibiotic | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention is point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Retrospective data collection using the same methods throughout |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | Data were collected from case records, and allocation was not concealed. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data reported on all 110 included participants. |
| Free of selected reporting (ITS) ? | Low risk | Exclusion rates similar pre‐ (13/69) and post‐ (11/65) intervention. |
| Free of other bias (ITS) ? | High risk | Data only collected for 7 months pre‐ and 8 months postintervention, so secular trends possible. |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in ICU | |
| Interventions | FORMAT, Interventions: reminders (circumstantial and physical, procalcitonin‐based decision support algorithm); structural (introduction of procalcitonin testing) DELIVERER: department (ICU) physician POWER CALCULATION: no information provided | |
| Outcomes | PRESCRIBING: Exposure: duration of all antibiotic therapy in days | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: SS has served as consultant and received payments from B.R.A.H.M.S AG for speaking engagements. All other authors declare no conflicts of interest. ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | No explanation of randomisation process |
| Allocation concealment (selection bias) | Unclear risk | Details of allocation process not provided. |
| Blinding (performance bias and detection bias) | High risk | Open trial |
| Incomplete outcome data (attrition bias) | Low risk | Done, Table 1 and text regarding excluded patients |
| Selective reporting (reporting bias) | Unclear risk | No explicit statement, so selective outcome reporting is possible. |
| Other bias | Low risk | Done, all biases addressed. |
| Baseline Outcomes similar? | Unclear risk | No baseline outcome measurement |
| Free of contamination? | Low risk | Done, procalcitonin results not available for controls. |
| Baseline characteristics similar? | Low risk | Done, mainly similar (IC days slightly different) |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: restrictive by expert approval and removal DELIVERER: pharmacists | |
| Outcomes | PRESCRIBING: Choice: cephalexin dosing units | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | > 2 years' data pre‐ and postintervention |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: no statistical analysis of time series, presented as chart. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: physicians in the hospital CLINICAL PROBLEM: use of IV and oral quinolones SETTING: university hospital in the USA | |
| Interventions | FORMAT, Interventions: reminders (circumstantial and physical, computerised decision support system integrated into an existing provider order entry system) DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive use of IV quinolones | |
| Outcomes | PRESCRIBING: Choice: number of orders for oral quinolone FINANCIAL: savings on drug costs in USD | |
| Notes | FINANCIAL SUPPORT: Funding: NIH Training Grant T32 AI 07474‐08 and Vanderbilt Clinical Research Scholar Award K12 RR17697 (TH). Competing Interests: DAT and RAM receive authorship royalties through Vanderbilt University from the commercial distribution of WizOrder. STR has received consulting fees from McKesson Information Solutions, which has licensed WizOrder for commercial distribution. None of the other authors has related disclosures or potential conflicts of interest. ADDITIONAL DATA: email response from authors with additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Objective outcome measure |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was increase in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | 1 year of data pre‐ and postintervention |
| Free of other bias (ITS) ? | Low risk | Objective primary outcome, cost analysis adjusted to 2003 prices. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | Interventions: educational outreach by review and recommend change; restrictive antibiotic policy but mode of restriction not clear DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: cephalosporin use measured with costs as a percentage of cephalosporins plus penicillins plus aminoglycosides | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Only 12 months' data (9 months' pre‐ and 3 months' postintervention), so cannot control for seasonal effects. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in ICUs | |
| Interventions | FORMAT:Interventions: reminders (circumstantial and physical, drug‐escalation algorithm and intensified diagnostics based on daily procalcitonin measurements); structural (rapid procalcitonin testing) SAMPLE SIZE: yes, total 1200 participants. Details in Appendix 3 1200 participants were randomised and included in the analysis. | |
| Outcomes | PRESCRIBING: Choice: time to first antibiotic dose; number (%) ICU days spent with at least 3 antibiotics CLINICAL: intended 28‐day mortality; unintended (balancing) days in ICU; relative risk of renal impairment | |
| Notes | FINANCIAL SUPPORT: Funding: Danish State, the Lundbeck Foundation, the Toyota Foundation, the A.P. Møller Foundation, the Horboe Foundation, and the Capitol Region of Denmark. Competing Interests: Dr. Jensen received speaker fee and travel reimbursement from B.R.A.H.M.S Diagnostica and an unrestricted grant from the organisation for sample transport and analysis. The remaining authors have not disclosed any potential conflicts of interest. ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomisation was performed 1:1 using a computerised algorithm created by the database manager. |
| Allocation concealment (selection bias) | Low risk | Investigators were masked to assignment before randomisation. Concealed block size, pre‐stratified for site of recruitment, initial Acute Physiology and Chronic Health Evaluation, and age (entered in an encrypted screening form in a password‐protected website) |
| Blinding (performance bias and detection bias) | Low risk | Investigators, treating physicians, and the co‐ordinator were unaware of outcomes during the study. |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all randomised participants. |
| Selective reporting (reporting bias) | Low risk | Outcomes reported on all randomised participants. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | PCT measures only reported for intervention participants. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians and nurses in the paediatric ED | |
| Interventions | FORMAT: Interventions: audit and feedback at individual and group level; educational meetings, dissemination of educational materials; educational outreach by academic detailing at individual and group level; reminders (circumstantial (on electronic health record), physical (cards attached to computers, weekly email newsletter), and verbal); structural (placing antibiotics in front‐line Pyxis stock) | |
| Outcomes | PRESCRIBING: Choice: % of participants receiving first antibiotic dose within 60 minutes | |
| Notes | FINANCIAL SUPPORT: Funding: no external. Competing Interests: none declared ADDITIONAL DATA: authors provided additional data about intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Statistical process control chart |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Primary outcome was time to first antibiotic dose from patient administration system. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Primary outcome was time to first antibiotic dose from patient administration system. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Primary outcome was time to first antibiotic dose from patient administration system. |
| Free of selected reporting (ITS) ? | Low risk | Primary outcome was time to first antibiotic dose from patient administration system. |
| Free of other bias (ITS) ? | High risk | Only 8 months' pre‐intervention data, so seasonal effects cannot be excluded. |
| Methods | STUDY DESIGN: CITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians and nurses in the hospitals | |
| Interventions | FORMAT: Interventions: audit and feedback; educational outreach by review and recommend change | |
| Outcomes | PRESCRIBING: Exposure: days of therapy with all antibiotics/1000 OBD MICROBIAL: +ve C difficile tests per 1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: National Institutes of Health (grants R03‐AG040722 to RLPJ, K23‐DK087919 to PED, and R01‐AI063517 to RAB), Veterans Affairs Merit Review Program, Veterans Integrated Service Network 10 Geriatric Research Education and Clinical Center (VISN 10 GRECC). Competing Interests: RLPJ reports that she has consulted for GOJO and Pfizer and has received grant support ViroPharma. RAB reports that he has consulted for AstraZeneca and has received grant support from AstraZeneca, Ribx, Pfizer, and Steris. CJD reports that he has consulted for BioK, Optimer, and GOJO and has received grant support from ViroPharma, Merck, and Pfizer. All other authors report no conflicts of interest. ADDITIONAL DATA: email with additional data; further information about the intervention is given in Jump 2013. Microbial Risk of Bias: HIGH. Case definition low; Planned intervention low; Other infection control high, no data about infection control other than that the intervention also increased isolation of participants with C difficile infection. Moreover, the intervention discouraged repeat testing of participants with known C difficile infection, which may have biased the microbial outcome. | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Intervention: restrictive by removal from all wards Intervention Function: restriction | |
| Outcomes | PRESCRIBING: Choice: use of fluoroquinolones, DDD/100 OBD MICROBIAL: C difficile infections | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared Microbial Risk of Bias: HIGH, case definition yes, planned intervention no (part of response to outbreak), other infection control measures no (several important changes made at the same time as prescribing intervention) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | No, as this was during an outbreak |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy computer |
| Free of other bias (ITS) ? | High risk | < 1 year data postintervention, fluoroquinolones reintroduced |
| Methods | STUDY DESIGN: unintended consequences, cohort study Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the ED | |
| Interventions | FORMAT, Interventions: audit and feedback; financial, institution incentive | |
| Outcomes | UNINTENDED CONSEQUENCES: confirmation of admission diagnosis by chest X‐ray, mean antibiotic administration per patient admitted with CAP | |
| Notes | NRSI RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding of the effect of intervention unlikely in this study 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement class of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Low, outcome data and intervention status reported on all 518 patients 6. Measurement of outcome: Low, outcome measure objective and measured from patient administration system 7. Selection of the reported result: Low, single, prespecified analysis of the intervention‐outcome relationship FINANCIAL SUPPORT: Funding, none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in hospital CLINICAL PROBLEM: antibiotic use in adult patients with bacterial infections SETTING: 1 university hospital in the Netherlands | |
| Interventions | FORMAT: Intervention: structural (rapid microbiology laboratory testing) DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 1500 participants in total. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: total antibiotic use (average DDDs per patient) CLINICAL: Intended: mortality | |
| Notes | FINANCIAL SUPPORT: Funding: Dutch Association of University Hospitals (‘VAZ‐Doelmatigheidproject’ no. 99207). bioMerieux provided additional funding through an unrestricted grant. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Done, computer‐generated randomisation |
| Allocation concealment (selection bias) | High risk | No, states concealment was impossible. |
| Blinding (performance bias and detection bias) | High risk | No formal blinding attempted. |
| Incomplete outcome data (attrition bias) | Low risk | Done, Figure 1 |
| Selective reporting (reporting bias) | Low risk | Done, all outcomes reported. |
| Other bias | Low risk | No other apparent issues |
| Baseline Outcomes similar? | Unclear risk | No baseline measurement of outcome |
| Free of contamination? | Low risk | Done |
| Baseline characteristics similar? | Low risk | Done, Table 1 |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in hospital CLINICAL PROBLEM: antibiotic use in adult patients with bacterial infections SETTING: 1 tertiary‐care university medical centre in the Netherlands | |
| Interventions | FORMAT: Intervention: structural ‐ other (out‐of‐hours blood culture incubator intended to reduce laboratory turnaround time) Intervention Function: environmental restructuring DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: increase effective POWER CALCULATION: no information. In the Discussion, the authors say "our sample size was too small to study the impacts of time to positivity (Gram stain), identification, and susceptibility testing separately on outcome". | |
| Outcomes | PRESCRIBING: Choice: time to first antibiotic regimen change CLINICAL: Intended: mortality and length of stay | |
| Notes | FINANCIAL SUPPORT: Funding: Becton Dickinson provided the outside BACTEC incubator used in this study. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer‐generated list by independent epidemiologist |
| Allocation concealment (selection bias) | High risk | Allocation not concealed |
| Blinding (performance bias and detection bias) | Unclear risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | 1 episode of missing data in each arm of study |
| Selective reporting (reporting bias) | Low risk | Complete outcomes reported. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Low risk | States no significant differences at baseline. |
| Free of contamination? | Low risk | Rapid reporting only occurred for intervention participants. |
| Baseline characteristics similar? | Low risk | States no significant differences at baseline. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in hospital | |
| Interventions | FORMAT: no valid prescribing data. Restriction with educational meetings and dissemination of guideline. DELIVERER: specialist physician | |
| Outcomes | MICROBIAL: incidence of C difficile‐associated diarrhoea | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | > 1 year data in each of the 3 phases |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: no statistical analysis, mean cases per quarter compared between periods. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done: "The standard operating procedure for selection and processing stool specimens did not change over the study period. All stool specimens from inpatients with liquid or bloody diarrhoea and those receiving antibiotic therapy were tested for C. difficile toxin. C. difficile toxin was detected by cytotoxic activity on a fibroblast cell line, with specific neutralization by Clostridium sordelli antiserum" |
| Free of other bias (ITS) ? | High risk | NOT DONE for the intervention that was intended to reduce C difficile infection in Phase 3 Microbial Outcome Risk of Bias: Planned intervention: NOT DONE for unplanned intervention Phase 3 Case definition: DONE C difficile infection; all stool specimens from inpatients with liquid or bloody diarrhoea and those receiving antibiotic therapy were tested for C difficile toxin. Other infection control measures: DONE, well described and same in all 3 phases |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: audit and feedback; restrictive by expert approval | |
| Outcomes | PRESCRIBING: use of cephalosporins (DDD/1000 OBD) MICROBIAL: isolates of ESBL and new patients with ESBL infection | |
| Notes | FINANCIAL SUPPORT: Funding: City of Seoul grant #10920 and KICOS project grant (Battelle Institute, Korea University). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias HIGH. Case definition Low, planned intervention High (response to outbreak of ESBL), other infection control Unclear (no detail, and authors state that they did not take this into account in their analysis) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Not in original paper but re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Low risk | |
| Methods | STUDY DESIGN: CITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians, nurses, and pharmacists in the hospital | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings; dissemination of guidelines; educational outreach by review and recommend change; reminders (physical, intranet and pocket guidelines; circumstantial, verbal by pharmacy technicians and infection control nurses) The intervention also included the same components targeted at infection control measures (hand hygiene and isolation). | |
| Outcomes | PRESCRIBING: Choice: cefuroxime use in DDD/1000 OBD MICROBIAL: cases per 1000 OBD per month | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: the authors provided multiple additional files of information about the intervention, including examples of the feedback newsletters (in Danish). Microbial Risk of Bias: LOW, case definition low, planned intervention low, other infection control measures low | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | The antimicrobial stewardship intervention was simultaneous with an intervention to improve infection control practice (personal protection and isolation). |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computuers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from pharmacy and microbiology computuers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computuers |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computuers |
| Free of other bias (ITS) ? | Low risk | > 1 year of data pre‐ and postintervention. Microbial risk of bias low |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: dissemination of guideline; reminders (circumstantial and physical, decision support algorithm triggered by PCT test result); structural, introduction of PCT testing DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 107 participants in each group. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Choice and exposure: antibiotics prescribed and duration of antibiotic treatment CLINICAL: Balancing: length of stay and mortality | |
| Notes | FINANCIAL SUPPORT: Funding: Danish Medical Research Council and the Danish Lung Association Study ID: NCT00415753, 271‐05‐0765. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer generated |
| Allocation concealment (selection bias) | Low risk | Concealed until PCT test results available |
| Blinding (performance bias and detection bias) | Low risk | Objective outcome measure: length of stay from routine data system |
| Incomplete outcome data (attrition bias) | Low risk | States that 3 patients died, 2 in PCT and 1 in control |
| Selective reporting (reporting bias) | Low risk | Objective outcome measure: length of stay from routine data system |
| Other bias | Low risk | Adequately powered |
| Baseline Outcomes similar? | Unclear risk | No beseline outcome measures |
| Free of contamination? | Low risk | PCT results only available for intervention participants. |
| Baseline characteristics similar? | Low risk | Mostly similar apart from those with cancer (7 in PCT and 0 in control), although this was adjusted for using sensitivity analysis. |
| Methods | STUDY DESIGN: cluster RCT, hospital level Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: physicians responsible for antimicrobial prophylaxis CLINICAL PROBLEM: Preoperative antimicrobial prophylaxis SETTING: 44 acute care hospitals in the USA | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guideline; educational outreach by academic detailing DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective POWER CALCULATION: yes, 40 hospitals sampling 100 cases per measurement period. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Choice and exposure: 5 performance measures of antimicrobial prophylaxis (timing, receipt, duration, selection, and single preoperative dose) | |
| Notes | FINANCIAL SUPPORT: Funding: grant R01 HS11331‐01A1 from the Agency for Healthcare Research and Quality and Centers for Disease Control and Prevention. Competing Interests: none declared ADDITIONAL DATA: authors provided additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computerised random number generator |
| Allocation concealment (selection bias) | Low risk | By institution |
| Blinding (performance bias and detection bias) | Unclear risk | Does not say if it was blinded or not |
| Incomplete outcome data (attrition bias) | Low risk | Trained data collectors, completeness assured by project staff. |
| Selective reporting (reporting bias) | Low risk | All outcomes reported. |
| Other bias | High risk | High risk of selection bias, as hospitals nominated themselves to be included into the study. |
| Baseline Outcomes similar? | Low risk | See Table 3 |
| Free of contamination? | Low risk | By institution |
| Baseline characteristics similar? | Low risk | See Table 2 |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: patients receiving glycopeptides (teicoplanin or vancomycin) SETTING: 1 hospital in Hong Kong | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: DDD per month of glycopeptides CLINICAL: Balancing: cohort study of patients who died following Staphylococcus aureus bacteraemia | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Done, 32 months' pre‐ and 11 months' postintervention, so secular or seasonal effects unlikely. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before and after) with χ2 test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | 11 months' postintervention data, 32 months' pre‐intervention data |
| Free of other bias (ITS) ? | Low risk | Reliable primary outcome |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: 30 physicians | |
| Interventions | FORMAT: Interventions: reminders (circumstantial and physical, decision support lab score derived from PCT, C‐reactive protein, and urine dipstick); structural, introduction of PCT testing POWER CALCULATION: yes, 140 participants taking into account dropouts. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: % patients receiving antibiotics CLINICAL: re‐admission and time to clinical resolution | |
| Notes | FINANCIAL SUPPORT: Funding: commercial, bioMérieux for data management, statistical analysis, and loan of the procalcitonin assay. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Excel‐generated random numbers table |
| Allocation concealment (selection bias) | Low risk | Sealed envelopes |
| Blinding (performance bias and detection bias) | Low risk | Outcome measured from routine data. |
| Incomplete outcome data (attrition bias) | Low risk | Incomplete outcome data on 3 of 134 control and 4 of 140 intervention children. |
| Selective reporting (reporting bias) | Low risk | Outcomes reported on all remaining children. |
| Other bias | Low risk | The trial ended after completion of a sufficient number of children at the expected timing. |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | No lab score released for control children. |
| Baseline characteristics similar? | Low risk | Table 2 |
| Methods | STUDY DESIGN: CITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meeting with dissemination of guideline; educational outreach by review and recommend change | |
| Outcomes | PRESCRIBING: Choice: fluoroquinolone use in DDD/1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH, case definition Low, planned intervention Low, other infection control High, increase in use of alcohol‐based handrub throughout study period | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Low risk for prescribing outcome |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of anaysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Low risk | > 1 year of data pre‐ and postintervention |
| Methods | STUDY DESIGN: CBA Risk of Bias: HIGH | |
| Participants | PROVIDERS: all surgeons at the hospitals | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines; educational outreach by academic detailing; reminders (physical, posters) DELIVERER: pharmacist | |
| Outcomes | PRESCRIBING: Choice and exposure: appropriate duration and timing of prophylaxis FINANCIAL: drug cost savings in AUD | |
| Notes | FINANCIAL SUPPORT: Funding: Commonwealth Department of Health. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | CBA; "hospitals were paired being matched as far as possible for type size and surgical load" |
| Allocation concealment (selection bias) | High risk | Not done, CBA |
| Blinding (performance bias and detection bias) | High risk | Not stated; all hospitals in same Australian state, CBA so not possible |
| Incomplete outcome data (attrition bias) | Unclear risk | No statement |
| Selective reporting (reporting bias) | Low risk | Objective primary outcome measure on all patients |
| Other bias | Low risk | No other apparent biases found. |
| Baseline Outcomes similar? | Low risk | Done, pre‐intervention data for primary outcome similar in intervention and control hospitals. |
| Free of contamination? | Low risk | Intervention and control sites were different hospitals. |
| Baseline characteristics similar? | Unclear risk | Only information is about characteristics of hospital (teaching, rural, etc.), no data about case mix and unlikely to change over study period. |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: no valid prescribing outcome data. Restriction. DELIVERER: specialist physician | |
| Outcomes | MICROBIAL: Incidence (new cases per 1000 discharges per month) of ceftazidime‐resistant Klebsiella pneumoniae, MRSA, and cefotaxime‐resistant Acinetobacter species (ITS data) | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias MEDIUM: case definition: Low; planned intervention: Low; infection control practices: High. At the start of the intervention, contact precautions were changed to include patients with Clostridium difficile infection. | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Reliable primary outcome |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with t‐test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Unclear risk | Not clear, no information about protocols for clinical sampling or testing |
| Free of other bias (ITS) ? | High risk | Change in infection control practices at start of intervention |
| Methods | STUDY DESIGN: unintended consequences, cross‐sectional and cohort study Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT, Interventions: restrictive by prior approval | |
| Outcomes | UNINTENDED CONSEQUENCES: delay in ordering of restricted antibiotics | |
| Notes | ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding of the effect of intervention unlikely in this study 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Low, outcome data and intervention status complete for both cross‐sectional and cohort study 6. Measurement of outcome: Low, outcome measures objective and ascertained from patient administration system 7. Selection of the reported result: Low, single analysis of prespecified outcomes FINANCIAL SUPPORT: Funding: Centers for Education and Research on Therapeutics grant (U18‐HS10399) from the Agency for Healthcare Research and Quality (AHRQ), the Mentored Patient‐Oriented Research Career Development Award (K23‐AI‐060887‐01) of the NIH from the National Institute of Allergy and Infectious Diseases, Public Health Service grant (DK‐02987‐01) of the NIH, and an Improving Patient Safety Through Reduction in Medication Errors grant (P01‐HS11530‐01) from the AHRQ. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians at the hospital | |
| Interventions | FORMAT: Intervention: restrictive by expert approval, not clear if there was also removal DELIVERER: AMT DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: vancomycin use in DDD per 1000 patient days MICROBIAL: proportion of enterococci resistant to vancomycin | |
| Notes | FINANCIAL SUPPORT: Funding: Public Health Service (grant DK‐02987‐01) of the National Institutes of Health (to EL). This study was also supported in part by an Agency for Healthcare Research and Quality Centers for Education and Research on Therapeutics co‐operative agreement (U18‐HS10399). Competing Interests: no information ADDITIONAL DATA: authors provided additional prescribing data to enable segmented regression analysis Microbial Risk of Bias HIGH: Case definition: Low. Planned intervention: High: unplanned intervention in response to emergence of VRE over the previous 3 years. Other infection control measures: Low | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | |
| Shape of effect pre‐specified (ITS) ? | Low risk | |
| Unlikely to affect data collection (ITS) ? | Low risk | |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | |
| Incomplete outcome data addressed (ITS) ? | Low risk | |
| Free of selected reporting (ITS) ? | Low risk | |
| Free of other bias (ITS) ? | High risk | Microbial outcome risk of bias: HIGH . |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: dissemination of new antibiotic policy 3 months before the structural intervention; restrictive: the new antibiotic policy included requirement for expert approval; structural: introduction of universal screening for MRSA | |
| Outcomes | PRESCRIBING: Choice: no valid data for re‐analysis in the paper, but the authors' ARIMA time series analysis includes the effect of the change in antibiotic policy on the microbial outcomes MICROBIAL: S aureus bacteraemias, MRSA, and MSSA | |
| Notes | FINANCIAL SUPPORT: Funding: Scottish government Health Directorate. Competing Interests: IG has received personal and grant financial support from companies manufacturing diagnostics and therapeutics for MRSA. BE has received grant financial support from Novartis. Other authors: none ADDITIONAL DATA: authors provided additional data Microbial Risk of Bias MEDIUM: case definition High, MRSA screening introduced at the same time as change in antibiotic policy, planned intervention Low, other infection control Low for isolation and personal infection control | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | The change in antibiotic policy was 9 months after the introduction of MRSA screening. The authors' analysis suggests an independent effect from the policy change. |
| Analysed appropriately (ITS) ? | Low risk | ARIMA time series analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine patient administration systems |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine patient administration systems |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine patient administration systems |
| Free of selected reporting (ITS) ? | Low risk | Routine patient administration systems |
| Free of other bias (ITS) ? | Low risk | Other microbial ROB criteria low |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the ICUs | |
| Interventions | FORMAT: Interventions: reminders (circumstantial and physical, decision support algorithm triggered by PCT test result); structural, introduction of PCT testing DELIVERER: specialist physician (ICU and respiratory) POWER CALCULATION: yes, 250 participants in each group. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: antibiotic consumption as % ICU days and DDD/100 OBD CLINICAL: mortality, length of ICU stay, days on ventilator | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "patients were prospectively randomized", but no information about how |
| Allocation concealment (selection bias) | Unclear risk | "patients were prospectively randomized", but no information about how |
| Blinding (performance bias and detection bias) | Low risk | Procalcitonin only reported for intervention participants. |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all participants. |
| Selective reporting (reporting bias) | Low risk | Outcomes reported on all participants. |
| Other bias | High risk | Study did not achieve recruitment required by power calculation. |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | Procalcitonin only reported for intervention participants. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: physicians CLINICAL PROBLEM: patients receiving ceftriaxone SETTING: a hospital in the USA | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines; reminders (circumstantial and physical, letters sent to physicians when intervention needed plus posters) DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: grams of ceftriaxone and cefotaxime FINANCIAL: cost of intervention (0.5 FTE ID physician and savings on drug costs) | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy computer |
| Free of other bias (ITS) ? | High risk | > 1 year data pre‐ and postintervention, but only 4 postintervention time points (quarterly data) |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all staff in the hospital | |
| Interventions | FORMAT: Intervention: educational outreach by review and recommend change DELIVERER: specialist physicians (paediatric ID) COMPARISON: pre‐intervention DESIRED CHANGE: decrease in use of extended‐spectrum cephalosporins to reduce ESBL infections | |
| Outcomes | PRESCRIBING: Choice: days on target antibiotics/1000 OBD MICROBIAL: ESBL strains as % total isolates | |
| Notes | FINANCIAL SUPPORT: Funding: Wyeth Research. Competing Interests: none declared. ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias LOW: case definition yes, planned intervention yes, stable ESBL for 3 years pre‐intervention, other infection control yes | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Infection control policies unchanged throughout (page 631). |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Decrease |
| Unlikely to affect data collection (ITS) ? | Low risk | Data about prescribing and microbial outcomes were from routine, electronic data systems. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data about prescribing and microbial outcomes were from routine, electronic data systems. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data about prescribing and microbial outcomes were from routine, electronic data systems. |
| Free of selected reporting (ITS) ? | Low risk | Data about prescribing and microbial outcomes were from routine, electronic data systems. |
| Free of other bias (ITS) ? | Low risk | 4 years' data pre‐ and 3 years' data postintervention, so can account for temporal trends. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the units | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings (monthly with residents) with dissemination of educational materials | |
| Outcomes | PRESCRIBING: Choice: DDD/1000 OBD of target antibiotics FINANCIAL: intervention cost and savings (cost of all antibiotics) | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: authors provided additional data about the intervention and for the meta‐regression | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcomes were measured from electronic pharmacy data. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcomes were measured from electronic pharmacy data. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcomes were measured from electronic pharmacy data. |
| Free of selected reporting (ITS) ? | Low risk | Outcomes were measured from electronic pharmacy data. |
| Free of other bias (ITS) ? | Low risk | > 12 months' data pre‐ and postintervention |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in medical and surgical wards | |
| Interventions | FORMAT: Intervention: educational outreach by review and recommend change Intervention Functions: enablement, persuasion POWER CALCULATION: yes, 253 participants in each group. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: duration of all antibiotic treatment CLINICAL: Balancing: mortality, ICU admission, new course of antibiotic treatment, length of stay FINANCIAL: intervention cost and savings (supplementary file) MICROBIAL: secondary infection and/or colonisation with multidrug‐resistant bacteria in the 6 months following randomisation | |
| Notes | FINANCIAL SUPPORT:Funding: none. Competing Interests: none declared ADDITIONAL DATA: supplementary file online with data about financial and microbial outcomes, no response from authors to request for additional data Microbial Risk of Bias: case defintion low, planned intervention low, other infection control unclear, no information | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Eligible patients were allocated to either the intervention or the control group using a computer‐generated randomisation list, which was maintained independently of the IDP. |
| Allocation concealment (selection bias) | Low risk | Concealment of the allocation was maintained, as the physician in charge of the patient and the IDP were involved only after randomisation. |
| Blinding (performance bias and detection bias) | High risk | Blinding was not possible. |
| Incomplete outcome data (attrition bias) | Low risk | No participants were lost to follow‐up. |
| Selective reporting (reporting bias) | Low risk | Outcomes reported on all participants. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | IDP only visited intervention participants. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: Departments of Neurology and Neurosurgery PARTICIPANTS: all patients in the departments | |
| Interventions | FORMAT: no valid prescribing data, restriction by expert approval and removal DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | MICROBIAL: prevalence of gentamicin‐resistant Enterobacteriaceae in weekly screening stool swabs | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | NOT DONE, major changes in infection control 4 weeks before the antibiotic restriction. Separate effect cannot be estimated because no screening before change in infection control. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: no statistical analysis, time series data presented as run chart. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Screening protocol was the same pre‐ and postintervention. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Screening protocol was the same pre‐ and postintervention. |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | NOT CLEAR, no explicit statement about complete screening samples for all participants |
| Free of selected reporting (ITS) ? | Unclear risk | NOT CLEAR, no explicit statement about complete screening samples for all participants |
| Free of other bias (ITS) ? | High risk | Microbial Outcome Risk of Bias Criteria: Case definition: DONE colonisation by screening; Planned intervention: NOT DONE, in response to increase in GRE; Other infection control practices: NOT DONE changes 4 weeks before antibiotic restriction; Isolation: isolation of gentamicin‐resistantEnterobacteriaceae‐positive patients in either side‐rooms or cohorted with other positive patients; IC practices: increase in education plus several new hygiene practices: disposable washing gloves, elbow‐directed soap dispensers; new room‐cleaning protocol. Hygiene was emphasised and more stringent barrier precautions. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: incidences of MRSA SETTING: 1 general hospital in the UK | |
| Interventions | FORMAT: Intervention: educational meetings with dissemination of guideline; reminders, verbal on rounds Intervention Function: education, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce inappropriate | |
| Outcomes | PRESCRIBING: Choice: DDDs per 1000 OBD each month MICROBIAL: Episodes of MRSA blood isolates per 1000 OBD each month | |
| Notes | FINANCIAL SUPPORT: Funding: unrestricted educational grant from Wyeth. Competing Interests: LDL received honoraria for lectures from Bayer and Bard. ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB HIGH; case definition Low, planned intervention Low, other infection control High, no information about infection control other than screening for MRSA | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 18 months' pre‐ and 15 months' postintervention data |
| Analysed appropriately (ITS) ? | Low risk | Segmented regresssion analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis is point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Pharmacy data used pre‐ and postintervention. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Pharmacy data used pre‐ and postintervention. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Pharmacy data used pre‐ and postintervention. |
| Free of selected reporting (ITS) ? | Low risk | Pharmacy data used pre‐ and postintervention. |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: unintended consequences, cohort study Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: restrictive | |
| Outcomes | UNINTENDED CONSEQUENCES: accuracy of laboratory and clinical information provided in calls to the Antimicrobial Stewardship Program | |
| Notes | ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, the effects of inaccurate communication and each of the potential confounders on the risk of inappropriate antimicrobial recommendations were evaluated in bivariable analyses 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement of interventions: Low, antimicrobial recommendations were evaluated for appropriateness by a 3‐person panel of infectious diseases experts blinded to the accuracy of information communicated during the Antimicrobial Stewardship Program call 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: High, panelists could not agree on appropriateness of treatment for 37 patients. Outcome data complete for the 163 included patients 6. Measurement of outcome: Low, outcome measures objective and ascertained from patient administration system 7. Selection of the reported result: High, multiple secondary analyses were performed using the main study outcome FINANCIAL SUPPORT: Funding: National Institutes of Health, Agency for Healthcare Research and Quality, and University of Pennsylvania. Competing Interests: none declared. ADDITIONAL DATA: no response from authors to request for additional data | |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: Department of Emergency Medicine, ICU staff PARTICIPANTS: adults (age > 18) with sepsis | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm triggered by measurement of PCT); structural, introduction of PCT testing DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Exposure: duration of all antibiotic treatment CLINICAL: Balancing: 28‐day mortality, length of hospital stay, length of ICU stay, recurrence within 28 days | |
| Notes | Translated from Chinese FUNDING: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Random number table method |
| Allocation concealment (selection bias) | Unclear risk | No information about concealment |
| Blinding (performance bias and detection bias) | Unclear risk | No information about blinding |
| Incomplete outcome data (attrition bias) | Low risk | Outcome reported on all participants. |
| Selective reporting (reporting bias) | Low risk | Outcome reported on all participants. |
| Other bias | High risk | The study had 7 exclusion criteria that are not all clearly defined, so there is a high risk of selection bias, especially as allocation was probably not concealed. |
| Baseline Outcomes similar? | Unclear risk | No data about baseline outcomes |
| Free of contamination? | Low risk | PCT results only for intervention participants. |
| Baseline characteristics similar? | Low risk | Table 1, age, gender, APACHE score, comorbidities |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm triggered by measurement of PCT); structural, introduction of PCT testing DELIVERER: departmental physicians (Internal and Geriatric Medicine) POWER CALCULATION: yes, 90 participants per group. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: % treated with antibiotics CLINICAL: Balancing: length of hospital stay; clinical, laboratory, and spirometry outcomes at discharge; and results of spirometry at the 12‐month follow‐up examination, as well as the results of the Asthma Control Test | |
| Notes | FINANCIAL SUPPORT: Funding: Shanghai Fifth People’s Hospital Science Foundation and Minhang District Natural Science Foundation of Shanghai. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "Allocation to either intervention was conducted according to computer‐generated random numbers produced by an independent statistician." |
| Allocation concealment (selection bias) | Low risk | "After randomization, an opaque, sealed, sequentially numbered envelope containing the PCT or control protocol was prepared for each subject according to group assignment" |
| Blinding (performance bias and detection bias) | High risk | Blinding was not possible. |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all 180 randomised participants. |
| Selective reporting (reporting bias) | Low risk | Antibiotic use reported for all 180 participants. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | Procalcitonin only reported on intervention participants. |
| Baseline characteristics similar? | Low risk | Tables 1 and 2 |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all prescribers and staff PATIENTS: all inpatients CLINICAL PROBLEM: patients receiving antibiotic treatment and patients with MRSA infections SETTING: university‐affiliated veterans hospital in the USA | |
| Interventions | FORMAT: Interventions: educational meetings, in‐service training sessions with dissemination of guideline; reminders (circumstantial, electronic, triggered by prescribing target drugs) DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: change in use of levofloxacin, ciprofloxacin, and other antibiotics MICROBIAL: MRSA infection rate (number/1000 OBD) | |
| Notes | FINANCIAL SUPPORT: This article is the result of work supported with resources and the use of facilities at the Boise Veterans Affairs Medical Center, and is partially funded by an unrestricted educational grant from Wyeth Pharmaceuticals. Conflict of interest: no information ADDITIONAL DATA: email response from authors but no additional data Microbial ROB: MEDIUM. Case definition Low, Planned intervention Low, Other infection control High, prescribing intervention coincident with infection control interventions | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Data collected for 11 months postintervention. Season included as a variable in the model, and summer found to be associated with lower MRSA infection rate. Coincident with infection control intervention for norovirus outbreak, infection control variables included in the model and significantly associated with lower MRSA rate. |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Unclear risk | Not clear, no information about protocols for sampling or testing for MRSA over the study period |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Objective data about MRSA |
| Incomplete outcome data addressed (ITS) ? | Low risk | Identification of cases was the same in the pre‐ and postintervention phases. |
| Free of selected reporting (ITS) ? | Low risk | In addition to the primary outcome of MRSA infections, the figure shows percentage of MRSA for all Staphylococcus aureus isolates with a reduction coincident with the intervention. |
| Free of other bias (ITS) ? | High risk | NOT DONE data are MRSA infection rates in 6‐month time periods based on very small numbers of cases (80 cases in 3½ years). Microbial Outcome Risk of Bias: Case definition: MRSA infection. Screening for nosocomial infections was performed through daily review of hospital admissions and discharges, intravenous antibiotic use by patients admitted to the emergency department, and laboratory reports with case confirmation by review of medical records. “An infection was assumed to be caused by MRSA if cultures of blood, intravenous line, sputum, urine, tissue, or stool obtained at the time of symptom development yielded MRSA.” Planned intervention: YES. Intervention introduced in July 2003 in response to May 2003 SHEA recommendations that institutions where MRSA is endemic should consider limiting the use of broad‐spectrum antibiotics, especially fluoroquinolones. Other infection control: NOT DONE. Antibiotic intervention coincident with environmental decontamination and hand hygiene campaign because of norovirus outbreak. Data about some infection control variables showed no change after start of intervention. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: physicians in the hospital CLINICAL PROBLEM: antibiotic use in cardiology hospital, primary target fluorquinolone use SETTING: 1 cardiology hospital in Brazil | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change DELIVERER: Intervention 1 ID physician (2 h per day), Intervention 2 AMT (physician plus pharmacist, 4 h per day) COMPARISON: usual care DESIRED CHANGE: reduce inappropriate | |
| Outcomes | PRESCRIBING: Choice: monthly consumption (DDDs/100 OBD) of antibiotics, primary target fluoroquinolones FINANCIAL: hours of time to implement the intervention | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | States in discussion that most changes not related to any other external factor. |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention is point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from electronic pharmacy records |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from electronic pharmacy records |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from electronic pharmacy records |
| Free of selected reporting (ITS) ? | Low risk | Data from electronic pharmacy records |
| Free of other bias (ITS) ? | Low risk | > 12 months' data in each of the 3 study phases |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in cardiac surgery | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm triggered by measurement of PCT); structural, introduction of PCT testing POWER CALCULATION: unclear, target effect size decrease from 45% of antibiotic use in the standard group to 22% in the procalcitonin group, but no data about sample size | |
| Outcomes | PRESCRIBING: Exposure: % treated with antibiotics CLINICAL: ICU stays, hospital stay, rehospitalisation, incidence of infections, severe non‐infection complications, and mortality rate with 1‐year follow‐up FINANCIAL: cost of antibiotics and PCT tests | |
| Notes | FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer‐generated randomisation scheme |
| Allocation concealment (selection bias) | Unclear risk | No information about concealment |
| Blinding (performance bias and detection bias) | High risk | Blinding not possible |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all 205 participants. |
| Selective reporting (reporting bias) | Low risk | Antibiotic treatment reported on all participants. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | PCT only measured on intervention participants. |
| Baseline characteristics similar? | Low risk | Tables 1 and 2 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in medical and surgical wards | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines; reminders (physical, posters in the wards and monthly email to doctors) | |
| Outcomes | PRESCRIBING: Choice: time to first antibiotic dose | |
| Notes | FINANCIAL SUPPORT: Funding: Scottish Government Chief Scientist Office (CSO) Clinical Academic Training Fellowship (CAF/07/06). Competing Interests: salary costs for 2 investigators from CSO, no others declared ADDITIONAL DATA: email response from authors to request for additional data with additional detail from a PhD thesis | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | There was a national intervention (Scottish Patient Safety Program) that included reducing time to rescue of deteriorating patients throughout the pre‐ and postintervention phases. |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Objective primary outcome measure (time to first antibiotic dose) collected by single person (CM). |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Objective primary outcome measure (time to first antibiotic dose) collected by single person (CM). |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data collected on all participants. |
| Free of selected reporting (ITS) ? | Low risk | Outcome data collected on all participants. |
| Free of other bias (ITS) ? | Low risk | Data collected over winter months in pre‐ and postintervention period. |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change POWER CALCULATION: yes, 140 participants in each group | |
| Outcomes | PRESCRIBING: Choice: use of target drugs in DDD CLINICAL: length of stay, mortality, re‐admissions | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Eligible prescriptions were allocated daily to either the intervention or the control group using a computer‐generated randomisation list. |
| Allocation concealment (selection bias) | Low risk | Concealment of allocation was pharmacy controlled. Instruction in allocation concealment was provided. |
| Blinding (performance bias and detection bias) | High risk | Blinding was not possible. |
| Incomplete outcome data (attrition bias) | Low risk | Outcome data reported on all randomised participants. |
| Selective reporting (reporting bias) | Low risk | Outcome data reported on all randomised participants. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | High risk | The authors say: "To minimize contamination bias, that is, any change in antibiotic prescription practice in the control group, only the infectious diseases physicians and hospital pharmacists were informed about the implementation of the program." However, they were placing written recommendations in case notes for intervention patients, and physicians caring for those patients would also be caring for control patients. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: Controlled ITS Risk of bias: MEDIUM | |
| Participants | PROVIDERS: staff of Trauma & Burns ICU (TBICU), Medical ICU (MICU), and Surgical ICU (SICU) CLINICAL PROBLEM: adults needing intensive care SETTING: single > 500‐bed university hospital in the USA | |
| Interventions | FORMAT: Intervention: dissemination of guideline Intervention Function: education DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: use of vancomycin, 3rd‐generation cephalosporins, and piperacillin tazobactam per 1000 patient days MICROBIAL: MRSA infections and VRE infections per 1000 patient days | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB HIGH: Case definition: Low. Planned intervention: High for intervention ward (response to increasing VRE in previous 2 years). However, steady increase not an outbreak and VRE data presented for other wards with no intervention. Other infection control: High, no information about isolation or infection control practices before or after the intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Only 9 months' data pre‐intervention, so secular/seasonal effects possible. No information about infection control practices before or after the intervention |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: χ2 test, uncontrolled before‐after with Poisson regression analysis of VRE rates. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, objective outcome measure |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Objective outcome measure, VRE infections |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, objective outcome measure |
| Free of selected reporting (ITS) ? | Unclear risk | Not clear, no information about protocol for sampling or testing over study period |
| Free of other bias (ITS) ? | Low risk | >1 year of data pre‐ and post‐intervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in hospital | |
| Interventions | FORMAT: Interventions: educational meetings and dissemination of new antibiotic policy; educational outreach by academic detailing, "education of junior medical staff on the rationale behind the antibiotic selection was also carried out by clinical pharmacists" (p208); restrictive by compulsory order form and removal DELIVERER: department physician | |
| Outcomes | PRESCRIBING: Choice: dosage units of target antibiotic FINANCIAL: expenditure on antibiotics | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 12 months' data pre‐ and postintervention |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | Antibiotic costs were adjusted to 1989 prices. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Intervention: restrictive by expert approval and probably by review and make change Intervention Function: restriction DELIVERER: specialist physician COMPARISON: usual care | |
| Outcomes | PRESCRIBING: Choice: grams of chloramphenicol (thousands), data are also presented for other drugs (ampicillin, nafcillin, and cloxacillin) | |
| Notes | FINANCIAL SUPPORT: Funding: grants 5R01‐A1‐23, 2T01‐AJ‐08, and IT01‐Ai‐447 from the National Institute of Allergy and Infectious Diseases. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Data over 8 years, 4 years pre‐ and 4 years postintervention |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: staff from 12 medical wards CLINICAL PROBLEM: adults requiring IV antibiotic therapy SETTING: single university hospital in the UK | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of protocol for IV to oral switch; educational outreach by academic detailing; reminders (circumstantial, sticker in charts of patients receiving IV antibiotics and physical, posters in wards and at nursing stations) DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: modification of existing management (faster switch from IV to oral administration of antibiotics) | |
| Outcomes | PRESCRIBING: Choice: appropriateness of timing of IV to oral switch | |
| Notes | FINANCIAL SUPPORT: Funding: Greater Glasgow Health Board. Competing Interests: no information ADDITIONAL DATA: authors provided additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Not done, data were only collected for 4 weeks before and after the intervention, so secular changes could have accounted for any differences. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with χ2 test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Unclear risk | Not stated |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | No information about reliability or completeness of primary outcome |
| Free of selected reporting (ITS) ? | Unclear risk | No information about reliability or completeness of primary outcome |
| Free of other bias (ITS) ? | High risk | Only 4 weekly time points pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the elderly care unit | |
| Interventions | FORMAT: Interventions: dissemination of new antibiotic policy; restrictive by removal and by review and make change DELIVERER: pharmacist | |
| Outcomes | PRESCRIBING: Choice: monthly cost of cefuroxime (ITS data) MICROBIAL: cases of CDI per month (ITS data) | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB HIGH: Case definition: Low, CDI, definition unchanged during the study periods. Unplanned intervention: High, antibiotic restriction was implemented in response to increasing cases of CDI in the preceding 7 months despite increased infection control. Other infection control measures: High, changes to environmental cleaning and reminders about hand hygiene implemented 3 months before the start of intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | |
| Analysed appropriately (ITS) ? | Low risk | |
| Shape of effect pre‐specified (ITS) ? | Low risk | |
| Unlikely to affect data collection (ITS) ? | Low risk | |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | |
| Incomplete outcome data addressed (ITS) ? | Low risk | |
| Free of selected reporting (ITS) ? | Low risk | |
| Free of other bias (ITS) ? | Low risk | |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: physicians | |
| Interventions | FORMAT: Interventions: dissemination of guidelines; educational outreach by academic detailing; educational outreach by review and recommend change; reminders (physical, posters in clinical areas); restrictive by compulsory order form, expert approval required, removal and review and make change DELIVERER: specialist physician (ID) COMPARISON: usual care DESIRED CHANGE: reduction in established management (reduction in antibiotic costs) | |
| Outcomes | PRESCRIBING: Choice: cost of antibiotics (USD) as an indicator of use COSTS: cost of antibiotics | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Full year before and after |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | Antibiotic costs were adjusted to 1995 prices and excluded ancillary or administrative charges. |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: restrictive by expert approval required | |
| Outcomes | PRESCRIBING: Choice: use of ceftazidime, imipenem, and ceftriaxone reported as number of approvals for these drugs MICROBIAL: incidence of ceftazidime‐resistant Klebsiella pneumoniae as the rate per 1000 average daily census | |
| Notes | FINANCIAL SUPPORT: Funding: BMA Medical Foundation. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB: HIGH Case definition Low, Unplanned intervention High, Other infection control High | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Infection control intervention simultaneous with antibiotic intervention. 14 months' pre‐ and 11 months' postintervention, so secular change unlikely. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: run chart with no statistical analysis. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | High risk | Pre‐intervention data were incomplete. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. Criteria for sampling and testing were unchanged over the study period. |
| Free of other bias (ITS) ? | High risk | NOT DONE. Microbial Outcome Risk of Bias Criteria: Planned intervention: NOT DONE, unplanned intervention. Case definition: DONE, microbial outcome was colonisation by surveillance screening. Clinical infection was diagnosed by CDC definition but not used as an outcome. Infection or colonisation by case note review. Other infection control measures: NOT DONE, barrier precautions were instituted on colonised and infected patients at the same time that ceftazidime restriction was implemented. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: physicians in the neurosurgical ICU CLINICAL PROBLEM: antibiotic treatment for pneumonia in neurosurgical ICU SETTING: neurosurgical ICU in 1 hospital in Germany | |
| Interventions | FORMAT: Interventions: educational meeting with neurosurgeons and dissemination of guideline DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, in the new guideline the duration of antibiotic therapy for nosocomial pneumonia was reduced from 14 to 7 days, while for community‐acquired pneumonia the period fell from 10 to 5 days | |
| Outcomes | PRESCRIBING: Exposure: total antibiotic use and cost/1000 OBD FINANCIAL: changes in total antibiotic cost | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome from pharmacy database pre‐ and postintervention. |
| Knowledge of the allocation adequately prevented(ITS)? | Unclear risk | No mention of blinding |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome from pharmacy database pre‐ and postintervention. |
| Free of selected reporting (ITS) ? | Low risk | Outcome from pharmacy database pre‐ and postintervention. |
| Free of other bias (ITS) ? | Low risk | > 1 year of data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all patients in an adult surgical ICU | |
| Interventions | FORMAT: Interventions: dissemination of guidelines and educational meetings in departments of surgery and anaesthesiology DELIVERER: multidisciplinary AMT COMPARISON: pre‐intervention outcomes DESIRED CHANGE: reduction in use of cephalosporins and resistance in gram‐negative bacteria | |
| Outcomes | PRESCRIBING: Choice: use of cephalosporins in DDD/1000 OBD MICROBIAL: resistance to cephalosporins and piperacillin in gram‐negative bacteria isolated from clinical and surveillance cultures | |
| Notes | FINANCIAL SUPPORT: Funding: Federal Ministry of Education and Research (01Kl 9907). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB: HIGH Case definition Low; Planned intervention Low; Other infection control Unclear, no clear information about isolation or personal‐protection policies | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Low risk | Data for > 2 years' pre‐ and postintervention, so secular trends accounted for. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the ICU CLINICAL PROBLEM: reducing length of antibiotic prophylaxis for cerebrospinal shunts SETTING: ICU department in 1 teaching hospital in Germany | |
| Interventions | FORMAT: Intervention: educational meeting and dissemination of new policy. In autumn 2003, a comprehensive teaching session on antibiotic prophylaxis in cerebrospinal shunts was organised by the infection control and neurosurgery teams. This resulted in a revised recommendation of single‐shot prophylaxis with cefuroxime for shunt catheters, beginning in January 2004. Prior to implementation of this recommendation, cefuroxime was administered for the whole duration of external cerebrospinal fluid drainage, which could be up to 2 to 3 weeks. Intervention Functions: education, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, shorten duration of prophylaxis | |
| Outcomes | PRESCRIBING: Exposure: total antibiotic use in DDD/1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: Federal Ministry of Education and Research (01Kl 9907). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Says they could not control for changes over time and that an antimicrobial stewardship programme was implemented |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharamacy computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharamacy computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharamacy computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharamacy computers |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: ICU physicians PATIENTS: 302 adults in the ICU (154 intervention, 148 control) CLINICAL PROBLEM: VAP requiring antibiotics SETTING: single ICU in a teaching hospital in the USA | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Exposure: duration of all antibiotic therapy | |
| Notes | FINANCIAL SUPPORT: Funding: part commercial, Barnes‐Jewish Hospital Foundation and an unrestricted grant from Elan Pharmaceuticals. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "Patients were randomly assigned", but no details of how the sequence was generated |
| Allocation concealment (selection bias) | Unclear risk | Not stated |
| Blinding (performance bias and detection bias) | High risk | Not possible |
| Incomplete outcome data (attrition bias) | Low risk | Outcome data were missing from 4 (2.6%) patients in the intervention group and 8 (5.4%) patients in the control group. |
| Selective reporting (reporting bias) | Low risk | Done, outcomes were obtained from routine data systems. |
| Other bias | High risk | The policy was only implemented at weekends or on holidays when 1 of the 2 investigators was available in the hospital. |
| Baseline Outcomes similar? | Unclear risk | No data about duration of therapy before the intervention |
| Free of contamination? | High risk | Physicians managing patients in the control group would have seen reminders for the intervention group. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the Department of Paediatrics | |
| Interventions | FORMAT: Interventions: audit and feedback, educational meeting with dissemination of guideline; reminders (verbal (on rounds, so may have been circumstantial) and physical (pocket‐size guideline, screensavers)) | |
| Outcomes | PRESCRIBING: Exposure: % treated with antibiotics CLINICAL: length of stay, re‐admission | |
| Notes | FINANCIAL SUPPORT: Funding: no external. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Antibiotic use was 1 of 10 recommendations in the guideline; the other 9 would have impacted on clinical outcomes. |
| Analysed appropriately (ITS) ? | Low risk | Statistical process control charts |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | All outcome data from hospital patient administration system |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | All outcome data from hospital patient administration system |
| Incomplete outcome data addressed (ITS) ? | Low risk | All outcome data from hospital patient administration system |
| Free of selected reporting (ITS) ? | Low risk | All outcome data from hospital patient administration system |
| Free of other bias (ITS) ? | Low risk | Data collected over 3 winters, 1 pre‐ and 2 postintervention. |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: physicians in the Department of Internal Medicine CLINICAL PROBLEM: receiving antibiotic therapy SETTING: 1 university hospital in the Netherlands | |
| Interventions | FORMAT: 1st Intervention: audit and feedback; educational meetings with dissemination of guideline 2nd Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: % compliance with guideline; antibiotic cost FINANCIAL: antibiotic cost | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was increase in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data collection method was same throughout study. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | Subjective outcome without blinded assessment |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | Not stated whether compliance was assessed in all patients. |
| Free of selected reporting (ITS) ? | Unclear risk | Not stated whether compliance was assessed in all patients. |
| Free of other bias (ITS) ? | Low risk | The kappa value for the primary outcome measure was 0.71, which is below the level set by EPOC, but for the reasons given in the text we feel is adequate for assessment of compliance with an antibiotic guideline. Drug costs were adjusted to April 2001 prices. |
| Methods | STUDY DESIGN: CITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in hospital | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change. NB the authors describe their intervention as "audit and feedback", but there was no feedback of data over time about progress to goal, just review with feedback about individual patients DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Exposure: total antibiotic use (days of therapy/1000 patient days) | |
| Notes | FINANCIAL SUPPORT: Funding: Agency for Healtcare Quality and Reseach (grant U18‐HS10399). Competing Interests: none declared ADDITIONAL DATA: authors provided additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Unclear, there were some infection control initiatives running. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Yes, point of analysis is point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Yes, data collection was the same pre‐ and postintervention. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Yes, objective outcomes |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine data, so could assume complete. |
| Free of selected reporting (ITS) ? | Low risk | Yes, all relevant outcomes reported. |
| Free of other bias (ITS) ? | Low risk | Yes, all biases addressed. |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the ICU | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing POWER CALCULATION: yes, a total of at least 66 participants. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: duration of treatment in days CLINICAL: Balancing: mortality, relapse of infection, length of ICU stay, length of hospital stay | |
| Notes | FINANCIAL SUPPORT: Funding: commercial B.R.A.H.M.S AG (USD 50,000). Competing Interests: 2 authors received speaker honoraria from B.R.A.H.M.S AG. ADDITIONAL DATA: online supplementary file with addtional infromation about stopping rules in PCT group. No response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | The randomisation was performed using a computer‐based random number generation. |
| Allocation concealment (selection bias) | Low risk | Allocation was issued using opaque, sealed, numbered envelopes. |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | |
| Selective reporting (reporting bias) | High risk | 8/39 (20%) patients excluded from intervention versus 3/40 (7%) from control; 4 patients excluded from intervention for "complicated infections", which is likely to have biased the results on duration of antibiotic treatment. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | PCT only measured for intervention group. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in hospital CLINICAL PROBLEM: reduce cases of Clostridium difficile‐associated disease in hospital by restricting use of parenteral antibiotics SETTING: 1 teaching hospital in the USA | |
| Interventions | FORMAT: no valid prescribing data. Restriction and educational outreach ‐ review and recommend change DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce inappropriate | |
| Outcomes | MICROBIAL: incidence of Clostridium difficile‐associated disease | |
| Notes | FINANCIAL SUPPORT: Funding: Merit Review Funding and Department of Veterans Affairs. Competing Interests: none declared ADDITIONAL DATA: email from authors but no additional data Microbial ROB: MEDIUM: Case definition Low, Planned intervention Low, Other infection control High | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | MRSA control programme introduced simultaneously. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine data from microbiology computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine data from microbiology computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine data from microbiology computer |
| Free of selected reporting (ITS) ? | Low risk | Routine data from microbiology computer |
| Free of other bias (ITS) ? | High risk | Only 6 months' data postintervention Microbial Outcome Risk of Bias Criteria: Case definition: DONE, CDC definition of C difficile. Planned intervention: DONE. Other infection control measures: NOT DONE, MRSA control programme introduced simultaneously. |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing POWER CALCULATION: yes, 58 participants per group. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: duration of treatment in days CLINICAL: mortality, recurrence of infection, ICU length of stay, hospital length of stay, nosocomial infection | |
| Notes | FINANCIAL SUPPORT: Funding: Minas Gerais Research Foundation (Fundação de Amparo à Pesquisa do Estado de Minas Gerais). Competing Interests: 1 author received payment for lectures from bioMérieux. No others declared. ADDITIONAL DATA: online Microsoft Word document with additional information about the criteria for stopping antibiotics, no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomisation was performed using a table of computer‐generated random numbers. |
| Allocation concealment (selection bias) | Low risk | Sealed, opaque envelopes were used for the randomisation. |
| Blinding (performance bias and detection bias) | High risk | Not blinded |
| Incomplete outcome data (attrition bias) | Low risk | 1 patient excluded from intervention and 2 from control. Outcomes measured on all other randomised participants. |
| Selective reporting (reporting bias) | Low risk | Duration of antibiotics measured from patient administration system. |
| Other bias | Unclear risk | "Patients showing reduction in SOFA and no sign of active infection were to receive no more than 7 days of antibiotic therapy. We used the biomarker‐guided protocols to further reduce this duration (i.e., to less than seven days)". This suggests that the ID physicians imposed a ceiling of 7 days' treatment for these patients for both intervention and control groups. Study did not achieve required recruitment. |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | PCT only measured for intervention group. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: hospital physicians PATIENTS: inpatients with LRTI, 107 randomised (55 intervention, 52 control) CLINICAL PROBLEM: admitted to hospital for treatment of LRTI SETTING: 2 Dutch hospitals | |
| Interventions | FORMAT: Interventions: educational meetings; dissemination of written information about study procedures, test characteristics discussed and results from previous studies; structural, rapid laboratory testing (PCR) for viral and atypical bacterial pathogens DELIVERER: specialist physician (Medical Microbiology) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, a total of 100 patients. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: % patients treated CLINICAL: mortality, median duration of antibiotic treatment FINANCIAL: cost of hospitalisation, all diagnostic and treatment costs | |
| Notes | FINANCIAL SUPPORT: Funding: Association of Academic Hospitals and the Dutch Health Insurance Council (grant 01233). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "Patients were randomly allocated ... by means of a computer generated table" |
| Allocation concealment (selection bias) | High risk | Allocation by investigators |
| Blinding (performance bias and detection bias) | High risk | Investigators were not blinded to patient randomisation. |
| Incomplete outcome data (attrition bias) | Low risk | Reported on all 107 patients |
| Selective reporting (reporting bias) | Low risk | |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | Test data only reported for intervention patients. |
| Baseline characteristics similar? | High risk | "slightly more patients in the intervention group had received previous antibiotic treatment ": 42% vs 23%, which is not "slighly more" |
| Methods | STUDY DESIGN: CITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospitals | |
| Interventions | FORMAT: Interventions: educational meetings (6 hospitals), dissemination of algorithms (3 hospitals), educational outreach by review and recommend change (2 hospitals), restrictive automatic stop order (1 hospital), unspecified "hospital wide restriction" (3 hospitals). NB the authors describe the intervention in 2 hospitals as "audit and feedback", but there was no feedback of data over time about progress to goal, just review with feedback about individual patients. Intervention Functions: education, enablement, persuasion, restriction | |
| Outcomes | PRESCRIBING: Choice: use of target antibiotics in DDD/1000 OBD and in days of therapy MICROBIAL: C difficile infection (cases per 10,000 OBD) | |
| Notes | FINANCIAL SUPPORT: Funding: Agency for Healthcare Research and Quality; US Department of Health and Human Services. Competing Interests: none declared ADDITIONAL DATA: email response from authors with additional data about the intervention used by each of the 6 intervention hospitals | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | High risk | Intervention targets and intervention design were different in each of the 6 hospitals. Microbial ROB MEDIUM: case definition low, planned intervention low, other infection control UNCLEAR |
| Methods | STUDY DESIGN: NRT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all staff in the ED | |
| Interventions | FORMAT: Intervention: structural, rapid laboratory test for influenza Intervention Function: environmental restructuring | |
| Outcomes | PRESCRIBING: Exposure: % children prescribed antibiotics | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: email response from authors but no additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Does not say how groups were allocated |
| Allocation concealment (selection bias) | Unclear risk | Says there was blinding but unclear who was blinded. |
| Blinding (performance bias and detection bias) | Unclear risk | Says there was blinding but unclear who was blinded. |
| Incomplete outcome data (attrition bias) | Low risk | All included |
| Selective reporting (reporting bias) | Low risk | Yes, all outcomes reported. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No baseline outcome data |
| Free of contamination? | High risk | Within same ward |
| Baseline characteristics similar? | Low risk | Yes, Table 1 |
| Methods | STUDY DESIGN: cluster RCT, stepped wedge, service level Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in 6 hospital services | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change; reminders (circumstantial, physical, written recommendation on each patient reviewed) NB the authors describe their intervention as "audit and feedback", but there was no feedback of data over time about progress to goal, just review with feedback about individual patients. POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Choice: use of target antibiotics in days of therapy/1000 OBD MICROBIAL: Clostridium difficile infection and infection with antibiotic‐resistant organisms FINANCIAL: time required to implement the intervention in critical‐care wards is described in Elligson 2012a. | |
| Notes | FINANCIAL SUPPORT: Funding: Ontario Ministry of Health and Canadian Institutes of Health Research. Competing Interests: none declared ADDITIONAL DATA: email response with additional details about the intervention from authors including Elligson 2012a describing the design and cost of implementing the intervention in critical‐care wards Microbial Risk of Bias: MEDIUM: case definition low, unplanned intervention low, other infection control UNCLEAR | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | The order of implementation of the intervention on the 6 clinical services was determined by random number generation performed by a statistician uninvolved in daily stewardship activities. |
| Allocation concealment (selection bias) | Low risk | Following a 6‐month control period during which none of the services received antimicrobial stewardship (1 May 2010 to 31 October 2011), the intervention was introduced to each additional service at 1‐month intervals, beginning on 1 November 2010. By 1 April 2011, clinical rollout was complete. |
| Blinding (performance bias and detection bias) | High risk | Blinding was not possible. |
| Incomplete outcome data (attrition bias) | Low risk | Prescribing outcome data were from pharmacy computer. |
| Selective reporting (reporting bias) | Low risk | Prescribing outcome data were from pharmacy computer. |
| Other bias | Low risk | Unit of analysis was service, and clustering was included in the model. "Negative binomial regression, accounting for clustering at the level of service using random effects as well as for secular and seasonal trends, was used to compare overall targeted antimicrobial utilization in the control and intervention periods for the analysis involving patients qualifying for the stewardship intervention as well as the analysis of all admitted patients. The unit of analysis was each service’s mean monthly targeted days of therapy count. The covariates included in these multivariable models were study period, study month (as a continuous variable), and season" |
| Baseline Outcomes similar? | Low risk | Table 4 |
| Free of contamination? | High risk | Contamination could have occurred during the rollout of intervention over 6 months. |
| Baseline characteristics similar? | Low risk | |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: no reliable prescribing data. The intervention was removal of restriction, but only 1 prescribing outcome data point during restriction and 3 after restriction lifted. | |
| Outcomes | MICROBIAL: monthly MRSA rate (%) | |
| Notes | FINANCIAL SUPPORT:Funding: Centre Hospitalier Universitaire de Caen and the French Health Ministry (Programme Hospitalier de Recherche Clinique National). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | MRSA data from microbiology computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | MRSA data from microbiology computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | MRSA data from microbiology computer |
| Free of selected reporting (ITS) ? | Low risk | MRSA data from microbiology computer |
| Free of other bias (ITS) ? | Low risk | MICROBIAL RISK OF BIAS: case defiinition Low, planned intervention Low, other infection control Low, use of alchohol‐based hand rub (ABHR) unchanged during period of fluoroquinolone restriction (2001‐2) and for 3 years after restriction lifted (2003‐5). Data are also presented for a further 6 years of increased use of ABHR (2006‐11). |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all paediatric physicians in the hospitals | |
| Interventions | FORMAT: Intervention: publication of American Academy of Pediatricians (AAP) bronchiolitis guidelines Intervention Functions: education but no information about how the guidelines were disseminated | |
| Outcomes | PRESCRIBING: Exposure: % children treated with antibiotics | |
| Notes | FINANCIAL SUPPORT: Funding: Academic Pediatric Association. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data. AAP 2006 Bronchiolitis Guidelines downloaded. | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from patient administration systems |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from patient administration systems |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from patient administration systems |
| Free of selected reporting (ITS) ? | Low risk | Data from patient administration systems |
| Free of other bias (ITS) ? | Low risk | Monthly data points for 20 months' pre‐ and 60 months' postintervention. "Guideline published in October 2006. Study phases: preguideline (November 2004 to March 2005), postguideline early (November 2007 to March 2008), and postguideline late (November 2011 to March 2012). These time periods were selected for the unadjusted analysis because they represent 3 bronchiolitis seasons, before and after guideline publication; the 2006 to 2007 season was not included because this is the year the guideline was published and was a period of distribution and assimilation. For the adjusted segmented regression analysis, publication of the guidelines, October 2006, was considered the event point." |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by review and recommend change; reminders (physical and verbal, posters and intervention promoted at weekly ward meetings) DELIVERER: pharmacist | |
| Outcomes | PRESCRIBING: Choice: expenditure on oral co‐amoxiclav | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Only 5 months' pre‐intervention data, so secular changes possible. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: cluster RCT, service level Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: antibiotic prescribing SETTING: 3 hospitals in 3 countries: Israel, Germany, and Italy | |
| Interventions | FORMAT: Intervention: reminders (circumstantial, triggered by prescription of antibiotics); structural, computer decision support system Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 1500 patients with microbiologically documented infections. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Choice: appropriate antibiotic treatments COST: Costs, which included the estimated ecological cost of inappropriate antibiotic treatment CLINICAL: Balancing: length of stay, 30‐day mortality | |
| Notes | FINANCIAL SUPPORT: Funding: EU Fifth Framework, Information Society Technologies, IST‐9999‐11459. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "Wards were randomly allocated ... by drawing a random code from a closed opaque box" |
| Allocation concealment (selection bias) | High risk | Allocation could not be concealed. |
| Blinding (performance bias and detection bias) | Low risk | Primary outcome was measured by the CDSS. |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all patients. |
| Selective reporting (reporting bias) | Low risk | The primary outcome was objective, based on whether or not the prescriber selected one of the CDSS top 3 recommendations. |
| Other bias | High risk | Trial was underpowered for the primary outcome measure. Adjustment of drug costs for changes in prices not necessary because the intervention lasted only 6 months. |
| Baseline Outcomes similar? | Low risk | Table 1, cohort study before trial |
| Free of contamination? | Low risk | Only intervention wards had CDSS. |
| Baseline characteristics similar? | Low risk | |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: restrictive, no valid prescribing data DELIVERER: specialist physician | |
| Outcomes | MICROBIAL: cases of CDAD per month (ITS data). Prevalence of clindamycin‐resistant Clostridium difficile | |
| Notes | FINANCIAL SUPPORT: none | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Enough data to account for seasonal variation, and infection control measures did not change over study period. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: run chart with no statistical analysis. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | High risk | Not done, the method of detection of C difficile toxin changed from cell culture assay in the first 4 years of the study to a latex test in the final year (5 months after the start of clindamycin restriction). |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | High risk | Not done, change in method of testing for C difficile during the study period (see case definition). |
| Free of other bias (ITS) ? | High risk | Microbial Outcome Risk of Bias Criteria: Case definition: NOT DONE Infection: diarrhoea with positive assay for C difficile cytotoxin and antibiotic therapy within the previous 60 days. However, the method of detection of toxin changed from cell culture assay in the first 4 years of the study to a latex test in the final year (5 months after the start of clindamycin restriction). Planned intervention: NOT DONE Response to an outbreak of CDAD starting 12 months before restriction. Other infection control, isolation, and IC practices: DONE Infection control measures were identical in the year before and after the start of clindamycin restriction. Hospital staff education and increased availability of gloves and improvement of environmental hygiene were implemented a year before restriction of clindamycin with no apparent impact on the frequency of new cases. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: physicians, surgeons, paediatricians, obstetricians‐gynaecologists, and intensivists | |
| Interventions | FORMAT: Interventions:Intervention 1: reminders (posters, not circumstantial); educational meetings and dissemination of guidelines; restrictive by expert approval. Intervention 2: reminder (circumstantial, on blood pressure cuffs in operating theatre); educational meetings and dissemination of guidelines Intervention Functions:Intervention 1: education, environmental restructuring, persuasion. Intervention 2: education, enablement, environmental restructuring, persuasion DELIVERER: pharmacists | |
| Outcomes | PRESCRIBING: Choice: reduction in incidence of incorrect antibiotic prescriptions (dosing intervals and timing of surgical prophylaxis). | |
| Notes | FINANCIAL SUPPORT: Funding: International Clinical Epidemiology Network (INCLEN, grant #1004‐97‐6501) and by Pontificia Universidad Javeriana (grant #12‐24‐01‐ 31). Competing Interests: no information | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: ARIMA analysis, selected in preference to segmented regression analysis because of nonlinear outcome data. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the surgical ICU CLINICAL PROBLEM: excessive antibiotic use SETTING: surgical ICU in a university hospital in Hungary | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change; restrictive by expert approval DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Exposure: total antibiotic consumption (DDD per 100 patient days) | |
| Notes | FINANCIAL SUPPORT: no information | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Routine data from pharmacy |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Routine data from pharmacy |
| Incomplete outcome data addressed (ITS) ? | Low risk | Routine data from pharmacy |
| Free of selected reporting (ITS) ? | Low risk | Routine data from pharmacy |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Intervention: restrictive by expert approval Intervention Function: restriction | |
| Outcomes | PRESCRIBING: Choice: use of cephalosporins in DDD/100 OBD MICROBIAL: % ESBL‐producing Klebsiella pneumoniae and Escherichia coli | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH case definition Low, planned intervention Low, other infection control Unclear, no data about other infection control measures | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Unclear risk | 1 year (6 x 2‐monthly time points) pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all prescribers in the hospital | |
| Interventions | FORMAT: Intervention: restriction by removal from availability in the hospital Intervention Function: restriction DELIVERER: Infection Control Committee COMPARISON: pre‐intervention DESIRED CHANGE: reduction in use of targeted carbapenems and in resistance | |
| Outcomes | PRESCRIBING: Choice: use of target antibiotics MICROBIAL: carbapenem resistance in Pseudomonas aeruginosa | |
| Notes | FINANCIAL SUPPORT: Funding: Fundo de Incentivo a Pesquisa e Eventos, Hospital de Clinicas de Port Alegre. Competing Interests: none declared | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | No outbreak or other changes coincident with intervention |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Low risk | Data for 18 months' pre‐ and 3 years' postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: reduce linezolid use SETTING: 1 hospital in the USA | |
| Interventions | FORMAT: Interventions:Intervention 1: educational mettings or dissemination of educational materials. Intervention 2: reminders, structural, circumstantial ‐ computerised physician order entry system (CPOE) and educational meetings or dissemination of educational materials Intervention Functions:Intervention 1: education. Intervention 2: education, enablement, environmental restructuring DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: reduce inappropriate | |
| Outcomes | PRESCRIBING: Choice: linezolid use (DDD per 1000 patient days) | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Yes, reports on all likely influencing interventions. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Yes, the point of analysis is the point of the intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Yes, pharmacy data used both pre‐ and postintervention. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome is objective. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Yes, pharmacy data, so should be complete. |
| Free of selected reporting (ITS) ? | Low risk | Yes, all outcomes reported. |
| Free of other bias (ITS) ? | High risk | < 1 year data for phases 1 and 2 |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: doctors in the ED | |
| Interventions | FORMAT: Interventions: structural, rapid influenza testing DELIVERER: specialist physicians, Department of Pediatrics COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Exposure: % children treated | |
| Notes | FINANCIAL SUPPORT: Funding: New Vaccine Surveillance Network and Robert Wood Johnson Generalist Physicians Faculty Scholars Program. Competing Interests: none declared | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Random number generator |
| Allocation concealment (selection bias) | Unclear risk | Does not say, but possibly not due to nature of study |
| Blinding (performance bias and detection bias) | Unclear risk | No information |
| Incomplete outcome data (attrition bias) | Low risk | From records, outcomes on all included children |
| Selective reporting (reporting bias) | Low risk | From records, outcomes on all included children |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No baseline outcomes taken. |
| Free of contamination? | Low risk | Influenza testing only on children in intervention group |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines; reminders (physical, posters, and on intranet) | |
| Outcomes | PRESCRIBING: Choice: use of ciprofloxacin in DDD/1000 OBD CLINICAL: mortality, re‐admission (cohort data) | |
| Notes | FINANCIAL SUPPORT: Funding: commercial Merck, Pfizer, Astellas, and the Medbuy Corporation. Hamilton Health Sciences Foundation (Jack Hirsh Fellowship). Competing Interests: 1 author received honoraria from Merck and Astellas for lectures. All other authors: none to declare ADDITIONAL DATA: email response from authors with guideline and additional data about the intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: dissemination of guidelines; restrictive by removal and expert approval Note that the published paper says: "The policy was widely disseminated in the hospital but no specific measures were put in place to enforce compliance". However, the antibiotic policy provided by the authors says: “Cephalosporins and fluoroquinolones. These agents will NOT be ward stock on any general medical or surgical wards – continuation of therapy beyond 24 hours (in Medicine) and single dose prophylaxis (in Surgery) requires consultant review, prescription by consultant and discussion with Micro ID”. DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: use of cephalosporins and quinolones (combined) in DDD/1000 OBD MICROBIAL: Clostridium difficile infection | |
| Notes | FINANCIAL SUPPORT: Funding: part commercial, Optimer Pharmaceuticals and US Department of Veterans Affairs. Competing Interests: 1 author declared multiple commercial sources of research funding and held patents relevant to C difficile infection licensed to ViroPharma ADDITIONAL INFORMATION: authors provided the 2008 version of the hospital antibiotic policy, which included details about the restrictions on use of target drugs Microbial Risk of Bias MEDIUM: case definition yes, planned intervention yes, infection control no (a cohorting ward was introduced at the same time as the antibiotic policy) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | For prescrbiing outcome |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Pharmacy computer |
| Free of other bias (ITS) ? | High risk | 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the Medical ICU | |
| Interventions | FORMAT: Interventions: audit and feedback; educational outreach by academic detailing; reminders (physical, circumstantial, stickers placed in notes of patients receiving target antibiotics) DELIVERER: departmental physician (ICU consultant) COMPARISON: usual care DESIRED CHANGE: increase appropriate | |
| Outcomes | PRESCRIBING: Choice: % appropriate treatment | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL INFORMATION: email from authors with additional information about intervention. The intervention design is described in more detail in Pulcini 2008. | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | |
| Unlikely to affect data collection (ITS) ? | Low risk | Primary outcome was appropriateness of treatment at 24 to 96 hours, which was the same in pre‐ and postintervention period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Dual data entry, the ICU consultant was blinded to study period, although the ID physician was not. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data on all participants |
| Free of selected reporting (ITS) ? | Low risk | No evidence of selective reporting |
| Free of other bias (ITS) ? | High risk | < 1 year of data (25 weeks) in the pre‐ and postintervention phases |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the ICU CLINICAL PROBLEM: PCT for guiding duration of antibiotic therapy SETTING: 1 hospital in China | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing DELIVERER: department physician (ICU) COMPARISON: usual care DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Exposure: duration of all antibiotic treatment CLINICAL: Balancing: mortality and length of stay FINANCIAL: cost of hospitalisation, but no information about cost of intervention | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Says it was randomised, but no further information |
| Allocation concealment (selection bias) | Unclear risk | No information |
| Blinding (performance bias and detection bias) | Unclear risk | No mention of blinding |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes on all 71 participants |
| Selective reporting (reporting bias) | Low risk | |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No information |
| Free of contamination? | Low risk | PCT results only reported for intervention participants. |
| Baseline characteristics similar? | Low risk | Yes, Table 1 |
| Methods | STUDY DESIGN: NRT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: receiving treatment with piperacillin/tazobactam, imipenem, and meropenem SETTING: 1 hospital in Thailand | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce excessive | |
| Outcomes | PRESCRIBING: Choice: use of target antibiotics CLINICAL: Balancing: mortality, length of stay FINANCIAL: cost of target antibiotics and all antibiotics. No information about cost of intervention | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: email from authors but no additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | By hospital number, even number in last digit received intervention |
| Allocation concealment (selection bias) | High risk | Not concealed |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | Complete data reported. |
| Selective reporting (reporting bias) | High risk | Some outcomes (favourable clinical outcome, death from infection) subject to selective outcome reporting. No discussion of why there was a significant difference for death because of infection but no difference in the % of patients alive on discharge from hospital. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | Baseline frequency of inappropriate treatment was 50% in January 2007, but no information about risk of inappropriate treatment in the control group in August 2007. |
| Free of contamination? | High risk | Invervention and control participants were in the same hospital, and physicians were likely to have patients in both groups. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital CLINICAL PROBLEM: unnecessary double coverage for infection with anaerobic bacteria SETTING: 1 hospital in Thailand | |
| Interventions | FORMAT: Interventions: restrictive by prior approval, the intervention was removal of this restriction | |
| Outcomes | PRESCRIBING: Choice: cumulative incidence of unnecessary treatment in DDD/admission | |
| Notes | FINANCIAL SUPPORT: Funding: National Institutes of Health grant K24‐AI080942. Competing Interests: 1 author received research support from Merck, Ortho‐McNeil, Cubist, and AstraZeneca. ADDITIONAL DATA: email response from authors but no additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 1 year of data pre‐ and postintervention |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | High risk | Primary outcome (unnecessary DACT) not objective. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | Primary outcome (unnecessary DACT) not objective. |
| Incomplete outcome data addressed (ITS) ? | High risk | Figure 1 includes 4 months with no unnecessary DACT, but it is not clear whether this was because there was no DACT or because all DACT was necessary. With the exception of July 2008, these months had relatively high use of ampicillin/sulbactam and metronidazole, so suggests they missed some DACT patients. |
| Free of selected reporting (ITS) ? | Unclear risk | Not clear if outcome was reported on all patients. |
| Free of other bias (ITS) ? | Low risk | |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines; reminders (circumstantial and physical, on computer order form when prescribing antibiotics); restrictive by compulsory order form, expert approval, and removal | |
| Outcomes | PRESCRIBING: Choice: Primary: use of cefotaxime or ceftriaxone Secondary: use of other antibiotics: gentamicin, benzyl penicillin, carbapenems, piperacillin, ticarcillin, and ciprofloxacin FINANCIAL: cost of intervention | |
| Notes | FINANCIAL SUPPORT: Funding: Royal Melbourne Hospital. Competing Interests: none declared. ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | 8 months data pre‐intervention, 15 months postintervention, not enough to adjust for seasonal variation |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with Kruskal‐Wallis test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital. Number, age, and time since qualification NOT CLEAR. 3 intensive care units, 3 general medical, and 1 general surgical | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change | |
| Outcomes | PRESCRIBING: Choice: % episodes of vancomycin use deemed inappropriate | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Data only collected for 3 months pre‐ and 6 months postintervention, so secular/seasonal changes possible. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Unclear risk | The reliability of the assessment of appropriate vancomycin use was not reported. |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | Retrospective assessment of appropriateness without concealment of study phase. |
| Incomplete outcome data addressed (ITS) ? | High risk | Assessment of appropriateness from retrospective assessment of all patients treated in 1 month but only done every 4 to 6 months. |
| Free of selected reporting (ITS) ? | Unclear risk | Not clear, data were only collected intermittently. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all paediatricians in the hospitals | |
| Interventions | FORMAT: Intervention: dissemination of national guidelines Intervention Function: education | |
| Outcomes | PRESCRIBING: Choice: % patients treated with guideline‐recommended antibiotics | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: 1 author has received research funding from Merck and Cubist and has served as a consultant for Merck, Pfizer, Astellas Pharma, and Cubist, and 3 authors have received research funding from Pfizer. ADDITIONAL DATA: email from authors with no additional data. Paediatric infectious diseases guidelines available online (cid.oxfordjournals.org/content/early/2011/08/30/cid.cir531.full) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from Pediatric Health Information System |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from Pediatric Health Information System |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from Pediatric Health Information System |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from Pediatric Health Information System |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational meetings and dissemination of protocol; reminders (physical, pocket‐size guideline); restrictive by compulsory order form and expert approval COMPARISON: data for 3 years after implementation of the programme | |
| Outcomes | PRESCRIBING: Choice: anti‐infective expenditure per hospital patient | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 4 years' data pre‐ and 3 years' data postintervention, so enough data to account for seasonal change |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with Fisher's exact test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | High risk | There is no information about change in price of antibiotics over the study period. |
| Free of selected reporting (ITS) ? | Unclear risk | The intervention was targeted at specific antibiotics, but no information is provided about their use or cost. |
| Free of other bias (ITS) ? | Unclear risk | No adjustment of antibiotic costs for change in price, so change in price of antibiotics (rather than change in use) over the study period may have been responsible for reduction in cost per patient over the study period. No data about number of admissions pre‐intervention. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines; educational outreach by academic detailing; reminders (circumstantial, physical, and verbal: newsletters, posters, pocket charts, educational rounds, and triggered by prescribing of target drugs); reminders (physical); restrictive by compulsory order form plus automatic 3‐day stop order for all antibiotics and review and make change (therapeutic substitution of selected drugs) | |
| Outcomes | PRESCRIBING: Choice: vancomycin and ceftazidime use in units, antibiotic cost as a percentage of total drug cost | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | > 12 months' data pre‐ and postintervention, enough to account for seasonal change |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: cluster RCT, hospital level Risk of Bias: HIGH | |
| Participants | PROVIDERS: doctors managing patients with community‐acquired pneumonia | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guideline; reminders (physical, posters, electronic and pocket versions of guideline) DESIRED CHANGE: increase in compliance of initial treatment with guideline recommendation and decrease in duration of treatment POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Choice: % guideline compliant for initial treatment CLINICAL: Balancing: mortality, length of stay | |
| Notes | FINANCIAL SUPPORT: Funding: German Medical Assembly grant 06‐69 and German Federal Ministry of Education and Research grant 01K10103‐105. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Computer, by hospital |
| Allocation concealment (selection bias) | Low risk | |
| Blinding (performance bias and detection bias) | Unclear risk | Not clear who collected outcome data or whether they were blinded to allocation |
| Incomplete outcome data (attrition bias) | Unclear risk | All outcome data given as %, so unclear if some patients were missing. |
| Selective reporting (reporting bias) | Unclear risk | No details of data collection |
| Other bias | High risk | Intervention period (1 April 2007 to 29 February 2008) was different than control period (1 September 2006 to 28 February 2007). |
| Baseline Outcomes similar? | High risk | Duration of inpatient antibiotic at baseline was appropriate in only 47% intervention (versus 57% control). |
| Free of contamination? | Low risk | Randomised by site |
| Baseline characteristics similar? | High risk | 75% inpatients in control group versus 50% for intervention, also fewer CURB 0 and more CURB 3 |
| Methods | STUDY DESIGN: cluster RCT, hospital level Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guideline; educational outreach by academic detailing; reminders (physical, desktop on computers, and pocket card) POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Choice: % patients compliant with guideline for selected drug, timing (within 4 h of presentation), switching from IV to oral and streamlining CLINICAL: Balancing: mortality, length of stay | |
| Notes | FINANCIAL SUPPORT: Funding: The Netherlands Organisation for Health Research and Development (Zon/Mw; 2300.0024). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Blinded researcher coin flip, hospital level |
| Allocation concealment (selection bias) | Low risk | Allocation at hospital level |
| Blinding (performance bias and detection bias) | Unclear risk | No information |
| Incomplete outcome data (attrition bias) | Low risk | Outcome data for all patients |
| Selective reporting (reporting bias) | Unclear risk | All relevant outcomes reported. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Low risk | Table 3, also pair‐matched clusters for important variables |
| Free of contamination? | Low risk | Allocation at hospital level |
| Baseline characteristics similar? | Low risk | No clinically relevant differences |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the ICU | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Exposure: duration of antibiotic treatment (days) CLINICAL: Balancing: length of hospital stay | |
| Notes | FINANCIAL SUPPORT: Funding: none declared. Competing Interests: 1 author had speaking engagements for B.R.A.H.M.S AG. ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | No information |
| Allocation concealment (selection bias) | Unclear risk | No information |
| Blinding (performance bias and detection bias) | High risk | Not blinded |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all patients. |
| Selective reporting (reporting bias) | Low risk | Outcomes reported on all patients. |
| Other bias | High risk | Only 27 of 125 screened patients were randomised. |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | PCT only measured for intervention patients. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing POWER CALCULATION: yes, 1002 participants total. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: % patients treated | |
| Notes | FINANCIAL SUPPORT: Funding: grant SNF 3200BO‐116177/1 from the Swiss National Science Foundation and contributions from santésuisse and the Gottfried und Julia Bangerter‐Rhyner Foundation and participating hospitals. B.R.A.H.M.S Inc, the major manufacturer of the procalcitonin assay, provided all assay‐related material, Kryptor machines if not already available on site, and kits and maintenance required for 10,000 measurements related to the study. Competing Interests: 3 authors received support from B.R.A.H.M.S Inc to attend meetings and fulfil speaking engagements, and 1 author served as a consultant and received research support from B.R.A.H.M.S Inc. ADDITIONAL DATA: authors provided additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Prespecified, computer‐generated randomisation list |
| Allocation concealment (selection bias) | Low risk | Centralised, password‐protected website |
| Blinding (performance bias and detection bias) | Low risk | Password‐protected website with instructions for PCT and control groups |
| Incomplete outcome data (attrition bias) | Low risk | Only 1 of 1381 patients lost to follow‐up; 16 (2%) patients in PCT group and 6 (1%) patients in control group withdrew. |
| Selective reporting (reporting bias) | Low risk | > 95% surviving patients completed 30‐day interview. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No baseline outcome data |
| Free of contamination? | High risk | The study was conducted in 6 hospitals, but patients in each hospital were in both intervention and control groups. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT: Interventions: reminders (circumstantial and physical, point‐of care electronic prompt (triggered by operating room admission)); reminders (physical); restrictive; structural | |
| Outcomes | PRESCRIBING: Choice: % patients with first dose administered within 1 hour of incision CLINICAL: Intended: surgical‐site infection | |
| Notes | FINANCIAL SUPPORT: Funding: Lehigh Valley Hospital Network and Allentown Anesthesia Associates. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Statistical process control charts |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Electronic data for prescribing |
| Knowledge of the allocation adequately prevented(ITS)? | Unclear risk | Infection control personnel were blinded for assessment of wound infection, unsure about compliance data. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Electronic data for prescribing |
| Free of selected reporting (ITS) ? | Low risk | Electronic data for prescribing |
| Free of other bias (ITS) ? | Low risk | 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the long‐term care facility | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines and treatment algorithms; reminders (physical, pocket guidelines) The guideline has 16 algorithms for management of clinical problems (fever, leukocytosis, confusion, diarrhoea) and common infections in older people. | |
| Outcomes | PRESCRIBING: Exposure: antibiotic days/100 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: Chicago Antimicrobial Resistance Project, Centers for Disease Control and Prevention (U50/ CCU515853). Competing Interests: no information ADDITIONAL DATA: email response with the guideline. The guidelines are supposed to be available online, but the link does not work. | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Days of antimicrobial use calculated automatically by pharmacy computer. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Days of antimicrobial use calculated automatically by pharmacy computer. |
| Incomplete outcome data addressed (ITS) ? | Unclear risk | 1 and 2 January 2000 start data were censored, but this was reported and would have little impact on the other 48 data points. |
| Free of selected reporting (ITS) ? | Unclear risk | Days of antimicrobial use calculated automatically by pharmacy computer. |
| Free of other bias (ITS) ? | High risk | Only 10 months' pre‐intervention data, so secular trends could not be addressed. |
| Methods | STUDY DESIGN: RCT Risk of bias: MEDIUM | |
| Participants | PROVIDERS: residents on medical and surgical wards CLINICAL PROBLEM: adult patients receiving IV antibiotics for 3 to 4 days with no modification since starting treatment SETTING: single 800‐bed university hospital in Switzerland. Data collected over 5 months. POWER CALCULATION: yes, 135 patients in each group, but the trial was underpowered because the observed effect was lower than predicted. Details in Appendix 3 | |
| Interventions | FORMAT: Interventions: dissemination of questionnaire about guidelines; reminders (circumstantial and physical, questionnaire mailed to the resident in charge of patients who were receiving IV antibiotic treatment. The questionnaire asked 3 questions regarding possible adaptation of antibiotic therapy on day 3 or 4, and was collected 24 hours later. If the resident had not yet completed it at that time, he/she was reminded once to do so.) DELIVERER: AMT COMPARISON: control patients with no intervention DESIRED CHANGE: reduction in established management (reduction in duration of IV therapy) TIMING: intervention at the point of decision making (potential modification 3 to 4 days after start of antibiotics) | |
| Outcomes | PRESCRIBING: Choice: % of patients discontinuing IV antibiotics and hazard ratio adjusted for patients' Karnofsky functional index | |
| Notes | FINANCIAL SUPPORT: Funding: Quality Improvement Committee of the Lausanne University Hospital and grant 32–63128.00 of the Swiss National Science Foundation. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "Patients allocated ... by using a computer generated randomizations list" |
| Allocation concealment (selection bias) | Low risk | "Concealment of allocation was achieved as the physician in charge of the patient was involved after randomizations" |
| Blinding (performance bias and detection bias) | High risk | "This was a randomised, controlled, open trial" |
| Incomplete outcome data (attrition bias) | Low risk | Primary outcome measure (duration of IV antibiotics) collected on all patients. Only 70% of questionnaires returned for the intervention group, which could account for the intervention effect being lower than expected. However, this did not affect outcome assessment. |
| Selective reporting (reporting bias) | Low risk | Complete primary outcome data |
| Other bias | High risk | The study was underpowered. |
| Baseline Outcomes similar? | Low risk | Pre‐study group, data collected for 2 months before intervention to estimate the magnitude of possible observation bias (Figure 2). |
| Free of contamination? | Low risk | The pre‐intervention group data were comparable to the control group, suggesting minimal observation bias. |
| Baseline characteristics similar? | Low risk | Presented in Table 1 |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the ICU | |
| Interventions | Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing DELIVERER: departmental physicians (ICU) POWER CALCULATION: yes, 165 participants per group. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: duration of antibiotic treatment (days) CLINICAL: mortality, re‐admission, and length of hospital stay | |
| Notes | FINANCIAL SUPPORT: Funding: Intensive Care Foundation of Australia and New Zealand. Material support was provided by Roche Diagnostics, Thermo Fisher Scientific, and bioMérieux. Roche Diagnostics and Thermo Fisher Scientific provided additional unrestricted grant funding. Competing Interests: none declared ADDITIONAL DATA: email from authors with additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Patients were variable block randomised 1:1 via a secured central study website into either a PCT‐guided (PCT) group or clinician‐guided (standard care) group. Randomisation was stratified according to the presence of septic shock (defined by the receipt of inotropes and/or any vasopressors within the previous 24 hours). |
| Allocation concealment (selection bias) | Low risk | See above |
| Blinding (performance bias and detection bias) | High risk | Single‐blind |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on 196/200 intervention and 198/200 control participants. |
| Selective reporting (reporting bias) | Low risk | |
| Other bias | Low risk | 1567 patients screened, but 1167 excluded; full details of how many patients met each of the exclusion criteria (Figure 1) |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | PCT only reported for intervention participants. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians on 2 respiratory wards | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Choice: score on 6 indicators of inappropriate antibiotic use: indication, choice, dosage, dosing schedule, duration, conversion CLINICAL: Balancing: length of stay FINANCIAL: cost (mean, SD) of antibiotics and total patient costs | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Says it was randomised, no further information. |
| Allocation concealment (selection bias) | High risk | Patients from 2 wards were randomised, and there is no information about allocation concealment. |
| Blinding (performance bias and detection bias) | Low risk | "At the end of the study, a blinded coordinating investigator recorded the patients’ data" |
| Incomplete outcome data (attrition bias) | Low risk | All outcomes reported. |
| Selective reporting (reporting bias) | Low risk | All outcomes reported. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No information |
| Free of contamination? | High risk | Intervention and control patients were on both wards. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: RCT with nested ITS analysis (Figures 3 and 4) Risk of Bias: HIGH | |
| Participants | PROVIDERS: unit of randomisation ‐ 396 physicians in 7 specialties. Non‐physicians (nurses or pharmacists) who were authorised to enter orders that required eventual signing off by physicians were also randomised. | |
| Interventions | FORMAT: Interventions: dissemination of guideline; reminders (circumstantial, delivered through computer screen at the time of physician order entry and after 72 hours of therapy). The reminder required prescribers to produce a response: when someone would enter an order for intravenous vancomycin, a pop‐up screen would appear and display the appropriate indications for vancomycin use, which was a checkbox list of indications based on CDC guidelines. Users had to pick a reason or enter free text under 'other' in order to proceed. Intervention Functions: education, enablement, environmental restructuring DELIVERER: AMT POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Choice: initiation and renewal of vancomycin therapy. Duration of vancomycin therapy on a per‐prescriber basis. Total use of vancomycin in the hospital. FINANCIAL: estimated savings | |
| Notes | FINANCIAL SUPPORT: Funding: grant R01‐HS08927 from the Agency for Healthcare Policy and Research. Competing Interests: no information ADDITIONAL DATA: email from authors with additional details about the intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "The study was a randomised controlled trial"; no details on how randomisation sequence was generated |
| Allocation concealment (selection bias) | High risk | States "possible that physicians in the control group could learn of the intervention from physicians in the study group" |
| Blinding (performance bias and detection bias) | High risk | Not done |
| Incomplete outcome data (attrition bias) | Unclear risk | Not clear for primary outcome |
| Selective reporting (reporting bias) | Low risk | Based on numbers of vancomycin orders |
| Other bias | Low risk | No issues noted. |
| Baseline Outcomes similar? | Unclear risk | No information about pre‐intervention vancomycin use |
| Free of contamination? | High risk | States "possible that physicians in the control group could learn of the intervention from physicians in the study group" |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: RCT, allocation by patient Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians on 1 ICU | |
| Interventions | FORMAT: Intervention: restrictive by expert approval and review and make change Intervention Function: restriction COMPARISON: choice, number, and duration of antibiotics at the discretion of the care providers POWER CALCULATION: yes, 88 patients per group. The study was terminated early. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: total duration of all antibiotic treatment CLINICAL: Balancing: mortality, length of ICU stay MICROBIAL: number of patients with "antimicrobial resistance and/or superinfections" from randomisation until hospital discharge FINANCIAL: total costs of care for patients with CPIS < 6 at 3 days and no extrapulmonary infections | |
| Notes | FINANCIAL SUPPORT: Funding: Bayer Corporation. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "Patients were randomized to either the control group or experimental group"; no information about how randomisation sequence was generated |
| Allocation concealment (selection bias) | Unclear risk | Not stated |
| Blinding (performance bias and detection bias) | High risk | Page 509: "Because the study was not blinded, physicians and care providers could see the results" |
| Incomplete outcome data (attrition bias) | Low risk | Most outcomes are reported for 78 (96%) episodes of care; antimicrobial resistance and superinfection in 74 (91%) of episodes. |
| Selective reporting (reporting bias) | Low risk | No problems found. |
| Other bias | High risk | Microbial Risk of Bias HIGH. Case definition for microbial outcome NOT CLEAR: "Follow‐up respiratory cultures or cultures from clinical specimens performed 7 to 28 d after initiation of antibiotics were evaluated to assess the emergence of antimicrobial resistance or superinfections. Emergence of resistance was defined as the detection of new antimicrobial resistance pattern in the old or previously isolated organism. Superinfection was defined as the detection of the following organisms not present at study entry: Acinetobacter species, Serratia marcescens, Pseudomonas aeruginosa, Stenotrophomonas maltophilia, Enterobacter species, Citrobacter species, methicillin‐resistant Staphylococcus aureus (MRSA), Enterococcus species, and Candida species." It is therefore impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics. Infection control NOT CLEAR. Planned intervention YES |
| Baseline Outcomes similar? | Unclear risk | Not stated, no information about pre‐intervention duration of antibiotic treatment |
| Free of contamination? | Unclear risk | Not stated |
| Baseline characteristics similar? | Low risk | See Table 1 in study. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of antimicrobial order form; educational outreach by review and recommend change of cases of inappropriate prescribing by ID consultant; restrictive by compulsory order form Figure 2 suggests that expenditure increased sharply in the final year of the study when ID consultant review ceased. DELIVERER: specialist physician (ID) COMPARISON: data for 4 years' pre‐restriction | |
| Outcomes | PRESCRIBING: Choice: restricted drugs cost in million THB/200,000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: Ramathibodi Research Fund. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 4 years' data pre‐ and postintervention |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: run charts with no statistical analysis. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | High risk | NOT DONE, there is no information about change in price of antibiotics over the study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Unclear risk | NOT CLEAR, no adjustment of antibiotic costs for change in price, so change in price of antibiotics (rather than change in use) over the study period may have been responsible for some of the change in cost. Data were not adjusted for number of admissions or occupied bed days. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: physicians (numbers not clear) CLINICAL PROBLEM: adult patients receiving imipenem treatment SETTING: 1 hospital in the USA | |
| Interventions | FORMAT: Intervention: educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: pharmacist COMPARISON: usual care in the pre‐intervention phase DESIRED CHANGE: reduce excessive | |
| Outcomes | PRESCRIBING: Choice: Monthly use (doses) of imipenem CLINICAL: cohort data about length of stay and hospital charges for patients with a primary diagnosis of infection | |
| Notes | FINANCIAL SUPPORT: Funding: Washington State University College of Pharmacy and Pullman Memorial Hospital. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | |
| Incomplete outcome data addressed (ITS) ? | Low risk | |
| Free of selected reporting (ITS) ? | Low risk | |
| Free of other bias (ITS) ? | Low risk | Yes for primary outcome but fatally flawed (UBA) for secondary outcomes. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Intervention: restrictive, removal of restriction by expert approval Intervention Function: restriction | |
| Outcomes | PRESCRIBING: Choice: use of ceftriaxone in DDD/1000 OBD MICROBIAL: number of ESBL‐producing strains/1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: LOW case definition Low, planned intervention Low, other infection control Low | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Low risk | 24 months' data pre‐ and postintervention |
| Methods | STUDY DESIGN: cluster RCT, service level Risk of Bias: HIGH | |
| Participants | PROVIDERS: 17 Internal Medicine services randomly assigned to intervention (9 services) or control (8 services) POWER CALCULATION: no information. The methods say that the statistical model adjusted for clustering, but no results are given (see risk of bias). | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of policy for necessary use; educational outreach by review and recommend change, either verbal (face to face or telephone) or by email POWER CALCULATION: no information. Note from Statistician: The study adjusted for some clustering, but possibly only in the repeated measures, not in the hospitals. Just using the results from Table 2, I do not get the P value that they state in the table using a unit of analysis error approach. This suggests to me that they are adjusting for "things". I therefore think on balance that it is probably OK to use the results. | |
| Outcomes | PRESCRIBING: Choice: % patients with target antibiotics discontinued. Exposure: % patients with all antibiotics discontinued CLINICAL: inpatient mortality, transfer to ICU, length of stay, and re‐admission within 30 days of discharge FINANCIAL: estimated annual cost of the intervention | |
| Notes | FINANCIAL SUPPORT: Funding: Brigham and Women’s Hospital and Arthritis Foundation Investigator Award. Competing Interests: no information ADDITIONAL DATA: email from authors with information about the intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "We assigned services to intervention or control status using a blocked randomization design" |
| Allocation concealment (selection bias) | Unclear risk | Not concealed |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Unclear risk | Figure 2 and text give %, no denominator. |
| Selective reporting (reporting bias) | Unclear risk | Figure 2 and text give %, no denominator. |
| Other bias | Unclear risk | The methods say: "To estimate the relative reduction in unnecessary use of target antibiotics in the intervention group, we used a fixedeffects model (PROC GENMOD in SAS statistical software).20 This model used a log‐linear link function, assumed a Poisson distribution, and accounted for overdispersion. Experimental group assignment (intervention or control) was the independent variable of interest, the individual service was considered a class effect, and covariates included level of baseline prescribing and time, modeled as both a linear and categorical effect. The interaction between assignment and time was also assessed. We further considered a linear randomeffects model to account for variation between services (PROC MIXED in SAS statistical software)20; the results of this analysis were similar to those found in the fixed‐effects models with respect to the level of statistical significance, and only the fixedeffects model results are presented." However, no model outputs are given in the results (only point estimates), and the discussion says only: "This significant effect of the intervention remained after adjusting for baseline prescribing, clustering of repeated measures within a given service, and duration of the intervention." |
| Baseline Outcomes similar? | Low risk | Figures 1 and 2 |
| Free of contamination? | High risk | The services were in the same hospital. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change | |
| Outcomes | PRESCRIBING: Choice: quarterly cost of all antimicrobials CLINICAL: Balancing: cohort data for mortality, length of stay, and unplanned re‐admission. The DRG case mix index was monitored to ensure that changes in outcomes were not related to this index. | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | No information is given about changes in drug pricing over the 12 years of data collection, which is likely to have changed the outcome measure. In addition, there were changes in pharmacy data systems after the intervention, but the timing is clearly documented in Figure 1. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data were from the Pharmacy Administration and were independent from the AMT. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data were from the Pharmacy Administration and were independent from the AMT. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data were from the Pharmacy Administration and were independent from the AMT. |
| Free of selected reporting (ITS) ? | Low risk | Data were from the Pharmacy Administration and were independent from the AMT. |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: dissemination of antibiotic policy Intervention Function: education COMPARISON: 10 quarters (30 months) pre‐intervention | |
| Outcomes | PRESCRIBING and FINANCIAL: Choice: average cost of antibiotics per patient. Prices were indexed to 1980. | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 2 years' data pre‐ and postintervention, enough to account for seasonal effects |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: regression analysis testing for structural break associated with intervention. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, drug costs were adjusted to 1980 prices. |
| Free of other bias (ITS) ? | Low risk | Drug costs were adjusted to 1980 prices and adjusted for number of discharges or deaths. |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the neonatal ICU | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing POWER CALCULATION: unclear. The trial was designed to obtain a power of 90% to detect a 30% difference between the 2 groups in the duration of antibiotic therapy, with an estimated standard deviation of 50%. Sample size: no information | |
| Outcomes | PRESCRIBING: Exposure: duration, % treated > 72 h | |
| Notes | FINANCIAL SUPPORT: Funding: commercial B.R.A.H.M.S Diagnostica (Berlin, Germany) provided the testing kits for PCT. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomised using assignment cards in envelopes |
| Allocation concealment (selection bias) | Low risk | Randomised using assignment cards in envelopes |
| Blinding (performance bias and detection bias) | High risk | Not blinded |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes on all patients |
| Selective reporting (reporting bias) | Low risk | Outcomes on all patients |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | PCT results only reported for intervention patients. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in Internal Medicine | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing POWER CALCULATION: yes, total 186 participants. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: % antibiotic use for the exacerbation and in the subsequent 6 months CLINICAL: Balancing: length of stay, death, symptom scores | |
| Notes | FINANCIAL SUPPORT: Funding: part commercial University Hospital Basel. B.R.A.H.M.S provided procalcitonin assays for this investigator‐driven study. Competing Interests: 1 author served as consultant and received payments from B.R.A.H.M.S to attend meetings and for travel expenses, speaking engagements, or research. ADDITIONAL DATA: email response from authors with additional information about intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | No details: "Patients satisfying the entry criteria were randomly assigned to one of two groups at the time of admission to the emergency department " |
| Allocation concealment (selection bias) | Unclear risk | No details |
| Blinding (performance bias and detection bias) | High risk | SIngle‐blind |
| Incomplete outcome data (attrition bias) | Low risk | Outcome data reported on all patients. |
| Selective reporting (reporting bias) | High risk | 11 (10%) patients excluded from intervention and 7 (6%) from control group for "absence of COPD according to GOLD", but this should have occurred pre‐randomisation. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | High risk | No data |
| Free of contamination? | Low risk | PCT only reported for intervention patients. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: RCT Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all staff in adult ICUs | |
| Interventions | FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing POWER CALCULATION; yes, 84 participants total. Details in Appendix 3 | |
| Outcomes | PRESCRIBING: Exposure: duration of antibiotic treatment CLINICAL: Balancing: mortality, hospital length of stay | |
| Notes | FINANCIAL SUPPORT: Funding: Swiss National Foundation, Margarete und Walter Liechtenstein Foundation, Freiwillige Akademische Gesellschaft Basel, Will Rogers Foundation, and participating hospitals. B.R.A.H.M.S AG funded assay material and logistics. Competing Interests: not clear. The published paper says that a statement of interest for the study itself is available but the web address provided online and in print does not work. ADDITIONAL DATA: email response from authors with additional information about intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Block size 20 envelopes |
| Allocation concealment (selection bias) | Low risk | Sealed, opaque envelopes |
| Blinding (performance bias and detection bias) | High risk | Primary outcome measure required collection of data from case noes by investigators. |
| Incomplete outcome data (attrition bias) | Low risk | Text shows that primary outcome was reported for all 101 randomised patients. |
| Selective reporting (reporting bias) | Low risk | Text shows that primary outcome was reported for all 101 randomised patients. |
| Other bias | Low risk | Multivariate analysis to adjust primary outcome for age, microbiology and centre effect |
| Baseline Outcomes similar? | Unclear risk | No data about baseline outcomes |
| Free of contamination? | Low risk | Procalcitonin only measured for intervention patients. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: cluster RCT, professional level Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: A total of 1971 clinicians were assigned to either an intervention group receiving a nearly hard‐stop alert or a control group receiving the standard practice. POWER CALCULATION: "It is generally accepted that randomization of at least 100 subjects will produce balance between the study groups and, of course, the present sample size is much larger than this." | |
| Interventions | FORMAT: Interventions: reminder (circumstantial) and restrictive by compulsory electronic order form that would not allow concomitant orders of warfarin and trimethoprim‐sulfamethoxazole. The only exception allowed by the order form was the indication of Pneumocystis carinii pneumonia prophylaxis. Expert approval was allowed for other patients when discussed with pharmacy. Intervention Functions: enablement, environmental restructuring, restriction | |
| Outcomes | PRESCRIBING: Choice: the proportion of desired responses (i.e. not reordering the alert‐triggering drug within 10 minutes of firing) CLINICAL: Balancing: 2 potential adverse outcomes of the computerised hard‐stop alert were monitored and reported to the Institutional Review Board. The first was a delay in obtaining trimethoprim‐sulfamethoxazole when the practitioner believed that an infection was best treated with trimethoprim‐sulfamethoxazole and when the potential warfarin interaction was judged less important than the need for the antibiotic. The second was unintentional warfarin cessation in a patient previously undergoing long‐term warfarin therapy. The study therefore also assessed the incidence of warfarin cessation on the day when an order of trimethoprim‐sulfamethoxazole was attempted in a patient already receiving warfarin. | |
| Notes | FINANCIAL SUPPORT: Funding: University of Pennsylvania Health System and Agency for Healthcare Research and Quality. Competing Interests: none declared ADDITIONAL DATA: email response from authors with additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Number randomisation |
| Allocation concealment (selection bias) | Low risk | "Each medical practitioner has a unique access code to use the electronic ordering system, and the order system menu can be varied by individual user. In addition, we wanted to keep each practitioner in the same study group for the duration of the study to minimize contamination between the 2 groups. However, there is the possibility" |
| Blinding (performance bias and detection bias) | High risk | Not blinded |
| Incomplete outcome data (attrition bias) | Low risk | Outcome reported on all patients, determined electronically. |
| Selective reporting (reporting bias) | Low risk | Outcome reported on all patients, determined electronically. |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No information |
| Free of contamination? | Low risk | "We attempted to reduce contamination by trying to complete this study as rapidly as possible. It was initially planned to last 7 months but had to be terminated early." |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all cardiac surgeons and other professionals | |
| Interventions | FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines and evidence base | |
| Outcomes | PRESCRIBING: Choice and exposure: time to first antibiotic dose, % of prophylaxis ≤ 24 h | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | From Taiwan Quality Improvement Project database |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from Taiwan Quality Improvement Project database. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome reported on all patients pre‐ and postintervention. |
| Free of selected reporting (ITS) ? | Low risk | Objective outcomes from Taiwan Quality Improvement Project database |
| Free of other bias (ITS) ? | High risk | < 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the Department of Medicine | |
| Interventions | FORMAT: Interventions: dissemination of guidelines; restrictive by expert approval DELIVERER: AMT | |
| Outcomes | PRESRIBING: Choice: monthly cost of target antibiotics CLINICAL: cohort data about mortality | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy computer |
| Free of other bias (ITS) ? | Unclear risk | < 1 year data pre‐ and postintervention. During the 18‐month study period, no adjustment was made to antibiotic costs for changes in prices, so changes in cost may have been due to changes in price as well as use. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by review and recommend change; reminders (verbal (on rounds) and physical (laminated pocket cards and posters)); restrictive by removal of target drugs from clinical areas | |
| Outcomes | PRESCRIBING: Choice: use of target antibiotics in DDD/1000 OBD MICROBIAL: monthly cases of C difficile infection | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: 1 author was paid lecture fees and provided sponsorship to attend conferences by pharmaceutical companies unrelated to this study. ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias LOW: case definition Low (new cases), planned intervention Low, other infection control Low, fully reported in ORION format | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Change in site ‐ moved to another building |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | From electronic records, so unlikely |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | From electronic records, so unlikely |
| Incomplete outcome data addressed (ITS) ? | Low risk | From electronic records, so unlikely |
| Free of selected reporting (ITS) ? | Low risk | From electronic records, so unlikely |
| Free of other bias (ITS) ? | Low risk | 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by academic detailing | |
| Outcomes | PRESCRIBING: Choice: use of target drugs in DDD/1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: no external. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH, Case definition Low, Unplanned intervention High (outbreak), Other infection control High. "In August 2006, the hospital director organized a steering group (SG) with the assignment to implement the necessary measures to contain the outbreak, including reinforcement of hygienic measures, such as hand disinfection, use of disposable gloves and aprons, and isolation of patients colonized or infected with ESBL‐KP.14 In addition to hygienic measures, the SG decided to perform an antibiotic intervention." | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | DDD from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | DDD from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | DDD from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | DDD from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the mixed medical and surgical paediatric ICU | |
| Interventions | FORMAT: Intervention: restrictive, probably by expert approval ("Prohibition of ceftazidime use unless the patient's microbiological results indicated that the drug was necessary for cure.") Intervention Function: restriction DELIVERER: specialist (ID) physician | |
| Outcomes | PRESCRIBING: Choice: ceftazidime use in doses | |
| Notes | FINANCIAL SUPPORT: Funding: grant HD31323‐02 from the National Institutes of Health. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | NOT CLEAR, data for 7 months pre‐ and 12 months postintervention, not enough to adjust for seasonal variation |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with χ2 test. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | High risk | < 1 year data pre‐intervention |
| Methods | STUDY DESIGN: NRT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians (paediatricians) on the ICU | |
| Interventions | FORMAT: no valid prescribing data. Restrictive by removal, monthly rotation of the antibiotic regimen used for empirical prescribing of patients with proven or suspected gram‐negative infections DELIVERER: specialist physician (ICU) | |
| Outcomes | MICROBIAL: incidence of colonisation with multiantibiotic‐resistant aerobic gram‐negative bacilli | |
| Notes | FINANCIAL SUPPORT: Funding: grant HD 31323‐05 from the National Institutes of Health Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: MEDIUM Case definition Low, Planned intervention Low, Other infection control Unclear | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | NRT with monthly rotation of regimens |
| Allocation concealment (selection bias) | High risk | Not possible with this study design |
| Blinding (performance bias and detection bias) | High risk | Not possible with this study design |
| Incomplete outcome data (attrition bias) | Unclear risk | Not stated whether screening samples obtained from all patients |
| Selective reporting (reporting bias) | Unclear risk | Not stated whether screening samples obtained from all patients |
| Other bias | Unclear risk | NOT CLEAR Microbial Outcome Risk of Bias Criteria Case definition: DONE Colonisation by screening. "For the purpose of this study, an 'antibiotic‐resistant Gram‐negative organism' was defined as any Gram‐negative bacillus resistant to gentamicin, piperacillin‐tazobactam, or ceftazidime. Pharyngeal and rectal swab specimens were obtained on all infants every Monday, Wednesday, and Friday". Planned intervention: DONE; Other infection control, Isolation: IC practices: NOT CLEAR Not described, but it is reasonable to assume that they were the same for the intervention and control groups due to the controlled clinical trial design. |
| Baseline Outcomes similar? | Unclear risk | Not stated |
| Free of contamination? | Unclear risk | Not stated, but doctors likely to have been managing patients in more than 1 study phase. |
| Baseline characteristics similar? | Low risk | Results, paragraph 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all paediatric surgeons and anaesthetists | |
| Interventions | FORMAT: no valid prescribing data. Audit and feedback | |
| Outcomes | CLINICAL: surgical‐site infection rate per 100 procedures | |
| Notes | FINANCIAL SUPPORT: Funding: Ohio Business Roundtable, the Cardinal Health Foundation, and the Ohio Children’s Hospital Association. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Administration of antibiotics within 1 hour was 1 component of the bundle; the other 2 were avoiding shaving and encouraging use of clorhexidine for disinfection. In addition, 9 months after the intervention began an additional antibiotic element was added to encourage administration of an additional dose for operations lasting more than 3 hours. |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Patient administration systems and routine collection of surgical‐site infection data by each hospital's infection prevention teams |
| Knowledge of the allocation adequately prevented(ITS)? | High risk | Infection prevention teams were not prevented from knowing about allocation. |
| Incomplete outcome data addressed (ITS) ? | Low risk | |
| Free of selected reporting (ITS) ? | Low risk | Data reported for all months when operations took place |
| Free of other bias (ITS) ? | High risk | Only 8 months' pre‐intervention data |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change (intervention and control); structural, rapid processing and reporting of antimicrobial susceptibility tests (intervention only) DESIRED CHANGE: reduce excessive POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Choice: % changes in therapy in response to recommendation FINANCIAL: savings in drug costs | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Not stated; "the organism from the patient was randomly assigned" |
| Allocation concealment (selection bias) | Unclear risk | Not stated |
| Blinding (performance bias and detection bias) | Unclear risk | Not stated as blind |
| Incomplete outcome data (attrition bias) | Low risk | Table 2 reports primary outcome for all 226 randomised patients. |
| Selective reporting (reporting bias) | Low risk | Table 2 reports primary outcome for all 226 randomised patients. |
| Other bias | Low risk | No other apparent biases found. |
| Baseline Outcomes similar? | Unclear risk | No information about recommendations for changes in therapy before the intervention |
| Free of contamination? | Unclear risk | Likely to be contamination as doctors managing control patients would receive advice on intervention patients. |
| Baseline characteristics similar? | Unclear risk | No information |
| Methods | STUDY DESIGN: ITS Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in one orthopaedic unit | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change. The intervention is reported in 2 phases, the 1st delivered by "Dedicated ID specialist and one internist" and the 2nd delivered by "ID specialist with experience in Infection Control". | |
| Outcomes | PRESCRIBING: Choice: use of IV and oral antibiotics in DDD/1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: none.Competing Interests: none declared. ADDITIONAL DATA: email response from authors with additional data about the intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Insufficient information to assess |
| Analysed appropriately (ITS) ? | Low risk | Time series analysis with ARIMA modelling |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Unclear risk | Outcome data were from routine pharmacy systems. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data were from routine pharmacy systems. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data were from routine pharmacy systems. |
| Free of selected reporting (ITS) ? | High risk | The information in Table 1 does not include total antibiotic use or cost, so cannot be used to support the claims made in the paper. |
| Free of other bias (ITS) ? | High risk | < 1 year pre‐intervention data Insufficient information to assess. In particular, it is not clear what difference the "ID specialist with experience in Infection Control" would make compared with "Dedicated ID specialist and one internist". |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guideline and letter; educational outreach by review and recommend change; reminders (physical, pocket‐size guideline) | |
| Outcomes | PRESCRIBING: Choice and exposure: use of individual targeted drugs in DDD/1000 OBD; use of all antibiotics in DDD/1000 OBD MICROBIAL: Clostridium difficile infections/1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: National Foundation for Infectious Diseases. Competing Interests: 1 author has been on the speakers’ bureau for Wyeth; served on advisory boards for Wyeth and Cubist; and received grants from Wyeth, Genzyme, and Arpida. 1 author has been on the speakers’ bureau for Wyeth Canada; served on advisory boards for Bayer, Wyeth, ViroPharma, and Acambis; and received grants from Genzyme. ADDITIONAL DATA: email from authors but no additional data Microbial Risk of Bias: HIGH Case definition Low. Planned intervention High, response to epidemic of infection caused by high‐virulence strain. Other infection control High, the rate of CDI was already declining in response to infection control intervention when antimicrobial intervention began. | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Antimicrobial intervention followed an infection control intervention, so it is not possible to assess the independent impact on C difficile infection. Moreover, the infection control intervention could have been responsible for some or all of the reduction in total antibiotic use. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | High risk | Microbial Risk of Bias HIGH > 1 year of data pre‐intervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the Departments of Internal Medicine, Gastroenterology, Surgery, Urology, and Pulmonary Diseases | |
| Interventions | FORMAT: Interventions: educational meetings; educational outreach by review and recommend change | |
| Outcomes | PRESCRIBING: Choice: use of ciprofloxacin in prescriptions/1000 OBD | |
| Notes | FINANCIAL SUPPORT:Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data for primary outcome measure were from pharmacy computer. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data for primary outcome measure were from pharmacy computer. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data for primary outcome measure were from pharmacy computer. |
| Free of selected reporting (ITS) ? | Low risk | Data for primary outcome measure were from pharmacy computer. |
| Free of other bias (ITS) ? | High risk | Only 3 months' pre‐ and 6 months' postintervention data, so cannot be adjusted for seasonal trends. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospitals PATIENTS: all patients undergoing elective surgery CLINICAL PROBLEM: surgical prophylaxis across 4 surgical disciplines SETTING: 14 hospitals in the Netherlands | |
| Interventions | FORMAT, Interventions: audit and feedback; educational meetings with dissemination of guidelines DELIVERER: AMT COMPARISON: pre‐intervention periods DESIRED CHANGE: reduce excessive duration of surgical prophylaxis | |
| Outcomes | PRESCRIBING: Exposure: total antibiotic use in DDD/100 procedures CLINICAL: Balancing: cohort data on surgical‐site infections | |
| Notes | FINANCIAL SUPPORT: Funding: Netherlands Organisation for Health Research and Development (ZonMw). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Only 6 months' pre‐ and postintervention data, and the model was not adjusted for seasonal trends. |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. Change in price unlikely to be a problem because only 6 months' data pre‐ and postintervention. |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the ED | |
| Interventions | FORMAT: Interventions: audit and feedback with action planning; educational meetings with dissemination of care algorithm and forms to facilitate care; educational outreach by academic detailing; reminders (circumstantial, root‐cause analysis of individual cases not meeting goal); reminders (physical, posters, email, and verbal, during rounds) | |
| Outcomes | PRESCRIBING: Choice: time (minutes) to first antbiotic dose BALANCING MEASURE OF UNINTENDED CONSEQUENCES: "For balancing measures during the improvement period, we chose to follow the timeliness of first b‐agonist treatment of patients with asthma and the left without being seen (LWBS) rate." | |
| Notes | FINANCIAL SUPPORT:Funding: no external funding. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Statistical process control chart |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from patient administration system |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from patient administration system |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from patient administration system |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from patient administration system |
| Free of other bias (ITS) ? | Low risk | > 12 months' data pre‐ and postintervention |
| Methods | STUDY DESIGN: RCT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational outreach by review and recommend change POWER CALCULATION: no information | |
| Outcomes | PRESCRIBING: Choice: number of patients changed to oral antibiotic therapy | |
| Notes | FINANCIAL SUPPORT: Funding: commercial, Pharmacia and Upjohn. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "A list of random numbers was generated from Sigmastat version 1.0 statistical software" |
| Allocation concealment (selection bias) | Unclear risk | Not stated, but open label, so unlikely to be concealed |
| Blinding (performance bias and detection bias) | High risk | "Open label" |
| Incomplete outcome data (attrition bias) | Low risk | No problems found. |
| Selective reporting (reporting bias) | Low risk | No problems found. |
| Other bias | Low risk | No other apparent biases found. |
| Baseline Outcomes similar? | Unclear risk | Not stated |
| Free of contamination? | Unclear risk | Not stated |
| Baseline characteristics similar? | Low risk | See Table 1 in paper |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in 16 adult ICUs | |
| Interventions | FORMAT: Intervention: educational outreach by review and recommned change; restrictive by expert approval Intervention Functions: education, enablement, persuasion, restriction | |
| Outcomes | PRESCRIBING: Choice: primary outcome is cost of all antimicrobials (Figure 4G). Also reports impact on use of 7 target antibacterials and use of antifungals in DDD/1000 OBD. CLINICAL: Balancing: mortality, ICU re‐admission (segmented regression analysis) | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from pharmacy computer |
| Free of other bias (ITS) ? | High risk | > 12 months' data pre‐ and postintervention. However, no adjustment of primary outcome for changes in drug pricing over the 5 years of the study. |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all anaesthetists in the hospital | |
| Interventions | FORMAT: Interventions: reminders (physical, electronic on screen during all surgical procedures, not just those requiring prophylaxis) | |
| Outcomes | PRESCRIBING: Choice: % patients with first dose within 1 hour of incision | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcome data from electronic patient record, Anaesthesia Information Management System (AIMS) |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcome data from electronic patient record (AIMS) |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcome data from electronic patient record (AIMS) |
| Free of selected reporting (ITS) ? | Low risk | Outcome data from electronic patient record (AIMS) |
| Free of other bias (ITS) ? | High risk | Only 6 months' data pre‐intervention |
| Methods | STUDY DESIGN: controlled ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: operating theatre teams at participating hospitals PARTICIPANGS: low‐income women needing C‐section CLINICAL PROBLEM: infection after C‐section SETTING: 2 hospitals in Colombia | |
| Interventions | FORMAT: Interventions: audit and feedback in the form of run charts for the 2 key process measures (secondary outcomes) with data collected and displayed by the clinical teams; dissemination of flow charts with revised system for administration of prophylactic antibiotics COMPARISON: physician choice about antibiotic and timing DESIRED CHANGE: reduce infection after C‐section TIMING: before clinical decision making; the intervention was continued for 2 years | |
| Outcomes | PRESCRIBING: Choice: percentage of women who received prophylaxis; percentage who received prophylaxis within 1 hour CLINICAL: Intended: SSI rate per 100 C‐sections | |
| Notes | INSTRUCTIONS: action plan provided, specific target but no specified time for target to be achieved FINANCIAL SUPPORT: Funding: International Society for Infectious Diseases, Paul Schliesman Memorial Traveling Fellowship, and the Von L. Meyer Award. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | Data collection method was the same pre‐ and postintervention. |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data collection method was the same pre‐ and postintervention. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Prescribing outcome data were from electronic systems. |
| Incomplete outcome data addressed (ITS) ? | Low risk | For prescribing outcome. Not stated whether SSI was evaluated in all patients |
| Free of selected reporting (ITS) ? | Low risk | For prescribing outcome. Not stated whether SSI was evaluated in all patients |
| Free of other bias (ITS) ? | High risk | < 1 year of data in each of the 3 study phases |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all attending emergency physicans, physician assistants, and emergency nurses | |
| Interventions | FORMAT: Interventions: audit and feedback; reminders (physical, electronic ‐ weekly emails) Intervention Functions: enablement, environmental restructuring, persuasion | |
| Outcomes | PRESCRIBING: Choice: mean time to first antibiotic dose (minutes) | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed; our analysis questions the authors' conclusion that the intervention was effective. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | TFAD from patient administration system |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | TFAD from patient administration system |
| Incomplete outcome data addressed (ITS) ? | Low risk | TFAD from patient administration system, outcome reported on all included patients. |
| Free of selected reporting (ITS) ? | Low risk | "Patients were excluded if the time of antibiotic administration was not documented in the electronic medical record, if the patient was documented as having received antibiotics within 48 hours prior to arrival, or if the patient was referred from another facility or clinic with a known diagnosis of pneumonia." Exclusion rate in pre‐intervention period (37/281, 13%) similar to intervention period (40/342, 12%). |
| Free of other bias (ITS) ? | High risk | Only 11 months' data pre‐ and postintervention |
| Methods | STUDY DESIGN: cluster NRT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the ICU | |
| Interventions | FORMAT: Intervention: reminder, verbal (on rounds) based on a scripted electronic checklist of issues to discuss about antibiotics Intervention Functions: environmental restructuring, persuasion | |
| Outcomes | PRESCRIBING: Choice: duration of empiric antibiotic treatment before narrowing choice, % patient days on which empiric antibiotics were used. Exposure: duration of all antibiotic treatment CLINICAL: Balancing: mortality (total, standardised mortality ratio, and adjusted odds of death), length of hospital stay, length of ICU stay | |
| Notes | FINANCIAL SUPPORT: National Heart, Lung, and Blood Institute (T32HL076139‐07) and Parker B. Francis Fellowship to CHW. Dr Weiss has received funding from the National Institutes of Health. Drs Sung and Rho received a travel award to present a research abstract at American Thoracic Society conference in May 2012 from Northwestern University. Dr Wunderink is a board member for Pfizer and has consulted for Crucell (now Johnson & Johnson), Trius, AstraZeneca, and GlaxoSmithKline. He has received grant support from bioMérieux and payment for lectures from the American Thoracic Society. The remaining authors have not disclosed any potential conflicts of interest. ADDITIONAL DATA: online supplementary data for this article and further details of intervention in Weiss 2011. No response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Coin toss to allocate 1 medical team to intervention and 1 to control |
| Allocation concealment (selection bias) | High risk | No concealment |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | Outcomes reported on all patients. |
| Selective reporting (reporting bias) | High risk | No information about inter‐rater reliability of primary outcome measure, which was not objective: "empirical antibiotics were defined as any antimicrobial agent administered without culture‐documented infection". |
| Other bias | High risk | Unit of analysis error, no adjustment for clustering |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | High risk | Intervention and control teams worked on the same ICU. |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: unintended consequences, cohort study Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the ED | |
| Interventions | FORMAT: Interventions: audit and feedback; financial, institution incentive | |
| Outcomes | UNINTENDED CONSEQUENCES: accuracy of admission diagnosis, antibiotic‐associated adverse drug events | |
| Notes | ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding of the effect of intervention unlikely in this study 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Low, outcome data and intervention status complete in all 548 patients 6. Measurement of outcome: High, outcome measures not objective, and investigators were not blinded to intervention status 7. Selection of the reported result: Low, single analysis of prespecified outcomes FINANCIAL SUPPORT: Funding: commercial: Pfizer, US Pharmaceutical Corporation. Competing Interests: none declared. ADDITIONAL DATA: no response from authors to request for additional data | |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Intervention: educational meetings; restrictive by compulsory order form Intervention Functions: education, restriction | |
| Outcomes | PRESCRIBING: Choice: use of moxifloxacin in DDD | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: Low for case definition, planned intervention, and other infection control | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | High risk | < 12 months' data in the pre‐intervention (5 months) and postintervention (7 months) phases Microbial Risk of Bias LOW: case definition Low, planned intervention Low, other infection control Low |
| Methods | STUDY DESIGN: ITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by review and recommend change; reminders (physical, newsletter and on all microbiology reports saying that ciprofloxacin should be prescribed on strict indications only) | |
| Outcomes | PRESCRIBING: Choice: prescribed daily doses of ciprofloxacin (IV and oral) MICROBIAL: % quionolone‐resistant gram‐negative clinical isolates | |
| Notes | FINANCIAL SUPPORT: Funding: Amphia Hospital, Breda/Oosterhout, Netherlands. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: LOW Case definition infection with quionolone‐resistant gram‐negative bacteria, Planned intervention Low, Other infection control Low, no changes (information in Discussion) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Outcomes from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Outcomes from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Outcomes from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Outcomes from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Low risk | 1 year data pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Intervention: dissemination of newsletter to all prescribers Intervention Function: education DELIVERER: pharmacists | |
| Outcomes | PRESCRIBING: Choice: use of amoxicillin and pivampicillin | |
| Notes | FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | Only 5 months' pre‐intervention data. Even with 26 months' postintervention data, could still be secular changes. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: run chart with no statistical analysis. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: unintended consequences, cohort study Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: restrictive by prior approval | |
| Outcomes | UNINTENDED CONSEQUENCES: delays of > 1 hour or > 2 hours in TFAD | |
| Notes | ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding of the effect of intervention unlikely in this study 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Low, outcome data and intervention status complete in all 3251 patients 6. Measurement of outcome: Low, outcome measures objective and ascertained from patient administration system 7. Selection of the reported result: Low, single analysis of prespecified outcomes FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Methods | STUDY DESIGN: NRT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Intervention: structural, rapid reporting of microbiology results Intervention Function: environmental restructuring | |
| Outcomes | PRESCRIBING: Exposure: % treated with antibiotics and duration if treated CLINICAL: Intended: length of stay | |
| Notes | FINANCIAL SUPPORT:Funding: Research Activity Committee of the Reinier de Graaf Hospital (project 620604). Competing Interests: no information ADDITIONAL DATA: email response and additional files (protocol) from authors | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | By lab number |
| Allocation concealment (selection bias) | High risk | Not concealed |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | States missing information was retrieved from records |
| Selective reporting (reporting bias) | Low risk | Outcomes on all patients |
| Other bias | Low risk | |
| Baseline Outcomes similar? | Unclear risk | No information |
| Free of contamination? | Low risk | Rapid reporting for intervention only |
| Baseline characteristics similar? | Low risk | Table 1 |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: educational meetings; restrictive by expert approval, automatic stop order after 72 hours' treatment, and by removal from formulary | |
| Outcomes | PRESCRIBING and FINANCIAL: total antibiotic costs and average antibiotic cost per day | |
| Notes | FINANCIAL SUPPORT: Funding: administration of Barnes Hospital. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | 25 months' pre‐ and 17 months' postintervention data |
| Analysed appropriately (ITS) ? | Low risk | Done in original paper: ordinary least squares regression analysis adjusting for pre‐existing time trends, re‐analysis with segmented regression performed for the purposes of comparison of effect size with other studies in the review. |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Unclear risk | The abstract states: "Even after some cost increases (not significant) in new and other antibiotics, the program saved $1.33 per antibiotic day", but it is not clear whether the analysis was adjusted for changes in the price of antibiotics during the 3½‐year study period. |
| Methods | STUDY DESIGN: cluster RCT, hospital level Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: a total of 25 hospitals, 13 control and 12 intervention, targeting 2 providers (lead obstetrician and senior midwife manager) in each hospital POWER CALCULATION: As only 25 obstetric units were available for randomisation, and accurate baseline figures for the rates and variability of the 4 marker clinical practices were not available, sample size calculation was not carried out. | |
| Interventions | FORMAT: educational meeting with dissemination of guideline and slides COMPARISON: 13 control hospitals with no intervention DESIRED CHANGE: increase effective | |
| Outcomes | PRESCRIBING: Exposure: % women that received antibiotic prophylaxis | |
| Notes | FINANCIAL SUPPORT: Funding: regional research implementation initiatives of the North Thames and South Thames regional health authorities; the Imperial Cancer Research Fund; and North Staffordshire Hospital Trust. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Obstetric units were allocated to intervention or control group by the toss of a coin. |
| Allocation concealment (selection bias) | Low risk | To eliminate bias during data collection at follow‐up by a second research midwife, and to allow blinded assessment of guideline quality, the allocation was concealed from everyone except JCW, DGA, RJ, and the first research midwife. |
| Blinding (performance bias and detection bias) | Low risk | To eliminate bias during data collection at follow‐up by a second research midwife, and to allow blinded assessment of guideline quality, the allocation was concealed from everyone except JCW, DGA, RJ, and the first research midwife. |
| Incomplete outcome data (attrition bias) | Low risk | "No unit was excluded after randomisation, all intervention units participated in the visits, and data on clinical practices were available for all units, although smaller numbers of case notes were obtainable than planned for steroid usage" |
| Selective reporting (reporting bias) | Low risk | See above |
| Other bias | Low risk | "To reduce the impact of ceiling effects, the proportion of cases in which clinicians failed to carry out each clinical practice was recorded for each obstetric unit at baseline and follow up, and then baseline to follow up ratios were computed to yield the risk ratio for failure to implement each practice in each unit." |
| Baseline Outcomes similar? | Unclear risk | "Accurate baseline figures for the rates and variability of the four marker clinical practices were not available" |
| Free of contamination? | Low risk | Randomisation by units that were located in different hospitals |
| Baseline characteristics similar? | Low risk | "Despite randomisation there were baseline differences in two of the four clinical practices" (use of ventouse and use of polyglycolic acid sutures). "There were no other baseline differences." (includes antibiotic prophylaxis) |
| Methods | STUDY DESIGN: cluster RCT, hospital level Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the ED | |
| Interventions | FORMAT: low‐intensity (control, 8 hospitals); moderate‐intensity (12 hospitals); and high‐intensity (12 hospitals) interventions Low‐intensity intervention: audit and feedback of baseline data; dissemination of guidelines Low‐intensity invervention functions: education, enablement Moderate‐intensity intervention: same as low intensity, but with additional on‐site educational meeting before patient enrolment Moderate‐intensity intervention additional function: education High‐intensity intervention: same as moderate with additional audit and feedback of data about management of individual patients within a week of enrolment plus 2 monthly feedback of group performance data; educational outreach through academic detailing with Plan Do Study Act cycles to discuss actions to be taken in response to group performance data High‐intensity intervention additional functions: education, enablement, persuasion DELIVERER: departmental physicians COMPARISON: usual care DESIRED CHANGE: increase effective: 4 process measures including time to first antibiotic dose POWER CALCULATION: Primary outcome was site of treatment rather than the antibiotic process measures. "We estimated that we would need 96 eligible patients per hospital (3072 in total) to achieve 80% power to detect a 12% difference across the intervention groups for the site‐of‐treatment decision among low‐risk patients." "For the site‐of‐treatment decision, this study achieved greater than 80% power to detect differences of 10% between high‐intensity and moderate‐intensity groups and differences of 12% between high‐intensity and low‐intensity groups according to separate 1‐tailed tests in which the level was 0.025." | |
| Outcomes | PRESCRIBING: Choice: time to first antibiotic dose and choice compliant with guideline CLINICAL: Intended: mortality and medical complications | |
| Notes | INSTRUCTIONS: action plan provided, no explicit target FINANCIAL SUPPORT: Funding: Agency for Healthcare Research and Quality (grant number R01 HS10049). National Institute of Allergy and Infectious Diseases (grant number K24 AI001769). Competing Interests: 1 author received consultancies, honoraria or grants from Genesoft Pharmaceuticals, Zynx Health Corporation, Healthcare Communications Inc., Stephen Lynn Klein, Kellogg Grants, and Pfizer Inc. ADDITIONAL DATA: email response from authors to request for additional data with care pathway, slide sets, order sheets, and protocol (Yealy 2004) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "After stratifying emergency departments by state, teaching status, and annual volume, our statistician randomly assigned these departments to low‐intensity, moderate‐intensity, and high‐intensity guideline implementation strategies in the ratio of 2:3:3, respectively" |
| Allocation concealment (selection bias) | Low risk | |
| Blinding (performance bias and detection bias) | High risk | Not blinded |
| Incomplete outcome data (attrition bias) | Low risk | Incomplete chart review on only 19 (0.6%) of 3219 patients |
| Selective reporting (reporting bias) | Low risk | |
| Other bias | Low risk | "The target sample size included an adjustment of 30% to account for the clustering of patients within providers." |
| Baseline Outcomes similar? | Unclear risk | No data |
| Free of contamination? | Low risk | Cluster RCT |
| Baseline characteristics similar? | Low risk | Demographic characteristics differed between eligible patients who were and were not enrolled. Moreover, authors observed some imbalances in levels of illness severity across the intervention groups; however, their analyses of the site of treatment were performed separately for low‐risk and higher‐risk patients, and their multivariable analyses were not sensitive to the few imbalances that were observed at baseline. |
| Methods | STUDY DESIGN: CITS Risk of Bias: LOW | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT: Interventions: audit and feedback; educational outreach by review and recommend change | |
| Outcomes | PRESCRIBING: Choice: use of target antibiotics in DDD/1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: 1 author received research funding and speaker’s honoraria from Pfizer, AstraZeneca, Janssen‐Cilag, and Merck Sharp & Dohme. ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Low risk | |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Prescribing outcome in DDD from pharmacy computer |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Prescribing outcome in DDD from pharmacy computer |
| Incomplete outcome data addressed (ITS) ? | Low risk | Prescribing outcome in DDD from pharmacy computer |
| Free of selected reporting (ITS) ? | Low risk | Prescribing outcome in DDD from pharmacy computer |
| Free of other bias (ITS) ? | Low risk | Same 11 months of data (Aug‐Jun) in consecutive years pre‐ and postintervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: structural, computerised decision support system | |
| Outcomes | PRESCRIBING: Choice: use of broad‐spectrum antibiotics in DDD/1000 OBD | |
| Notes | FINANCIAL SUPPORT: Funding: Victorian Department of Human Services Quality Branch and Australian Commonwealth Biotechnology Information Fund, which funded the development of Guidance DS. Competing Interests: none declared ADDITIONAL DATA: email from authors with additional data about intervention (Richards 2003; Thursky 2006) Microbial Risk of Bias: MEDIUM (Other infection control High) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Acinetobacter outbreak during intervention period resulting in hand hygiene and staff education interventions. Also see Table 4. |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Low risk | > 1 year data pre‐ and postintervention, so low risk for prescribing outcome Microbial Risk of Bias: MEDIUM Case definition Low, % susceptibility of Pseudomonas isolates, Planned intervention Low for outcome (outbreak was of Acinetobacter), Other infection control High, enhanced during prescribing intervention |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Intervention 1: restrictive by expert approval (same intervention format as Kim 2008) Intervention 1 functions: restriction Intervention 2: addition of reminders (circumstantial, electronic triggered by computerised antibiotic order, the system is described in more detail in Kim 2008) | |
| Outcomes | PRESCRIBING: Choice: use of carbapenems in DDD/1000 OBD MICROBIAL: infections with CRAB (carbapenem‐resistant Acinetobacter baumanii)/1000 OBD CLINICAL: Balancing measures of adverse effects, all‐cause mortality | |
| Notes | FINANCIAL SUPPORT: Funding: commercial, Merck Sharp & Dohme. Competing Interests: supported by Merck Sharp & Dohme Corp. ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH case definition Low, planned intervention Low, other infection control High, ICU cleaning intervention during Phase 3 | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | High risk | Intensive environmental cleaning implemented in 2012 in ICU, which was intended to reduce infections with CRAB (microbial outcome). |
| Analysed appropriately (ITS) ? | Low risk | Segmented regression analysis |
| Shape of effect pre‐specified (ITS) ? | Low risk | Point of intervention was point of analysis. |
| Unlikely to affect data collection (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Data from pharmacy and microbiology computers |
| Incomplete outcome data addressed (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of selected reporting (ITS) ? | Low risk | Data from pharmacy and microbiology computers |
| Free of other bias (ITS) ? | Low risk | > 1 year data in each study phase |
| Methods | STUDY DESIGN: ITS Risk of Bias: MEDIUM | |
| Participants | PROVIDERS: all physicians in the hospital | |
| Interventions | FORMAT: Interventions: restrictive by review and make change (substitution of amikacin for gentamicin) and expert approval from the Infectious Diseases Division DESIRED CHANGE: decrease excessive | |
| Outcomes | PRESCRIBING: Choice: gentamicin usage as a percentage of total aminoglycoside usage | |
| Notes | FINANCIAL SUPPORT: Funding: Veterans Adminstration and Bristol‐Myers Squibb. Competing Interests: none declared, but Bristol‐Myers Squibb was the manufacturer of amikacin ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Intervention independent (ITS) ? | Unclear risk | 3 months' data before, 15 months' during, and 22 months' after the restriction. Not enough data to adjust for seasonal variation. |
| Analysed appropriately (ITS) ? | Low risk | Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after). |
| Shape of effect pre‐specified (ITS) ? | Low risk | Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. |
| Unlikely to affect data collection (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Knowledge of the allocation adequately prevented(ITS)? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Incomplete outcome data addressed (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of selected reporting (ITS) ? | Low risk | Done, data were from routine systems and unlikely to change over study period. |
| Free of other bias (ITS) ? | Low risk | No other apparent biases found. |
| Methods | STUDY DESIGN: CBA Risk of Bias: HIGH | |
| Participants | PROVIDERS: all physicians | |
| Interventions | FORMAT: Intervention: educational outreach through review and recommend change in 2 hospitals Intervention Functions: enablement, persuasion | |
| Outcomes | PRESCRIBING: Choice: use of target antibiotics in DDD/1000 OBD CLINICAL: hospital standardised mortality ratio MICROBIAL: Clostridium difficile infection rates FINANCIAL: total and direct acquisitional cost of targeted antimicrobials | |
| Notes | FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH case definition Not Clear, planned intervention Low, other infection control measures Not Clear | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Study sites selected from baseline antimicrobial use. |
| Allocation concealment (selection bias) | High risk | No concealment |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | Data from pharmacy computer |
| Selective reporting (reporting bias) | Low risk | Data from pharmacy computer |
| Other bias | Low risk | |
| Baseline Outcomes similar? | High risk | Table 2 |
| Free of contamination? | Low risk | Intervention and control sites different hospitals |
| Baseline characteristics similar? | High risk | Several potentially important differences between intervention and control sites |
| Methods | STUDY DESIGN: NRT Risk of Bias: HIGH | |
| Participants | PROVIDERS: all surgeons in the hospital | |
| Interventions | FORMAT: Intervention: dissemination of guideline; reminder (circumstantial, electronic, automated intra‐operative alert) Intervention Functions: education, enablement, environmental restructuring, persuasion | |
| Outcomes | PRESCRIBING: Exposure: % patients who received additional intra‐operative antibiotics CLINICAL: Intended: wound infection rate | |
| Notes | FINANCIAL SUPPORT: Funding: Centers for Disease Control and Prevention cooperative agreement, UR8/CCU115079, University Hospital of Lausanne, and the Leenaards Foundation. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Based on a case number assigned to every surgical procedure performed in the hospital, independent of the study itself |
| Allocation concealment (selection bias) | High risk | No concealment |
| Blinding (performance bias and detection bias) | High risk | No blinding |
| Incomplete outcome data (attrition bias) | Low risk | Outcome on all 273 patients |
| Selective reporting (reporting bias) | Low risk | Outcome on all 273 patients |
| Other bias | Low risk | Outcome on all 273 patients |
| Baseline Outcomes similar? | Low risk | Cohort data before start of trial |
| Free of contamination? | High risk | Control patients were operated on by the same surgeons, and the reminder for intervention patients is likely to have increased awareness of the need for additional doses. |
| Baseline characteristics similar? | Low risk | Table 1 |
AB: antibiotic
AKI: acute kidney injury
AMT: multidisciplinary antibiotic management team
APACHE: Acute Physiology and Chronic Health Evaluation
ARGNB: antibiotic‐resistant gram‐negative bacilli
ARGPB: antibiotic‐resistant gram‐positive bacilli
ARIMA: autoregressive integrated moving average
ASP: Antimicrobial Stewardship Program
BCT: behaviour change technique
CAP: community‐acquired pneumonia
CBA: controlled before‐after study
CBC: complete blood count
CDAD: Clostridium difficile‐associated diarrhoea
CDC: Centers for Disease Control and Prevention
CDI: Clostridium difficile infection
CDSS: clinical decision support system
CI: confidence interval
CITS: comparative interrupted time series
CPIS: clinical pulmonary infection score
CRP: C‐reactive protein
C‐section: Caesarean section
DACT: double anaerobic coverage therapy
DDD: defined daily dose
DRG: diagnosis‐related group
ED: emergency department
EPOC: Effective Practice and Organisation of Care
ER: emergency room
ESBL‐EB: extended‐spectrum beta‐lactamase‐producing Enterobacteriaceae
FTE: full‐time equivalent
GRE: glycopeptide‐resistant enterococci
IC: infectious control
ICD: International Classification of Diseases
ICU: intensive care unit
ID: infectious diseases
IDP: infectious diseases physician
IHC: Intermountain Healthcare
IL‐8: interleukin‐8
ITS: interrupted time series
IQR: interquartile range
IV: intravenous
LOS: length of stay
MRSA: methicillin‐resistant Staphylococcus aureus
MSSA: methicillin‐sensitive Staphylococcus aureus
LRTI: lower respiratory tract infection
MICU: medical intensive care unit
NHAP: nursing home‐acquired pneumonia
NIH: National Institutes of Health
NRT: non‐randomised (controlled) trial
NRSI: non‐randomised studies of interventions
OBD: occupied bed day
OR: odds ratio
PA: parenteral antibiotics
PCR: polymerase chain reaction
PCT: procalcitonin
RCT: randomised controlled trial
RCOG: Royal College of Obstetricians and Gynaecologists
RDD: recommended daily doses
ROB: risk of bias
ROBINS‐I: risk of bias in non‐randomised studies of interventions
RR: risk ratio
SCIP: Surgical Care Improvement Project
SD: standard deviation
SE: standard error
SHEA: Society for Healthcare Epidemiology of America
SICU: surgical intensive care unit
SNF: skilled nursing facilities
SSI: surgical‐site infection
TFAD: time to first antibiotic dose
TREAT: computerised decision support system for antibiotic treatment
UBA: uncontrolled before‐after study
VAP: ventilator‐associated pneumonia
VRE: vancomycin‐resistant enterococci
Characteristics of excluded studies [ordered by study ID]
Ir a:
| Study | Reason for exclusion |
| RCT with no relevant data. Antibiotics were only part of a complex care plan for 6% of participants in the intervention group, and the outcome data do not include information about the effect of the intervention on antibiotic prescribing. | |
| ITS of antibiotic cycling with no interpretable data because there are no pre‐cycling data. Only provides data for 4 phases of cycling. | |
| ITS with no interpretable data. 2 different interventions (education, then restriction via order form) with 3 points before the education intervention and 3 after, but the restriction intervention started after the 4th point. | |
| ITS with no interpretable data because no clearly defined point in time at which the intervention started. | |
| NRT with no interpretable data. Unacceptable allocation bias ("the allocation of a patient to a particular group was determined by the attending physician"). | |
| ITS with no relevant data. The only valid outcome data are about compliance with a guideline about generic documentation of prescription rather than any specific antibiotic prescribing outcome. The data about time to first antibiotic dose are UBA. | |
| ITS with no interpretable data because no clearly defined point in time at which the intervention started. Only 4 data points for antibiotic use, and the intervention included multiple components in addition to antibiotic use, so even if an intervention effect could be calculated reliably it could not be attributed to change in antibiotic prescribing. | |
| RCT with no interpretable data because of incomplete and selective reporting of outcome data. The primary outcome measure was length of stay, but 32% of participants in the intervention group were excluded because they had prolonged length of stay. | |
| RCT with no interpretable data. Only report data for participants whose physicians followed recommendations. | |
| RCT with no interpretable data. Only report data for participants whose physicians followed recommendations. | |
| NRT with no interpretable data. The first part compared the drugs that the Antibiotic Consultant programme recommended, with the drugs actually prescribed by physicians. Data from the second part are presented in an uninterpretable format, with the denominator as cultures, not participants or physicians. | |
| Cluster RCT with no relevant data. Intervention targeted 5 care processes for women having an abortion. Only 1 included antibiotic prescribing within a composite (antibiotic prophylaxis or screening for lower genital tract organisms). Effect of intervention on prescribing cannot be estimated. | |
| Cluster RCT with no interpretable data. The study included 9 hospitals with 32 hospitalisation units (wards). Patients were included if they had drugs dispensed from an electronic system. Baseline: Jan‐June 2003 baseline, no intervention 1. Jan‐June 2004, intervention in half of the wards that were randomised in each hospital 2. Jan‐June 2005, cross‐over, intervention in wards that were randomised to control in Period 2 There is no description of the randomisation process. The primary outcome measure was adherence to recommendations; text on page 658 says they do not present data about mortality or re‐admission, but that appears to be what is in Figure 4. Figure 4: legend (and text) says it is about DDD and cost of drugs, but labelling says it is mortality and re‐admission. We asked authors to clarify and provide valid outcome data but received no reply. | |
| ITS with no interpretable data. Describes 10 years of experience with aminoglycoside cycling, but the intervention periods cannot be mapped onto the outcome data about prescribing or resistance. | |
| ITS with no interpretable data due to inadequate control for the effect of other interventions (infection control measures; see detailed critique by Monnet 2000). | |
| ITS with unacceptable missing data and inappropriate statistical analysis. There are 3 monthly data points pre‐intervention, then a gap in colonisation data for 3 months at the start of the intervention period followed by 3 monthly data points from months 4 to 6 of the intervention phase. | |
| ITS with no interpretable data. There were no isolates of ESBL‐Klebsiella pneumoniae in the last 3 months of the intervention phase, but no data are provided about the number of specimens screened. Appropriate statistical analysis in original paper not done (averages pre‐ and postintervention with χ2 and Fisher's exact test). Re‐analysis not possible because there are only 2 reliable data points in the postintervention phase. | |
| RCT with no interpretable data. Only 29% of randomised doctors were followed up, and recommendations were only made in 6% of the intervention group. | |
| Cluster RCT with no relevant data. Antibiotic prescribing was only 1 component of a care pathway, results for impact on antibiotic prescribing and its contribution to outcome not reported separately. | |
| ITS with no interpretable data. No antibiotic data pre‐intervention, only data about MRSA; this information is uninterpretable without information about pre‐intervention antibiotic prescribing. | |
| RCT with no interpretable data. Statistical analysis of primary outcome measure (antibiotic costs) not done, and re‐analysis not possible from the data presented. | |
| ITS with no interpretable data. Figure 1 reports the number of participants with inappropriate antibiotic use, consultations, significant laboratory test results, and total number of blood cultures obtained. However, the number of participants in each category is not clear in the figure. We asked the authors for raw data but they were unable to provide this information. | |
| Cluster RCT in 10 skilled nursing facilities. The intention was to increase use of IV antibiotics for severe pneumonia. The comparison was between the same intervention delivered by a multidisciplinary team (intervention) versus a physician (control). There was no difference in the intervention effect, but the study provides no reliable evidence of intervention effect (UBA data in all 10 skilled nursing facilities). | |
| NRT in 1 hospital, no interpretable data because no protection against contamination and unreliable primary outcome measure. | |
| RCT with no relevant data. Not primarily an intervention on antibiotic therapy, compared stroke unit versus general medical ward. | |
| NRT with no relevant data. Antibiotic prescribing was only 1 of 3 components of a care pathway, results for impact on antibiotic prescribing and its contribution to outcome not reported separately. | |
| ITS with no interpretable data. The only time series data (Figures 2 and 3) are MRSA and Pseudomonas aeruginosa infections. The paper claims that a prophylaxis intervention in early 2007 was responsible for reduction in P aeruginosa and MRSA infections, whereas the figures clearly show the reduction happened between July and December 2006. The paper does not include valid data about prescribing outcomes, and the authors were unable to provide these data. | |
| CBA in 64 hospitals, no interpretable data because no clear point in time for the intervention. | |
| ITS with no interpretable data. Multiple interventions are described without clear definition of intervention points. | |
| RCT with no interpretable data. These are provider interventions, but allocation was by patient randomisation. The unequal numbers of patients in each group (134 Group A, 141 Group B, and 105 Group C) and the differences in baseline characteristics indicate unacceptable allocation bias. | |
| ITS with no interpretable data about the impact of antibiotic cycling on resistance because there are no pre‐cycling data. | |
| RCT with no relevant data. Antibiotics included in the indicators for treatment of hospitalised cases of pneumonia (compliance with policy, dose and duration) and diarrhoea (no use of antibiotics without bacterial identification), but no separate data are presented for these outcomes. The only data provided are mean scores on a single composite indicator for each condition. |
CBA: controlled before‐after study
DDD: defined daily dose
ESBL: extended‐spectrum beta‐lactamase
ITS: interrupted time series
MRSA: methicillin‐resistant Staphylococcus aureus
NRT: non‐randomised trial
RCT: randomised controlled trial
UBA: uncontrolled before‐after study
Data and analyses
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Dichotomous outcomes, increase in desired practice Show forest plot | 29 | 23394 | Risk Difference (M‐H, Fixed, 95% CI) | 0.15 [0.14, 0.16] |
| Analysis 1.1  Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Dichotomous outcomes, increase in desired practice. | ||||
| 2 Dichotomous outcomes, all RCTs with results of cluster RCTs adjusted by inflation factor Show forest plot | 29 | 5802 | Risk Difference (M‐H, Fixed, 95% CI) | 0.17 [0.15, 0.19] |
| Analysis 1.2  Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Dichotomous outcomes, all RCTs with results of cluster RCTs adjusted by inflation factor. | ||||
| 3 Dichotomous outcomes, low or medium 'Risk of bias' studies only Show forest plot | 15 | 13086 | Risk Difference (M‐H, Fixed, 95% CI) | 0.11 [0.10, 0.12] |
| Analysis 1.3  Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Dichotomous outcomes, low or medium 'Risk of bias' studies only. | ||||
| 4 Continuous outcomes, duration of all antibiotic treatment (days) Show forest plot | 14 | 3318 | Mean Difference (IV, Fixed, 95% CI) | ‐1.95 [‐2.22, ‐1.67] |
| Analysis 1.4  Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Continuous outcomes, duration of all antibiotic treatment (days). | ||||
| 5 Continuous outcomes, duration of all antibiotic treatment with results of cluster RCTs adjusted by inflation factor Show forest plot | 14 | 3318 | Mean Difference (IV, Fixed, 95% CI) | ‐1.95 [‐2.23, ‐1.67] |
| Analysis 1.5  Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Continuous outcomes, duration of all antibiotic treatment with results of cluster RCTs adjusted by inflation factor. | ||||
| 6 Continuous outcomes, low or medium 'Risk of bias' studies only Show forest plot | 3 | 755 | Mean Difference (IV, Fixed, 95% CI) | ‐3.06 [‐3.76, ‐2.37] |
| Analysis 1.6  Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Continuous outcomes, low or medium 'Risk of bias' studies only. | ||||
| 7 Continuous outcome, consumption of targeted antibiotic only, standardised mean reduction (original outcome cost, days or DDD) Show forest plot | 4 | 1053 | Std. Mean Difference (IV, Fixed, 95% CI) | ‐0.25 [‐0.37, ‐0.13] |
| Analysis 1.7  Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 7 Continuous outcome, consumption of targeted antibiotic only, standardised mean reduction (original outcome cost, days or DDD). | ||||
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Mortality, all RCTs Show forest plot | 28 | 15827 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.00 [‐0.01, 0.00] |
| Analysis 2.1  Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Mortality, all RCTs. | ||||
| 2 Mortality, all RCTs with results of cluster RCTs adjusted by inflation factor Show forest plot | 28 | 8332 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.01 [‐0.02, 0.01] |
| Analysis 2.2  Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Mortality, all RCTs with results of cluster RCTs adjusted by inflation factor. | ||||
| 3 Mortality, low or medium 'Risk of bias' RCTs Show forest plot | 8 | 6249 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.00 [‐0.02, 0.01] |
| Analysis 2.3  Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Mortality, low or medium 'Risk of bias' RCTs. | ||||
| 4 Length of stay, all RCTs Show forest plot | 15 | 3834 | Mean Difference (IV, Fixed, 95% CI) | ‐1.12 [‐1.54, ‐0.70] |
| Analysis 2.4  Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Length of stay, all RCTs. | ||||
| 5 Length of stay, all RCTs with results of cluster RCTs adjusted by inflation factor Show forest plot | 15 | 3834 | Mean Difference (IV, Fixed, 95% CI) | ‐1.22 [‐1.68, ‐0.76] |
| Analysis 2.5  Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Length of stay, all RCTs with results of cluster RCTs adjusted by inflation factor. | ||||
| 6 Length of stay, low or medium 'Risk of bias' RCTs only Show forest plot | 6 | 1731 | Mean Difference (IV, Fixed, 95% CI) | ‐0.85 [‐1.38, ‐0.32] |
| Analysis 2.6  Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Length of stay, low or medium 'Risk of bias' RCTs only. | ||||
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Mortality for trial patients Show forest plot | 11 | 7658 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.00 [‐0.02, 0.01] |
| Analysis 3.1  Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 1 Mortality for trial patients. | ||||
| 2 Length of stay for trial patients Show forest plot | 7 | 2276 | Mean Difference (IV, Fixed, 95% CI) | ‐1.50 [‐2.16, ‐0.83] |
| Analysis 3.2  Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 2 Length of stay for trial patients. | ||||
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Mortality for trial patients Show forest plot | 18 | 9173 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.00 [‐0.01, 0.01] |
| Analysis 4.1  Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 1 Mortality for trial patients. | ||||
| 2 Length of stay for trial patients Show forest plot | 8 | 1558 | Mean Difference (IV, Fixed, 95% CI) | ‐0.87 [‐1.42, ‐0.33] |
| Analysis 4.2  Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 2 Length of stay for trial patients. | ||||
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Enablement with feedback Show forest plot | 4 | 3747 | Risk Difference (M‐H, Fixed, 95% CI) | 0.19 [0.16, 0.22] |
| Analysis 5.1  Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 1 Enablement with feedback. | ||||
| 2 Enablement without feedback Show forest plot | 7 | 1827 | Risk Difference (M‐H, Fixed, 95% CI) | 0.13 [0.09, 0.17] |
| Analysis 5.2  Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 2 Enablement without feedback. | ||||
Figure 1 Study flow diagram.
Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
Blank sections in this graph are due to use of different ROB criteria for CBA, NRT and RCT versus ITS studies
Forest plot of comparison: 1 Prescribing: RCTs of all interventions to reduce unnecessary prescribing, outcome: 1.1 Dichotomous outcomes, increase in desired practice.
Forest plot of comparison: 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 1.4 Continuous outcomes, duration of all antibiotic treatment (days).
Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.1 Mortality, all RCTs.
Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.4 Length of stay, all RCTs.
Meta‐regression by effect modifier for 29 RCTs. A positive value for Beta indicates enhanced intervention effect. One RCT had both enabling and restrictive components in the intervention (Strom 2010).
Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.1 Enablement plus feedback.
Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.2 Enablement without feedback.
Meta‐regression by effect modifiers of intervention for 91 ITS studies. Outcome is effect on prescribing six months' postintervention. There are 16 studies with both enabling and restricting intervention components (Figure 11).
Meta‐regression of prescribing outcome by effect modifiers for 29 ITS studies of interventions that included restriction.
Meta‐regression by effect modifier for 43 ITS studies of interventions that included enablement but not restriction. Outcome is effect on prescribing six months' postintervention. Note that four studies with feedback were not included in this analysis because they also included restriction.
Meta‐regression by effect modifiers for 34 microbial outcomes 12 months' postintervention from 26 ITS studies. The bars show the results for unadjusted versus adjusted analyses, the comparison for unplanned interventions is with planned interventions in both the unadjusted and adjusted analysis.
CDI: Clostridium difficile infection
GPC: infection with antibiotic‐resistant gram‐positive cocci
GNB: infection with antibiotic‐resistant gram‐negative bacteria
Other infection control: 'Yes' means there were changes to infection control processes during the study period.
Meta‐regression by effect modifiers for 20 microbial outcomes 12 months' postintervention from 14 ITS studies of planned interventions that provided details about other infection control changes or interventions.
CDI: Clostridium difficile infection
GPC: infection with antibiotic‐resistant gram‐positive cocci
GNB: infection with antibiotic‐resistant gram‐negative bacteria
Other infection control: 'Yes' means there were changes to infection control processes during the study period.
Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Dichotomous outcomes, increase in desired practice.
Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Dichotomous outcomes, all RCTs with results of cluster RCTs adjusted by inflation factor.
Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Dichotomous outcomes, low or medium 'Risk of bias' studies only.
Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Continuous outcomes, duration of all antibiotic treatment (days).
Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Continuous outcomes, duration of all antibiotic treatment with results of cluster RCTs adjusted by inflation factor.
Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Continuous outcomes, low or medium 'Risk of bias' studies only.
Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 7 Continuous outcome, consumption of targeted antibiotic only, standardised mean reduction (original outcome cost, days or DDD).
Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Mortality, all RCTs.
Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Mortality, all RCTs with results of cluster RCTs adjusted by inflation factor.
Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Mortality, low or medium 'Risk of bias' RCTs.
Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Length of stay, all RCTs.
Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Length of stay, all RCTs with results of cluster RCTs adjusted by inflation factor.
Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Length of stay, low or medium 'Risk of bias' RCTs only.
Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 1 Mortality for trial patients.
Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 2 Length of stay for trial patients.
Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 1 Mortality for trial patients.
Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 2 Length of stay for trial patients.
Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 1 Enablement with feedback.
Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 2 Enablement without feedback.
| Patient or population: adults or children undergoing inpatient antibiotic prophylaxis or treatment Settings: mainly high‐income countries (North America or Western Europe) Intervention: any intervention targeting healthcare professionals that aimed to improve antibiotic prescribing to hospital inpatients Comparison: usual care (varied across studies) | |||||
| Effectiveness: prescribing outcomes from RCTs | |||||
| Outcomes | Absolute effect* | No of participants (No of studies) | Certainty of the evidence (GRADE) | Comments | |
| Without intervention | With intervention | ||||
| Proportion of participants who were treated according to antibiotic prescribing guidelines Follow‐up to end of study | 43 per 100 | 58 per 100 | 23,394 participants (29 RCTs) | ⊕⊕⊕⊕ | We have graded the certainty of evidence as high because heterogeneity was explained by prespecified effect modifiers (see below). The intervention effect varied between the studies, but the direction of effect was consistent. Restricting the analysis to studies at low risk of bias gave a similar result (RD 11%, 95% CI 10% to 12%). |
| Difference: 15 more participants per 100 (95% CI 15 to 23) received appropriate treatment following intervention. | |||||
| Duration of all antibiotic treatment | 11.0 days | 9.1 days | 3318 participants (14 RCTs) | ⊕⊕⊕⊕ | |
| Difference: 1.95 fewer days per participant (95% CI 2.22 to 1.67) | |||||
| Mortality Follow‐up to end of study | 11 per 100 | 11 per 100 | 15,827 participants 28 (RCTs) | ⊕⊕⊕⊝1 | Mortality and length of stay were measured to determine the impact of reduced antibiotic use on clinical outcomes. The results were similar for studies that targeted antibiotic choice or exposure. Only 1 of the interventions in the RCTs with mortality or length‐of‐stay outcomes had a restrictive component (Singh 2000). This evidence is therefore at high risk of indirectness because 7 studies in the next section of the table (see below) raise concerns about the safety of restrictive interventions. Moreover, the ITS studies showed that restrictive components were included in 42 (34%) of 123 hospital interventions. |
| Difference: 0 more deaths per 100 participants (95% CI 1 to 0 fewer) | |||||
| Mean length of hospital stay per participant | 12.9 days | 11.8 days | 3834 participants 15 (RCTs) | ⊕⊕⊕⊝1 | |
| Difference: 1.1 fewer days per participant (95% CI 1.5 to 0.7 fewer) | |||||
| Delay in treatment | Restrictive interventions increased the risk of delay in all 3 studies. The risk to patients resulted in termination of the RCT by the Trial Monitoring Committee. | 1 RCT, 2 cohort | ⊕⊕⊝⊝2 | The evidence from these 7 studies of unintended consequences raises concerns about the directness of the evidence of safety from the 29 RCTs in the previous section of the table (see above). | |
| Negative professional culture | Loss of trust in infection specialists because of failure to record approvals for restricted drugs or provide warning about stopping treatment Misleading or inaccurate information from prescribers in order to meet criteria for restricted drugs. In 1 hospital, misdiagnosis of hospital‐acquired infection was large enough to trigger an outbreak investigation. | 1 case control, 2 cohort, 1 qualitative | ⊕⊕⊖⊖3 | ||
| Effect modifiers (heterogeneity) for immediate effect of intervention on prescribing outcomes: | |||||
| Effect modifier | Adjusted effect in meta‐regression | Number of studies | Certainty of the evidence (GRADE) | Comments | |
| Enablement | 15.12 (8.45 to 21.8) | 29 RCTs | ⊕⊕⊕⊕ | The effect of enablement and restriction is similar in the RCTs and ITS studies. Of the 29 RCTs, only 8 (31%) of interventions were hospital‐wide, the majority being in single units. In contrast, 64 (70%) of the interventions in ITS studies were hospital‐wide. | |
| 12.86 (4.11 to 21.6) | 91 ITS | ||||
| Restriction | 34.91 (13.52 to 56.29) | 29 RCTs | ⊕⊕⊕⊕ | ||
| 24.69 (13.74 to 35.64) | 91 ITS | ||||
| Addition of feedback to enablement | 10.88 (7.16 to 19.32) | 23 RCTs | ⊕⊕⊕⊝2 | Feedback was included in 4 (17%) of 23 RCTs and 20 (47%) of 43 ITS studies with interventions that included enablement. There were not enough interventions with goal setting and action planning to analyse as effect modifiers. | |
| 15.63 (0.56 to 30.70) | 43 ITS | ||||
| Addition of enablement to restriction | 38.36 (18.94 to 57.78) | 29 ITS | ⊕⊕⊖⊖3 | Enablement was included in 13 (45%) of 29 ITS studies with restrictive interventions. | |
| *The risk WITHOUT the intervention is based on the median control group risk across studies. The corresponding risk WITH the intervention (and the 95% confidence interval for the difference) is based on the overall relative effect (and its 95% confidence interval). | |||||
| GRADE Working Group grades of evidence | |||||
| Details of five GRADE criteria for all outcomes from RCTs are in Appendix 2. 1We downgraded the evidence to moderate because of indirectness. | |||||
| Intervention Function | Definition | Intervention components |
| Education | Increasing knowledge or understanding | Educational meetings; Dissemination of educational materials; Educational outreach |
| Persuasion | Using communication to induce positive or negative feelings or to stimulate action | Educational outreach by academic detailing or review and recommend change |
| Restriction | Using rules to reduce the opportunity to engage in the target behaviour (or increase the target behaviour by reducing the opportunity to engage in competing behaviours) | Restrictive |
| Environmental restructuring | Changing the physical context | Reminders (physical) such as posters, pocket‐size or credit card‐size summaries or on laboratory test reports; Structural (e.g. new laboratory tests or rapid reporting of results) |
| Enablement | Increasing means/reducing barriers to increase capability or opportunity | Audit and feedback; Decision support through computerised systems or through circumstantial reminders that were triggered by actions or events related to the targeted behaviour; Educational outreach by review and recommend change |
| Study | Prescribing target | Restriction | Design of analysis | Effect estimate | 95% CI |
| Choice of drug | No | Cohort | Incidence rate ratio 1.1 | 0.9 to 1.5 | |
| Choice of drug | No | Cohort | Increase by 1.4% | ‐1.2% to 4.1% | |
| Choice of drug | Yes | ITS, segmented regression | Change in slope ‐0.0172 | No data | |
| Choice of drug | Yes | Cohort | +0.43 per 1000 OBD | No data | |
| *Mortality was measured in all patients in the hospital rather than just those patients who were the targets of the interventions. CI: confidence interval | |||||
| Study | Prescribing target | Restrictive | Design of analysis | Effect estimate | 95% CI |
| Exposure, % treated | No | Cohort | ‐0.5 days | No data | |
| Choice of drug | No | Cohort | ‐0.1 days | ‐0.49 to +0.29 | |
| *Length of stay was measured in all patients in the hospital rather than just those patients who were the targets of the interventions. CI: confidence interval | |||||
| Study | Prescribing target | Design of analysis | Effect measure | Effect estimate | 95% CI |
| Antibiotic choice | ITS, segmented regression | Risk of postoperative acute kidney injury | Increase 98% | 93.8% to 94.2% | |
| Exposure, duration | Cohort | Surgical‐site infection | Decrease 0.8% | ‐2.2% to 0.6% | |
| Time to first antibiotic dose | Cohort | Left without being seen rate | Decrease 0.4% | No data | |
| CI: confidence interval | |||||
| Study | Design | Patients | Intended target | Unintended consequence | Effect estimate | 95% CI |
| Interventions with a restrictive component | ||||||
| Qualitative | 36 physicians | Reduce unnecessary use of restricted antibiotics | Inaccurate feedback | Not quantified; qualitative study | ||
| Case control | Not clear | Increase in physician‐based diagnosis of nosocomial infection | No denominator data | |||
| Cohort | 120 | Failure to warn prescribers about discontinuation | — | — | ||
| Cohort | 222 | Reduce unnecessary laboratory tests | Delay in TFAD (HR > 1 shows delay less likely in intervention period) | Multivariate HR 1.56 | 1.17 to 2.07 | |
| Cross‐sectional | 15,440 | Reduce unnecessary use of restricted antibiotics | Orders for restricted antibiotics (% all orders) from 10 to 11 pm vs all other hours | — | — | |
| Cohort | 360 | % appropriate orders 10 to 11 pm vs 9 to 10 pm | ‐23.7% | ‐31.8% to ‐15.5% | ||
| Cohort | 200 | Risk of inaccurate information in orders judged inappropriate vs appropriate | OR 2.2 | 1.0 to 4.4 | ||
| Cohort | 3251 | Risk of 1‐hour delay in TFAD | OR 1.5 | 1.2 to 1.8 | ||
| Risk of 2‐hour delay in TFAD | OR 1.8 | 1.4 to 2.2 | ||||
| Interventions with no restrictive component | ||||||
| Cohort | 13,042 | Reduce time to first antibiotic dose for patients with community‐acquired pneumonia | % CAP diagnoses | 1% increase | No denominator data | |
| Cohort | 518 | % correct CAP diagnoses | ‐7.9% decrease | ‐15.4% to ‐0.4% | ||
| Cohort | 548 | % correct CAP diagnoses | ‐16.0% decrease | ‐7.6% to ‐24.4% | ||
| CAP: community‐acquired pneumonia | ||||||
| Intervention function and components | RCT | ITS |
| Enablement | 24 studies | 59 studies |
| Number of enabling or restrictive intervention components | 27 | 76 |
| Studies with > 1 Enabling intervention component | 2 8%* | 19 32%* |
| Audit and feedback | 4 17% | 24 41% |
| Computerised decision support | 1 4% | 3 5% |
| Circumstantial reminders | 16 67% | 18 31% |
| Review and recommend change | 6 25% | 31 53% |
| Restriction | 2 studies | 29 studies |
| Number of Restrictive intervention components | 3 | 41 |
| Studies with > 1 Restrictive intervention component | 1 50% | 10 34% |
| Expert approval | 1 50% | 18 62% |
| Compulsory order form | 1 50% | 7 24% |
| Removal | 0 | 10 34% |
| Review and make change | 1 50% | 6 21% |
| No Enablement or Restriction | 4 studies | 18 studies |
| Number of intervention components | 6 | 25 |
| Studies with > 1 intervention component | 2 50% | 6 33% |
| Educational materials or meetings | 3 75% | 16 89% |
| Educational outreach (academic detailing) | 1 25% | 6 33% |
| Physical reminders | 1 25% | 2 11% |
| Structural intervention | 1 25% | 1 6% |
| *The denominator for all percentages is the number of studies for each intervention function. One RCT, Strom 2010, and 16 ITS studies (Figure 11) included both enabling and restrictive intervention components. ITS: interrupted time series | ||
| Study | Intervention function | Intervention effect (95% CI) | Time intervention was in place | Effect of removal (95% CI) |
| Restriction | ‐87.5% ‐115.4 to ‐59.7 | 6 months | 398.9% 238.2 to 559.5 | |
| Restriction | ‐23.1% ‐53.7 to +7.4 | 9 months | 6.0% ‐23.4 to 35.4 | |
| Enablement | ‐28.6% ‐46.5 to ‐10.6 | 7 years | 31.0% 6.8 to 55.3 | |
| Restriction | No data | “long‐standing” | 301.2% 230.9 to 371.5 | |
| Restriction | 2 years | 255.8% 194.7 to 316.9 | ||
| CI: confidence interval | ||||
| Study | Design | Microbial outcome | Reason not in meta‐analysis |
| RCT | Colonisation with MRSA (nasal swab) and GNRB (rectal swabs) | Not comparable with any other RCT | |
| RCT | Number of cases of Clostridium difficile | Not in prescribing meta‐analysis | |
| RCT | Secondary infection and/or colonisation with multidrug‐resistant bacteria in the 6 months following randomisation | Not in prescribing meta‐analysis. It is impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics. | |
| RCT | CDI and infection with antibiotic resistant organisms cases/1000 OBD | Not in prescribing meta‐analysis | |
| RCT | Number of participants with "antimicrobial resistance and/or superinfections" from randomisation until discharge from hospital | It is impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics. | |
| CDI: Clostridium difficile infection | |||
| Prescribing target | Microbial outcome | N | Study ID |
| Cephalosporins | GNRB | 8 | Grohs 2014; Kim 2008; Knudsen 2014; Lee 2007; McNulty 1997; Meyer 2009; Petrikkos 2007; Tangdén 2011 |
| MRSA | 1 | ||
| Carbapenems | GNRB | 1 | |
| Fluoroquinolones | GNRB | 3 | |
| MRSA | 1 | ||
| High‐risk antibiotics | CDI | 6 | Aldeyab 2012; Chan 2011; Dancer 2013; Fowler 2007; Talpaert 2011; Valiquette 2007 |
| GNRB | 4 | ||
| MRSA | 6 | Aldeyab 2014; Ananda‐Rajah 2010; Chan 2011; Dancer 2013; Fowler 2007; Liebowitz 2008 | |
| Total antibiotic use | CDI | 2 | |
| MRSA | 1 | ||
| Vancomycin | VRE | 1 | |
| Total microbial | 34* | ||
| *Some studies had more than one microbial outcome, so the total is 34 microbial outcomes from 26 studies. CDI: Clostridium difficile infection | |||
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Dichotomous outcomes, increase in desired practice Show forest plot | 29 | 23394 | Risk Difference (M‐H, Fixed, 95% CI) | 0.15 [0.14, 0.16] |
| 2 Dichotomous outcomes, all RCTs with results of cluster RCTs adjusted by inflation factor Show forest plot | 29 | 5802 | Risk Difference (M‐H, Fixed, 95% CI) | 0.17 [0.15, 0.19] |
| 3 Dichotomous outcomes, low or medium 'Risk of bias' studies only Show forest plot | 15 | 13086 | Risk Difference (M‐H, Fixed, 95% CI) | 0.11 [0.10, 0.12] |
| 4 Continuous outcomes, duration of all antibiotic treatment (days) Show forest plot | 14 | 3318 | Mean Difference (IV, Fixed, 95% CI) | ‐1.95 [‐2.22, ‐1.67] |
| 5 Continuous outcomes, duration of all antibiotic treatment with results of cluster RCTs adjusted by inflation factor Show forest plot | 14 | 3318 | Mean Difference (IV, Fixed, 95% CI) | ‐1.95 [‐2.23, ‐1.67] |
| 6 Continuous outcomes, low or medium 'Risk of bias' studies only Show forest plot | 3 | 755 | Mean Difference (IV, Fixed, 95% CI) | ‐3.06 [‐3.76, ‐2.37] |
| 7 Continuous outcome, consumption of targeted antibiotic only, standardised mean reduction (original outcome cost, days or DDD) Show forest plot | 4 | 1053 | Std. Mean Difference (IV, Fixed, 95% CI) | ‐0.25 [‐0.37, ‐0.13] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Mortality, all RCTs Show forest plot | 28 | 15827 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.00 [‐0.01, 0.00] |
| 2 Mortality, all RCTs with results of cluster RCTs adjusted by inflation factor Show forest plot | 28 | 8332 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.01 [‐0.02, 0.01] |
| 3 Mortality, low or medium 'Risk of bias' RCTs Show forest plot | 8 | 6249 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.00 [‐0.02, 0.01] |
| 4 Length of stay, all RCTs Show forest plot | 15 | 3834 | Mean Difference (IV, Fixed, 95% CI) | ‐1.12 [‐1.54, ‐0.70] |
| 5 Length of stay, all RCTs with results of cluster RCTs adjusted by inflation factor Show forest plot | 15 | 3834 | Mean Difference (IV, Fixed, 95% CI) | ‐1.22 [‐1.68, ‐0.76] |
| 6 Length of stay, low or medium 'Risk of bias' RCTs only Show forest plot | 6 | 1731 | Mean Difference (IV, Fixed, 95% CI) | ‐0.85 [‐1.38, ‐0.32] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Mortality for trial patients Show forest plot | 11 | 7658 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.00 [‐0.02, 0.01] |
| 2 Length of stay for trial patients Show forest plot | 7 | 2276 | Mean Difference (IV, Fixed, 95% CI) | ‐1.50 [‐2.16, ‐0.83] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Mortality for trial patients Show forest plot | 18 | 9173 | Risk Difference (M‐H, Fixed, 95% CI) | ‐0.00 [‐0.01, 0.01] |
| 2 Length of stay for trial patients Show forest plot | 8 | 1558 | Mean Difference (IV, Fixed, 95% CI) | ‐0.87 [‐1.42, ‐0.33] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Enablement with feedback Show forest plot | 4 | 3747 | Risk Difference (M‐H, Fixed, 95% CI) | 0.19 [0.16, 0.22] |
| 2 Enablement without feedback Show forest plot | 7 | 1827 | Risk Difference (M‐H, Fixed, 95% CI) | 0.13 [0.09, 0.17] |
