L'inspection externe de la conformité aux normes pour l'amélioration des résultats en matière de santé
Appendices
Appendix 1. Search strategy
The Cochrane Library (DARE, CENTRAL)
1. [mh "health personnel"/ST]
2. (clinician* or consultant* or dentist* or doctor* or family practition* or general practition* or gynecologist* or gynaecologist* or hematologist* or haematologist* or internist* or nurse* or obstetrician* or occupational therapist* or paediatrician* or pediatrician* or pharmacist* or physician* or physiotherapist* or psychiatrist* or psychologist* or radiologist* or surgeon* or surgery or therapist* or counsel*or* or neurologist* or optometrist* or paramedic* or social worker* or health professional* or health personnel or healthcare personnel or health care personnel)
3. [mh "health facilities"/ST]
4. (hospital or hospitals or clinic or clinics or (primary near/2 care) or (health near/2 care))
5. {or #1‐#4}
6. [mh "peer review, health care"]
7. [mh benchmarking]
8. [mh accreditation]
9. [mh "management audit"]
10. [mh "clinical audit"]
11. (organi* next raid*)
12. (external* near/5 (accreditation or accredited or peer review or inspection or inspected or regulation or regulated or certified or certification or benchmark* or measured or measurement or evaluation or evaluated or audit or audits or auditing or assessment or assessed or monitored or visitation or surveillance or (control next program*)))
13. {or #6‐#12}
14. (standards or standard or performance or criterion or criteria or indicator* or (clinical next competence) or compliance or (clinical next improvement) or (quality next (improvement or management)) or (organi* next development) or (health next care next regulation))
15. #5 and #13 and #14
MEDLINE
1. exp health personnel/st
2. (clinician* or consultant* or dentist* or doctor* or family practition* or general practition* or gyn?ecologist* or h?ematologist* or internist* or nurse* or obstetrician* or occupational therapist* or p?ediatrician* or pharmacist* or physician* or physiotherapist* or psychiatrist* or psychologist* or radiologist* or surgeon* or surgery or therapist* or counsel?or* or neurologist* or optometrist* or paramedic* or social worker* or health professional* or health personnel or healthcare personnel or health care personnel).tw.
3. exp health facilities/st
4. (hospital or hospitals or clinic or clinics or (primary adj2 care) or (health adj2 care)).tw.
5. or/1‐4
6. peer review, health care/
7. benchmarking/
8. exp accreditation/
9. exp management audit/
10. exp clinical audit/
11. (organi?ation* adj raid*).tw.
12. (external* adj5 (accreditation or accredited or peer review or inspection or inspected or regulation or regulated or certified or certification or benchmark* or measured or measurement or evaluation or evaluated or audit or audits or auditing or assessment or assessed or monitored or visitation or surveillance or (control adj program*))).tw.
13. or/6‐12
14. (standards or standard or performance or criterion or criteria or indicator* or (clinical adj competence) or compliance or (clinical adj improvement) or (quality adj (improvement or management)) or (organi?ation* adj development) or (health adj care adj regulation)).tw.
15. 5 and 13 and 14
16. intervention?.ti. or (intervention? adj6 (clinician? or collaborat* or community or complex or design* or doctor? or educational or family doctor? or family physician? or family practitioner? or financial or GP or general practice? or hospital? or impact? or improv* or individuali?e? or individuali?ing or interdisciplin* or multicomponent or multi‐component or multidisciplin* or multi‐disciplin* or multifacet* or multi‐facet* or multimodal* or multi‐modal* or personali?e? or personali?ing or pharmacies or pharmacist? or pharmacy or physician? or practitioner? or prescrib* or prescription? or primary care or professional* or provider? or regulatory or tailor* or target* or team* or usual care)).ab.
17. (pre‐intervention? or preintervention? or "pre intervention?" or post‐intervention? or postintervention? or "post intervention?").ti,ab.
18. (hospital* or patient?).hw. and (study or studies or care or health* or practitioner? or provider? or physician? or nurse? or nursing or doctor?).ti,hw.
19. demonstration project?.ti,ab.
20. (pre‐post or "pre test*" or pretest* or posttest* or "post test*" or (pre adj5 post)).ti,ab.
21. (pre‐workshop or post‐workshop or (before adj3 workshop) or (after adj3 workshop)).ti,ab.
22. trial.ti. or ((study adj3 aim?) or "our study").ab.
23. (before adj10 (after or during)).ti,ab.
24. ("quasi‐experiment*" or quasiexperiment* or "quasi random*" or quasirandom* or "quasi control*" or quasicontrol* or ((quasi* or experimental) adj3 (method* or study or trial or design*))).ti,ab.
25. non‐randomized controlled trials as topic/
26. pilot projects/
27. pilot.ti. or (pilot adj (project? or study or trial)).ab.
28. (time points adj3 (over or multiple or three or four or five or six or seven or eight or nine or ten or eleven or twelve or month* or hour? or day? or "more than")).ab.
29. ("time series" adj2 interrupt*).ti,ab.
30. interrupted time series analysis/
31. controlled before‐after studies/
32. historically controlled study/
33. (multicentre or multicenter or multi‐centre or multi‐center).ti.
34. (control adj3 (area or cohort? or compare? or condition or design or group? or intervention? or participant? or study)).ab.
35. random*.ti,ab. or controlled.ti.
36. (control year? or experimental year? or (control period? or experimental period?)).ti,ab.
37. (utili?ation or programme or programmes).ti.
38. (during adj5 period).ti,ab.
39. ((strategy or strategies) adj2 (improv* or education*)).ti,ab.
40. (clinical trial or multicenter study).pt.
41. evaluation studies as topic/ or prospective studies/ or retrospective studies/
42. ((evaluation or prospective or retrospective) adj study).ti,ab.
43. or/16‐42
44. "comment on".cm. or review.pt. or (review not "peer review*").ti. or randomized controlled trial.pt.
45. (rat or rats or cow or cows or chicken? or horse or horses or mice or mouse or bovine or animal?).ti,hw. or veterinar*.ti,ab,hw.
46. exp animals/ not humans.sh.
47. or/44‐46
48. 43 not 47
49. exp randomized controlled trial/
50. controlled clinical trial.pt.
51. randomi#ed.ti,ab.
52. placebo.ab.
53. drug therapy.fs.
54. randomly.ti,ab.
55. trial.ab.
56. groups.ab.
57. or/49‐56
58. Clinical Trials as topic.sh.
59. trial.ti.
60. or/49‐52,54,58‐59
61. exp animals/ not humans/
62. 60 not 61
63. 48 or 62
64. 15 and 63
Embase
1. exp health care personnel/
2. exp health care facility/
3. (clinician* or consultant* or dentist* or doctor* or family practition* or general practition* or gyn?ecologist* or h?ematologist* or internist* or nurse* or obstetrician* or occupational therapist* or p?ediatrician* or pharmacist* or physician* or physiotherapist* or psychiatrist* or psychologist* or radiologist* or surgeon* or surgery or therapist* or counsel?or* or neurologist* or optometrist* or paramedic* or social worker* or health professional* or health personnel or healthcare personnel or health care personnel).tw.
4. (hospital or hospitals or clinic or clinics or (primary adj2 care) or (health adj2 care)).tw.
5. or/1‐4
6. exp *accreditation/
7. *"peer review"/
8. *medical audit/
9. (organi?ation* adj raid*).tw.
10. (external* adj5 (accreditation or accredited or peer review or inspection or inspected or regulation or regulated or certified or certification or benchmark* or measured or measurement or evaluation or evaluated or audit or audits or auditing or assessment or assessed or monitored or visitation or surveillance or (control adj program*))).tw.
11. or/6‐10
12. (standards or standard or performance or criterion or criteria or indicator* or (clinical adj competence) or compliance or (clinical adj improvement) or (quality adj (improvement or management)) or (organi?ation* adj development) or (health adj care adj regulation)).tw.
13. 5 and 11 and 12
14. intervention?.ti. or (intervention? adj6 (clinician? or collaborat* or community or complex or design* or doctor? or educational or family doctor? or family physician? or family practitioner? or financial or GP or general practice? or hospital? or impact? or improv* or individuali?e? or individuali?ing or interdisciplin* or multicomponent or multi‐component or multidisciplin* or multi‐disciplin* or multifacet* or multi‐facet* or multimodal* or multi‐modal* or personali?e? or personali?ing or pharmacies or pharmacist? or pharmacy or physician? or practitioner? or prescrib* or prescription? or primary care or professional* or provider? or regulatory or tailor* or target* or team* or usual care)).ab.
15. (pre‐intervention? or preintervention? or "pre intervention?" or post‐intervention? or postintervention? or "post intervention?").ti,ab.
16. (hospital* or patient?).hw. and (study or studies or care or health* or practitioner? or provider? or physician? or nurse? or nursing or doctor?).ti,hw.
17. demonstration project?.ti,ab.
18. (pre‐post or "pre test*" or pretest* or posttest* or "post test*" or (pre adj5 post)).ti,ab.
19. (pre‐workshop or post‐workshop or (before adj3 workshop) or (after adj3 workshop)).ti,ab.
20. trial.ti. or ((study adj3 aim?) or "our study").ab.
21. (before adj10 (after or during)).ti,ab.
22. ("quasi‐experiment*" or quasiexperiment* or "quasi random*" or quasirandom* or "quasi control*" or quasicontrol* or ((quasi* or experimental) adj3 (method* or study or trial or design*))).ti,ab.
23. quasi experimental study/
24. *experimental design/ or *pilot study/
25. pilot.ti. or (pilot adj (project? or study or trial)).ab.
26. (time points adj3 (over or multiple or three or four or five or six or seven or eight or nine or ten or eleven or twelve or month* or hour? or day? or "more than")).ab.
27. ("time series" adj2 interrupt*).ti,ab.
28. (multicentre or multicenter or multi‐centre or multi‐center).ti.
29. (control adj3 (area or cohort? or compare? or condition or design or group? or intervention? or participant? or study)).ab.
30. random*.ti,ab. or controlled.ti.
31. (control year? or experimental year? or (control period? or experimental period?)).ti,ab.
32. (utili?ation or programme or programmes).ti.
33. (during adj5 period).ti,ab.
34. ((strategy or strategies) adj2 (improv* or education*)).ti,ab.
35. *clinical trial/ or *multicenter study/
36. *evaluation study/ or *prospective study/ or *retrospective study/
37. ((evaluation or prospective or retrospective) adj study).ti,ab.
38. or/14‐37
39. (rat or rats or cow or cows or chicken? or horse or horses or mice or mouse or bovine or animal?).ti.
40. (exp animal/ or exp invertebrate/ or animal experiment/ or animal model/ or animal tissue/ or animal cell/ or nonhuman/ or exp experimental animal/) not (human/ or normal human/ or human cell/)
41. or/39‐40
42. 38 not 41
43. random*.ti,ab.
44. factorial*.ti,ab.
45. (crossover* or cross over*).ti,ab.
46. ((doubl* or singl*) adj blind*).ti,ab.
47. (assign* or allocat* or volunteer* or placebo*).ti,ab.
48. crossover procedure/
49. single blind procedure/
50. randomized controlled trial/
51. double blind procedure/
52. or/43‐51
53. exp animal/ not human/
54. 52 not 53
55. 42 or 54
56. 13 and 55
HMIC
1. exp health service staff/
2. exp health services/
3. exp health buildings/
4. (clinician* or consultant* or dentist* or doctor* or family practition* or general practition* or gyn?ecologist* or h?ematologist* or internist* or nurse* or obstetrician* or occupational therapist* or p?ediatrician* or pharmacist* or physician* or physiotherapist* or psychiatrist* or psychologist* or radiologist* or surgeon* or surgery or therapist* or counsel?or* or neurologist* or optometrist* or paramedic* or social worker* or health professional* or health personnel or healthcare personnel or health care personnel).tw.
5. (hospital or hospitals or clinic or clinics or (primary adj2 care) or (health adj2 care)).tw.
6. or/1‐5
7. exp "Peer review"/
8. exp Benchmarking/
9. exp Accreditation/
10. exp Clinical audit/
11. exp Management audit/
12. (organi?ation* adj raid*).tw.
13. (external* adj5 (accreditation or accredited or peer review or inspection or inspected or regulation or regulated or certified or certification or benchmark* or measured or measurement or evaluation or evaluated or audit or audits or auditing or assessment or assessed or monitored or visitation or surveillance or (control adj program*))).tw.
14. or/7‐13
15. (standards or standard or performance or criterion or criteria or indicator* or (clinical adj competence) or compliance or (clinical adj improvement) or (quality adj (improvement or management)) or (organi?ation* adj development) or (health adj care adj regulation)).tw.
16. 6 and 14 and 15
Clinical trials registries
1. accreditation AND external
2. peer review AND external
3. inspection AND external
4. regulation AND external
5. certification AND external
6. benchmark AND external
7. assessment AND external
Appendix 2. Data extraction form
Cochrane Effective Practice and Organisation of Care Group (EPOC)
Modified EPOC Group Data Abstraction Form
External inspection versus external standards for improving healthcare organisation behaviour, professional behaviour and patient outcomes
Data collection
Name of reviewer:
Date:
Study reference:
Is the healthcare organisation review performed (i) by an external body (independent of the organisation under review) and (ii) against external standards?
If not ‐ EXCLUDE!
1. Inclusion criteria
1.1 Study design
1.1.1 RCT designs
1.1.2 NRCTdesigns
1.1.3 CBA designs
a) Contemporaneous data collection
b) Appropriate choice of control site/activity
c) At least two intervention and two control sites
1.1.4 ITS designs
a) Clearly defined point in time when the intervention occurred
b) At least 3 data points before and 3 after the intervention
1.2 Methodological inclusion criteria
a) The objective measurement of performance/provider behaviour or health/patient outcomes
b) Relevant and interpretable data presented or obtainable
N.B. A study must meet the minimum criteria for EPOC scope, design and methodology for inclusion in EPOC reviews. If it does not, COLLECT NO FURTHER DATA.
2. Interventions
2.1 Type of external inspection intervention
(state all interventions for each comparison/study group)
Group 1:
Group 2:
Group 3:
2.2 Control(s)
3. Type of targeted behaviour (state more than one where appropriate)
4. Participants
4.1 Characteristics of participating providers
4.1.1 Profession
4.1.2 Level of training
4.1.3 Clinical specialty
4.1.4 Age
4.1.5 Time since graduation (or years in practice)
4.2 Characteristics of participating patients
4.2.1 Clinical problem
4.2.2 Other patient characteristics
a) Age
b) Gender
c) Ethnicity
d) Other (specify)
4.2.3 Number of patients included in the study
a) Episodes of care
b) Patients
c) Providers
d) Practices
e) Hospitals
f) Communities or regions
5. Setting
5.1 Reimbursement system
5.2 Location of care
5.3 Academic status
5.4 Country
5.5 Proportion of eligible providers (or allocation units)
6. Methods
6.1 Unit of allocation
6.2 Unit of analysis
6.3 Power calculation
6.4 Risk of bias assessment
(If the trial is an ITS go directly to 6.4.2 for the RoB assessment)
6.4.1 Risk of bias assessment for randomised controlled trials (RCTs), nonrandomised controlled trials (NRCTs) and controlled before‐after studies (CBAs)
a) Was the allocation sequence adequately generated? (cut and paste from the paper verbatim)
| Score YES | If a random component in the sequence generation process is described (e.g. referring to a random numbers table) |
|
| Score NO | If a non‐random method is used (e.g. performed by date of submission) |
|
| Score UNCLEAR | If not specified in the paper
|
|
b) Was the allocation adequately concealed?
| Score YES | If the unit of allocation was by institution, team or professional and allocation was performed at all units at the start of the study; or if the unit of allocation was by patient or episode of care and there was some kind of centralised randomisation scheme; an on‐site computer system or if sealed opaque envelopes were used |
|
| Score NO | If none of the above mentioned methods were used (or if a CBA) |
|
| Score UNCLEAR | If not specified in the paper |
|
c) Were baseline outcome measurements similar?
| Score YES | If performance or patient outcomes were measured prior to the intervention, and no important differences were present across study groups |
|
| Score NO | If important differences were present and not adjusted for in analysis.** |
|
| Score UNCLEAR | If RCTs have no baseline measure of outcome** |
|
d) Were baseline characteristics similar?
| Score YES | If baseline characteristics of the study and control providers are reported and similar |
|
| Score NO | If there is no report of characteristics in the text or tables or if there are differences between control and intervention providers |
|
| Score UNCLEAR | If it is not clear in the paper (e.g. characteristics are mentioned in the text but no data were presented) |
|
e) Were incomplete outcome data adequately addressed?
| Score YES | If missing outcome variables were unlikely to bias the results (e.g. the proportion of missing data was similar in the intervention and the control group, or the proportion of missing data was less than the effect size, i.e. unlikely to overturn the study results) |
|
| Score NO | If missing data were likely to bias the results |
|
| Score UNCLEAR | If not specified in the paper (do not assume 100% follow‐up unless stated explicitly) |
|
f) Was knowledge of the allocated interventions adequately addressed?*
| Score YES | If the authors state explicitly that primary outcome variables was assessed blindly, or the outcomes are objective, e.g. length of hospital stay |
|
| Score NO | If the outcomes were not assessed blindly |
|
| Score UNCLEAR | If not specified in the paper |
|
g) Was the study adequately protected against contamination?
| Score YES | If allocation was by community, institution or practice and it is unlikely that the control group received the intervention |
|
| Score NO | If it is likely that the control group received the intervention (e.g. if patients rather than professionals were randomised) |
|
| Score UNCLEAR | If professionals were allocated within a clinic or practice and it is possible that communication between intervention and control professionals could have occurred (e.g. physicians within practices were allocated to intervention or control) |
|
h) Was the study free from selective outcome reporting?
| Score YES | If there is no evidence that outcomes were selectively reported (e.g. all relevant outcomes in the methods section are reported in the results section) |
|
| Score NO | If some important outcomes are subsequently omitted from the results |
|
| Score UNCLEAR | If not specified in the paper |
|
i) Was the study free from other risks of bias?
| Score YES | If no evidence of other risks of bias |
|
| Score NO |
|
|
| Score UNCLEAR |
|
|
* If some primary outcomes were imbalanced at baseline, assessed blindly or affected by missing data and others were not, each primary outcome can be scored separately.
**If 'UNCLEAR' or 'No', but there are sufficient data in the paper to do an adjusted analysis (e.g. baseline adjustment analysis or intention‐to‐treat analysis) the criteria should be re scored to 'Yes'.
6.4.2 Risk of bias assessment for interrupted time series (ITS) designs
Note: if the ITS study has ignored secular (trend) changes and performed a simple t‐test of the pre versus post intervention periods without further justification, the study should not be included in the review unless reanalysis is possible.
a) Was the intervention independent of other changes? (cut and paste from the paper verbatim)
| Score YES | If there are compelling arguments that the intervention occurred independently of other changes over time and the outcome was not influenced by other confounding variables/historic events during study period |
|
| Score NO | If reported that intervention was not independent of other changes in time If Events/variables identified, note what they are |
|
| Score UNCLEAR | If not specified in the paper
|
|
b) Was the shape of the intervention effects pre‐specified?
| Score YES | If point of analysis is the point of intervention OR a rational explanation for the shape of intervention effect was given by the author(s). Where appropriate, this should include an explanation if the point of analysis is NOT the point of intervention. |
|
| Score NO | If it is clear that the condition above is not met
|
|
| Score UNCLEAR | If not specified in the paper |
|
c) Was the intervention unlikely to affect data collection?
| Score YES | If reported that intervention itself was unlikely to affect data collection (for example, sources and methods of data collection were the same before and after the intervention) |
|
| Score NO | If the intervention itself was likely to affect data collection (for example, any change in source or method of data collection reported) |
|
| Score UNCLEAR | If not stated in the paper |
|
d) Was knowledge of the allocated interventions adequately prevented during the study?***
| Score YES | If the authors state explicitly that the primary outcome variables were assessed blindly, or the outcomes are objective, e.g. length of hospital stay. Primary outcomes are those variables that correspond to the primary hypothesis or question as defined by the authors. |
|
| Score NO | If the outcomes were not assessed blindly |
|
| Score UNCLEAR | If not specified in the paper |
|
e) Were incomplete outcome data adequately addressed?***
| Score YES | If missing outcome measures were unlikely to bias the results (e.g. the proportion of missing data was similar in the pre and post intervention periods or the proportion of missing data was less than the effect size, i.e. unlikely to overturn the study result) |
|
| Score NO | If missing data were likely to bias the results |
|
| Score UNCLEAR | If not specified in the paper (do not assume 100% follow‐up unless stated explicitly) |
|
f) Was the study free from selective outcome reporting?
| Score YES | If there is no evidence that outcomes were selectively reported (e.g. all relevant outcomes in the methods section are reported in the results section) |
|
| Score NO | If some important outcomes are subsequently omitted from the results |
|
| Score UNCLEAR | If not specified in the paper |
|
g). Was the study free from other risks of bias?
| Score YES | If no evidence of other risks of bias, e.g. should consider if seasonality is an issue (i.e. if January to June comprises the pre intervention period and July to December the post, could the 'seasons’ have caused a spurious effect) |
|
| Score NO |
|
|
| Score UNCLEAR |
|
|
*** If some primary outcomes were assessed blindly or affected by missing data and others were not, each primary outcome can be scored separately.
6.5 Consumer involvement
7. Prospective identification by investigators of barriers to change
8. Intervention
8.1 Description of the external inspection intervention (cut and paste from paper verbatim):
8.1.1 Voluntary or mandatory review
8.1.2 Universally or targeted review (applied to all of an organisation or only sub‐sections of it, e.g. to a clinical area or a professional group)
8.1.3 Purpose and focus of the review
8.1.4 Type of external standards (description and evidence base of standards)
8.1.5 Who set the standards?
8.1.6 Who did the inspection? (governmental or non‐governmental organisation)
8.1.7 How were results used (by the external body) to bring about the desired quality improvements
8.1.8 Recipient
8.1.9 Timing
a) Frequency/number of inspections
b) Duration of inspection
9. Outcomes
9.1 Description of the main outcome measure(s)
a) Healthcare organisational change (e.g. organisational performance)
b) Health professional behaviour
c) Patient outcomes
d) Economic variables (only if reported)
-
Costs of the intervention
-
Changes in direct health care costs as a result of the intervention
-
Changes in non‐health care costs as a result of the intervention
-
Costs associated with the intervention are linked with provider or patient outcomes in an economic evaluation
9.2 Length of post intervention follow‐up period
9.3 Identify a possible ceiling effect:
a) Identified by investigator
b) Identified by reviewer
10. Results (use extra page if necessary)
10.1.1 For RCTs and NRCTs
10.1.2 For CBAs
10.1.3 For ITSs
Appendix 3. Dropped hospital quality indicators
| Dropped hospital quality indicators | Reasons |
| Neonatal mortality | High variation in documenting neonatal births between hospitals. |
| Surgical wound infections and time to surgery | Only 9 hospitals (6 intervention and 3 control) performed surgery regularly, and many records lacked information on infections and times to surgery. |
| Financial solvency | Strict budgeting control across all hospitals in the region, with reportedly no additional funds being assigned. |
Study flow diagram.
| External inspection of compliance with COHSASA hospital accreditation standards versus no external inspection | ||||
| Recipient: public hospitals Settings: KwaZulu province, the Republic of South Africa Intervention: external inspection of compliance with accreditation standards Comparison: no inspection | ||||
| Outcomes | Intervention effect (range) | No of studies (hospitals) | Certainty of the evidence | Comments |
| Compliance with COHSASA accreditation standards at 2 years' follow‐up (change in total compliance score for 21/28 service elements ‐ for 7/28 service elements, data were not available) | I: pre: 48% (not reported), post: 78% (not reported) C: pre: 43% (not reported), post: 43% (not reported) Mean intervention effect: 30% (23% to 37%), P < 0.001 | 1 (18) | ⊕⊖⊖⊖ Very low 1,2 | Uncertain whether external inspection leads to improved compliance with standards. |
| Compliance with COHSASA accreditation standards ‐ subgroup of critical criteria analysed at 2 years' follow‐up (compliance score for 19 generic service elements, involving 426 predefined critical criteria) | I: pre: 38% (21% to 46%), post: 76% (55% to 96%) C: pre: 37% (28% to 47%), post: 38% (25% to 49%) Mean intervention effect: 37% (not reported), P < 0.001 | 1 (18) | ⊕⊖⊖⊖ Very low 1,2 | Uncertain whether external inspection leads to improved compliance with standards. |
| Indicators for hospital quality of care at 2 years' follow‐up | Median intervention effect: 2.4% (‐1.9% to 11.8%) | 1 (18) | ⊕⊖⊖⊖ Very low 1,2 | Uncertain whether external inspection improves median quality indicator scores. Only 1 of the indicators was indicative of higher quality in accreditation hospitals. |
| Mortality and condition‐specific measures of outcome related to patients' health | ‐ | ‐ | ‐ | Not measured or reported. |
| Unanticipated/adverse consequences | ‐ | ‐ | ‐ | Not measured or reported. |
| Costs and cost effectiveness | ‐ | ‐ | ‐ | Not measured or reported. |
| High = This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different‡ is low. Moderate = This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different‡ is moderate. Low = This research provides some indication of the likely effect. However, the likelihood that it will be substantially different‡ is high. Very low = This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different‡ is very high. ‡ Substantially different = a large enough difference that it might affect a decision. | ||||
| 1 Downgraded two levels for very serious risk of bias due to unclear blinding, baseline compliance and time differences in outcome measurements. 2 Downgraded one level for serious imprecision due to the small sample size and wide confidence intervals. C: control; COHSASA: Council for Health Services Accreditation for South Africa; I: intervention. | ||||
| External inspection of compliance with the Code of Practice and the law related to healthcare‐acquired infections | ||||
| Recipient: all acute trusts Settings: England Intervention: external inspection of compliance with the Code of Practice and the Health Act 2006 related to healthcare‐acquired infections Comparison: no control group (time series) | ||||
| Outcomes | Mean intervention effect (95% CI) | No of studies (trusts) | Certainty of the evidence | Comments |
| MRSA infection rate | At 3 months: ‐100 (‐221.0 to 21.5) cases per quarter (P = 0.096) | 1 (168) | ⊕⊖⊖⊖ Very low1,2 | Uncertain whether external inspection lowers MRSA infection rates. Regression analysis showed similar MRSA rate before and after the external inspection (difference in slope 24.27, 95% CI ‐10.4 to 58.9; P = 0.147). |
| Unanticipated/adverse consequences | ‐ | ‐ | ‐ | Not measured or reported. |
| Costs and cost effectiveness | ‐ | ‐ | ‐ | Not measured or reported. |
| High = This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different‡ is low. Moderate = This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different‡ is moderate. Low = This research provides some indication of the likely effect. However, the likelihood that it will be substantially different‡ is high. Very low = This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different‡ is very high. ‡ Substantially different = a large enough difference that it might affect a decision. | ||||
| 1 Downgraded one level for serious risk of bias due to the probability that the intervention was not independent of other changes. 2 Downgraded one level for serious imprecision due to wide confidence intervals. CI: confidence interval; MRSA: methicillin‐resistant Staphylococcus aureus. | ||||
| Indicator | Intervention n = 10 | Control n = 10 | Intervention effect | P value | ||||
| Time 1 | Time 2 | Change | Time 1 | Time 2 | Change | |||
| 1. Nurse perceptions | 59.3 | 60.8 | 1.5 | 60.8 | 56.5 | ‐4.2 | 15.7 | 0.031 |
| 2. Patient satisfaction | 86.9 | 91.5 | 4.6 | 87.0 | 90.1 | 3.1 | 1.5 | 0.484 |
| 3. Medical education | 42.9 | 43.1 | 0.2 | 41.5 | 40.0 | ‐1.5 | 1.7 | 0.395 |
| 4. Medical records: accessibility | 85.4 | 77.5 | ‐7.9 | 79.4 | 68.4 | ‐11.0 | 3.1 | 0.492 |
| 5. Medical records: completeness | 47.1 | 49.1 | 2.0 | 48.6 | 44.9 | ‐3.7 | 5.7 | 0.114 |
| 6. Completeness of perioperative notes | 70.2 | 72.7 | 2.5 | 65.2 | 69.6 | 4.4 | ‐1.9 | 0.489 |
| 7. Ward stock labelling | 66.0 | 81.8 | 15.8 | 45.6 | 49.6 | 4.0 | 11.8 | 0.112 |
| 8. Hospital sanitation | 59.7 | 62.8 | 3.1 | 50.2 | 55.7 | 5.5 | ‐2.4 | 0.641 |
| All scores were standardised to a 100‐point scale, with 100 as high. Positive intervention effects represent improvements in intervention hospitals that exceeded the improvements in control hospitals. P values are based on an analysis of variance (ANOVA) model with intervention group and hospital size as main effects. | ||||||||
| Service elements | Intervention hospitals | Control hospitals | Mean intervention effect (95% CI) | P value | ||||||
| No of hospitals | Mean baseline | Mean external | Mean change | No of hospitals | Mean baseline | Mean external | Mean change | |||
| Management service | 9 | 50 | 79 | 29 | 10 | 42 | 44 | 2 | 27 (17 to 37) | < 0.001 |
| Administrative support | 9 | 57 | 73 | 16 | 10 | 56 | 52 | ‐4 | 20 (4 to 36) | 0.038 |
| Nursing management | 9 | 55 | 87 | 32 | 10 | 51 | 50 | ‐1 | 33 (24 to 43) | < 0.001 |
| Health and safety | 9 | 35 | 75 | 40 | 10 | 28 | 32 | 4 | 36 (23 to 51) | < 0.001 |
| Infection control | 9 | 45 | 88 | 43 | 10 | 39 | 42 | 3 | 40 (27 to 52) | < 0.001 |
| Operating theatre | 9 | 57 | 86 | 29 | 10 | 50 | 53 | 3 | 26 (16 to 35) | < 0.001 |
| Sterilising and disinfectant | 9 | 47 | 81 | 34 | 10 | 33 | 35 | 2 | 32 (22 to 41) | < 0.001 |
| Medical inpatient | 8 | 49 | 78 | 29 | 10 | 44 | 46 | 2 | 27 (17 to 35) | < 0.001 |
| Pharmaceutical | 9 | 41 | 75 | 34 | 10 | 42 | 38 | ‐4 | 38 (25 to 52) | < 0.001 |
| Paediatric inpatient | 8 | 51 | 78 | 27 | 10 | 44 | 46 | 2 | 25 (17 to 33) | < 0.001 |
| Maternity inpatient | 9 | 53 | 82 | 29 | 10 | 52 | 51 | ‐1 | 30 (23 to 36) | < 0.001 |
| Surgical inpatient | 9 | 48 | 81 | 33 | 10 | 46 | 46 | 0 | 33 (25 to 42) | < 0.001 |
| Laundry | 9 | 30 | 68 | 38 | 10 | 23 | 24 | 1 | 37 (26 to 47) | < 0.001 |
| Housekeeping | 9 | 37 | 73 | 36 | 10 | 33 | 32 | ‐1 | 37 (24 to 51) | < 0.001 |
| Maintenance | 9 | 51 | 74 | 23 | 10 | 43 | 44 | 1 | 22 (11 to 34) | 0.004 |
| Resuscitation | 9 | 31 | 83 | 52 | 10 | 25 | 25 | 0 | 52 (43 to 61) | < 0.001 |
| Food | 9 | 41 | 73 | 32 | 10 | 38 | 38 | 0 | 32 (24 to 41) | < 0.001 |
| Diagnostic | 9 | 44 | 79 | 35 | 10 | 38 | 39 | 1 | 34 (22 to 46) | < 0.001 |
| Critical care category 2 | 2 | 46 | 92 | 46 | 4 | 58 | 61 | 3 | 43 (15 to 70) | NA |
| Casual | 7 | 48 | 81 | 33 | 5 | 40 | 43 | 3 | 30 (17 to 44) | NA |
| Outpatient | 8 | 46 | 83 | 37 | 9 | 40 | 43 | 3 | 34 (20 to 47) | NA |
| Occupational | 3 | 42 | 85 | 43 | 4 | 43 | 47 | 4 | 39 (16 to 62) | NA |
| Physiotherapy | 7 | 46 | 84 | 38 | 4 | 38 | 42 | 4 | 34 (24 to 45) | NA |
| Laboratory | 9 | 46 | 85 | 39 | 8 | 43 | 40 | ‐3 | 42 (31 to 53) | < 0.001 |
| Medical life support | 9 | 37 | 74 | 37 | 10 | 21 | 23 | 2 | 35 (22 to 49) | 0.001 |
| Community health | 4 | 50 | 88 | 38 | 8 | 54 | 50 | ‐4 | 42 (28 to 56) | NA |
| Social work | 4 | 53 | 82 | 29 | 5 | 40 | 44 | 4 | 25 (6 to 41) | NA |
| Medical practitioner | 9 | 51 | 75 | 24 | 10 | 44 | 42 | ‐2 | 26 (13 to 40) | 0.004 |
| Overall services score | 9 | 48 | 78 | 30 | 10 | 43 | 43 | 0 | 30 (23 to 37) | < 0.001 |
| CI: confidence interval; COHSASA: Council for Health Services Accreditation for South Africa; NA: not available | ||||||||||
| Author Year | Compliance with COHSASA accreditation standards (28 service elements) | Hospital quality indicators (n = 8) |
| At 2 years' follow‐up: Total compliance score for 21/28 service elements, for which comparisons were possible, rose from 48% to 78% in intervention hospitals, while control hospitals maintained the same compliance score throughout (43%). Mean intervention effect was 30% (23% to 37%). Looking at the individual scores of compliance with the accreditation standards for each service element, the results were mixed, with 21/28 service elements showing a beneficial effect of the inspections. Mean intervention effect ranged from 20% to 52%, while data for the remaining 7 service elements were not available, i.e. some of the service elements were only evaluated in the higher‐level hospitals, so comparisons between the intervention and control hospitals was not appropriate due to small sample size. Subanalysis of the standards that a priori were deemed by the COHSASA as being 'critical' for a specific function was performed. As some of the 28 service elements evaluated in the accreditation process were not applicable for all hospitals, this left 19 generic service elements, yielding 424 critical criteria for the subanalysis. These critical criteria were mainly drawn from the following service elements: obstetric and maternity inpatient services; operating theatre and anaesthetic services; resuscitation services; paediatric services and medical inpatient services. Subanalysis showed improved mean compliance with the critical standards in intervention hospitals: total score rose from 38% (range 21% to 46%) to 76% (range 55% to 96%). Control hospitals maintained the same compliance score throughout: 37% (range 28% to 47%) before the intervention and 38% (range 25% to 49%) after the intervention. There was a difference in means between groups (P < 0.001). | Effects on the hospital quality indicators were mixed, with mean intervention effects ranging from ‐1.9 to +11.8, and only 1/8 indicators: 'nurses' perception of clinical care', showed a beneficial effect of the intervention (see below). Nurses' perception of clinical care Intervention hospitals: pre: 59.3%, post: 60.8% (change 1.5%); control hospitals: pre: 60.8%, post: 56.5% (change ‐4.2%); intervention effect: 5.7 percentage points (P = 0.031). Patient satisfaction with care: Intervention hospitals: pre: 86.9%, post: 91.5% (change 4.6%); control hospitals: pre: 87.0%, post: 90.1% (change 3.1%); intervention effect: 1.5 percentage points (P = 0.484). Patient medication education: Intervention hospitals: pre: 42.9%, post: 43.1% (change 0.2%); control hospitals: pre: 41.5%, post: 40.0% (change ‐1.5%); intervention effect: 1.7 percentage points (P = 0.395). Medical records: accessibility: Intervention hospitals: pre: 85.4%, post: 77.5% (change ‐7.9%); control hospitals: pre: 79.4%, post: 68.4% (change ‐11.0%); intervention effect: 3.1 percentage points (P = 0.492). Medical records: completeness (consisting of 2 components: admissions and discharge): Intervention hospitals: pre: 47.1%, post: 49.1% (change 2.0%); control hospitals: pre: 48.6%, post: 44.9% (change ‐3.7%); intervention effect: 5.7 percentage points (P = 0.114). Completeness of peri‐operative notes: Intervention hospitals: pre: 70.2%, post: 72.7% (change 2.5%); control hospitals: pre: 65.2%, post: 69.6% (change 4.4); intervention effect: ‐1.9 percentage points (P = 0.489). Ward stock labelling: Intervention hospitals: pre: 66.0%, post: 81.8% (change 15.8%); control hospitals: pre: 45.6%, post: 49.6% (change 4.0%); intervention effect: 11.8 percentage points (P = 0.112). Hospital sanitation*: Intervention hospitals: pre: 59.7%, post: 62.8% (change 3.1%); control hospitals: pre: 50.2%, post: 55.7% (change 5.5); intervention effect: ‐2.4 percentage points (P = 0.641). * Consisted of the assessment of 6 items (availability of soap, water, paper towels and toilet paper and whether toilets were clean and in working order) of which a composite score was developed. | |
| COHSASA: Council for Health Services Accreditation for South Africa. | ||
| Author Year | Infection rate |
| Date; No of cases Re‐analysis of the MRSA data, as an ITS: Difference (24.27, 95% CI ‐10.4 to 58.9) between the preslope (‐107.6) and the postslope (‐83.32) suggested similar infection rates before and after the inspection (P = 0.147). When the downward trend in MRSA rate before the intervention was considered, the results showed a mean (95% CI) decrease with 100 (‐221.0 to 21.5) cases at 3 months' follow‐up (P = 0.096), 75 (‐217.2 to 66.3) cases at 6 months' follow‐up (P = ‐0.259), 27 (‐222.1 to 168.2) cases at 12 months' follow‐up (P = 0.62), and an increase with 70 (‐250.5 to 391) cases per quarter at 24 months' follow‐up (P = 0.632). | |
| CI: confidence interval; ITS: interrupted time series; MRSA: methicillin‐resistant Staphylococcus aureus. | |
