Types of studies

We expect few randomised trials, so we will also include non‐randomised trials, controlled before‐after studies, and interrupted time series studies that meet methodological quality criteria of the Effective Practice and Organisation of Care Group (EPOC) (EPOC 2017a). We will include full‐text studies and will consider conference abstracts, as well as unpublished data.

Types of participants

This review will look at use of Lean management in the following healthcare settings.

• Hospital care.

• Primary care.

• Community or home care.

• Rehabilitation.

We will include all individuals impacted by implementation of Lean management, which we expect will include the following.

• All employees of the healthcare system, such as CEOs, healthcare professionals, administrative staff, and support staff.

• Patients and their families.

• Lean experts and key stakeholders.

However, because Lean impacts all individuals who interact with organisations that have adopted the management system, we will not exclude participants who are not listed. Instead, exclusion will be based on the focus of Lean implementation.

We will not include any studies focused on the following.

• Animal research.

• Drug discovery.

• Drug manufacturing.

• Medical device delivery.

• Medical device manufacturing.

• Software development.

• Teaching (e.g. techniques for teaching nursing).

Types of interventions

Despite increased use of Lean terminology and popularisation of Lean management approaches, especially in the hospital sector, little agreement has been reached regarding how Lean is described and defined in the literature (D’Andreamatteo 2015). In response to this shortcoming, we developed an operational definition of Lean management in health care (Rotter 2016). This definition requires that all included studies focus on interventions that (1) occurred in an organisation or a subunit of an organisation (e.g. department, ward) that has made a commitment to Lean philosophy (including a commitment to both Lean principles and continuous improvement); and (2) utilised at least one Lean assessment activity or Lean improvement activity whereby:

• Lean philosophy is a set of core ideas that make up Lean and include two components: a commitment to Lean principles and a commitment to continuous improvement;

• Lean principles refer to an overarching set of principles aimed at transforming workplace culture (Kruskal 2012). These include focus on eliminating waste; improving the flow of patients, providers, and supplies; and ensuring that all processes add value to customers (Black 2008; DelliFraine 2010; Holden 2011; Kim 2006; Mazzocato 2010; Poksinska 2010). Further, Lean principles suggest that problems are identified and addressed by front‐line staff members, as it is believed that the people doing the work are best suited to create solutions (Casey 2009; Holden 2011; Kruskal 2012);

• Commitment to continuous improvement refers to the acknowledgement that Lean does not occur as a single intervention but instead requires a commitment to continually improving the workplace (DelliFraine 2010; Holden 2011; Mazzocato 2010);

• Lean activities comprise a set of management practices, tools, or techniques that can be directly observed and are prescribed to improve the workplace (Shah 2007). Two types of Lean activities have been developed: assessment activities and improvement activities;

• Lean assessment activities work as analytical tools to identify waste and areas of possible improvement. These activities allow team members to see problems and identify opportunities to reduce waste and make improvements but do not prescribe specific solutions. Lean assessment activities include value stream mapping (VSM); spaghetti diagrams; rapid process improvement workshops (RPIWs); Gemba walks; and root cause analysis; and

• Lean improvement activities suggest specific ways to reduce waste and improve the workplace while setting up new working practices. These include actions and concepts such as 5S (‘Sort, Sweep, Simplify, Standardise, Sustain/Self‐Discipline’) events; levelled production; daily visual management (DVM) (including Kanban supply management); standard work; and 'Stop the Line' techniques.

We will include studies that test multi‐faceted Lean approaches (e.g. Lean management combined with the Six Sigma approach) if Lean components can be assessed separately from other process improvement techniques or tools used (e.g. Six Sigma).

Types of outcome measures

Electronic searches

We will search the Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE) for primary studies included in related systematic reviews.

For primary studies, we will search the following databases with neither date nor language limits. We will apply two methodological filters: the Cochrane Highly Sensitive Search Strategy (sensitivity‐ and precision‐maximising version ‐ 2008 revision) to identify randomised trials (Lefebvre 2011); and an EPOC methodology filter to identify non‐randomised designs. See Appendix 1 for a draft strategy for OVID Medline.

• Cochrane Central Register of Controlled Trials (CENTRAL) (Wiley).

• MEDLINE, 1946 ‐ In‐Process and other non‐indexed citations; Daily Update (OvidSP).

• Embase, 1947 – (OvidSP).

• Cumulative Index to Nursing and Allied Health Literature (CINAHL), 1980 – (EBSCO).

• Dissertations and Theses Database, 1861 (ProQuest).

• Web of Science.

  • Science Citation Index Expanded (1900‐).

  • Social Sciences Citation Index (1900‐).

  • Arts & Humanities Citation Index (1975‐).

  • Conference Proceedings Citation Index ‐ Science (1990‐).

  • Conference Proceedings Citation Index ‐ Social Science & Humanities (1990‐).

• Business Source Premiere (EBSCO).

• EconLit (economics research database) (EBSCO).

• Public Affairs Information Service (PAIS) (ProQuest).

• Social Work Abstracts (EBSCO).

Searching other resources

Selection of studies

We will import into all titles and abstracts retrieved by electronic searching into EndNote x7 (EndNote 2016). We will break results into five groups for screening. Pairs of review authors (LA and CP, JC and CP, MF and CP, AK and LH, TR and LK) will independently screen titles and abstracts for inclusion. We will code all potentially eligible studies as 'retrieve' (eligible or potentially eligible/unclear) or 'exclude'. We will retrieve full‐text study reports/publications for all studies coded as 'retrieve' by at least one review author. We will divide retrieved studies into three groups. Pairs of review authors (CP and MF, CP and AK, CP and LA) will independently screen the full‐text reports, will identify studies for inclusion, and will identify and record reasons for exclusion of ineligible studies. We will resolve disagreements through discussion. If consensus cannot be reached, we will consult a third review author (TR). We will report the number of studies excluded upon title review and full‐text review. We will report articles that appear relevant to the review question but were excluded from analysis along with reasons for exclusion as per the Cochrane Handbook for Systematic Reviews of Interventions, Section 7.2.5. (Higgins 2011).

We will collate multiple reports of the same study so that each study, rather than each report, is the unit of interest in the review. We will provide any information we can obtain about ongoing studies. We will record the selection process in sufficient detail to complete a PRISMA flow diagram and 'Characteristics of excluded studies' tables (Liberati 2009).

Data extraction and management

Pairs of two review authors will independently extract data according to the double data entry method by using a standardised data extraction form developed in Microsoft Access (Microsoft 2016). We will pilot‐test the data extraction form on three studies from the review and will extract all data directly from included studies. We will refer disagreements to a third review author. If necessary, we will contact authors of the primary studies for additional information. Areas of data extraction will include:

• study characteristics: author, publication year, healthcare setting (e.g. primary care, hospital, long‐term care), location (urban vs rural), country;

• intervention: type of Lean intervention used, Lean vs Lean Six Sigma, theoretical background used in implementation, target population;

• population characteristics (participants): age, gender, number of participants;

• population characteristics (healthcare providers): types of healthcare professionals involved, number of healthcare professionals involved, types of additional personnel involved (e.g. senseis, continuous quality improvement (CQI) group), number of additional personnel involved; and

• outcomes: participant, healthcare provider, utilisation and access, adverse effects or harms, and resource use.

Assessment of risk of bias in included studies

Two review authors will independently assess risk of bias for each study using criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions and guidance from the EPOC group (EPOC 2017c; Higgins 2011). We will resolve disagreements by discussion and will refer the matter to a third review author if agreement cannot be reached. For randomised trials, non‐randomised trials, and controlled before‐after studies, we will assess risk of bias according to the following domains.

• Random sequence generation.

• Allocation concealment.

• Baseline outcomes measurement.

• Baseline characteristics.

• Incomplete outcome data.

• Knowledge of intervention allocation.

• Protection from contamination.

• Selective outcome reporting.

• Other bias.

For interrupted time series studies, we will assess risk of bias according to the following domains.

• Independance from other changes.

• Specification of the shape of effects before analysis.

• Independance from data collection.

• Knowledge of intervention allocation.

• Incomplete outcome data.

• Selective outcome reporting.

• Other bias.

We will rate each potential source of bias as high, low, or unclear and will provide a quote from the study report together with a justification for our judgement in the 'Risk of Bias' table, as recommended in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). We will summarise risk of bias judgements across different studies for each of the domains listed. We will consider blinding separately for different key outcomes when necessary (e.g. for unblinded outcome assessment, risk of bias for all‐cause mortality may be different from that assigned for a patient‐reported pain scale). When information on risk of bias is related to unpublished data or correspondence with a trialist, we will note this in the risk of bias table.

When considering treatment effects, we will take into account the risk of bias for studies that contribute to that outcome.

Measures of treatment effect

We will estimate the effect of the intervention using risk ratios for dichotomous data, and mean differences for continuous data. We will present all measures with associated 95% confidence intervals (CIs). We will ensure that an increase in scores for continuous outcomes can be interpreted in the same way for each outcome, will explain the direction to the reader, and will report when directions were reversed, if this was necessary.

We will define pre‐intervention data points as all data collected before initiation of changes resulting from a Lean intervention. When possible, we will define postintervention data points as data collected following completion of changes resulting from a Lean intervention. However, we acknowledge that Lean dictates that improvement should be continuous, and therefore the implementation period may not be well defined. If this is the case, we will consider all data collected following initiation of changes resulting from a Lean intervention as postintervention data points; in such cases, we will note this and will consider it when drawing conclusions. In the case of interrupted time series studies, we will present the change in intercept for the postintervention phase as well as slope estimates for pre‐intervention and postintervention phases.

Unit of analysis issues

We will include cluster‐randomised trials and will combine them with randomised trials when clinical and statistical heterogeneity is low. We will deal with cluster‐randomised trial results that do not include an intracluster correlation coefficient (ICC) as per the section below on dealing with missing data. However, we will not exclude studies from the review because of unit of analysis issues. We will retain these studies and will include them in a subsequent sensitivity analysis.

Dealing with missing data

We will contact investigators to verify key study characteristics and to obtain missing outcome data when possible (e.g. when a study is identified as abstract only). For trials of cluster‐randomised design, we will utilise an estimate of an ICC taken from an ICC database of the University of Aberdeen (ICC Database 2008).

If we find before‐after comparisons that provide at least three pre‐intervention and three postintervention measures, we will reanalyse them using segmented time series regression techniques (EPOC 2017d). We will extract data from graphic presentations using Engauge Digitizer (Mitchell 2016). For each analysis, we will report the pre‐intervention intercept, the pre‐intervention slope, the postintervention intercept, and the postintervention slope.

Assessment of heterogeneity

If we find a sufficient number of studies, we will conduct a meta‐analysis. We will use the I² statistic to measure heterogeneity among the trials in each analysis. We will consider an I² value greater than 60% to serve as evidence of substantial heterogeneity of a magnitude at which statistical pooling is not appropriate. If we identify substantial heterogeneity, we will explore this by conducting prespecified subgroup analyses.

Assessment of reporting biases

We will attempt to contact study authors to ask them to provide missing outcome data. When this is not possible, and the missing data are thought to introduce serious bias, we will explore the impact of including such studies in the overall assessment of results by performing a sensitivity analysis. If we are able to pool more than 10 trials, we will create and examine a funnel plot to explore possible publication biases, while interpreting results with caution (Sterne 2011).

Data synthesis

We will undertake meta‐analyses only when this is meaningful (i.e. when interventions, participants, and the underlying clinical question are similar enough for pooling to make sense). Trialists commonly indicate that data are skewed by reporting medians and interquartile ranges. When we encounter this, we will note that the data are skewed and will consider the implications of this. When a single trial reports inclusion of multiple trial arms, we will include only relevant arms. If two comparisons (e.g. Lean in Hospital A vs usual care and Lean in Hospital B vs usual care) must be entered into the same meta‐analysis, we will halve the control group to avoid double counting.

Subgroup analysis and investigation of heterogeneity

We plan to group studies into the following categories.

• Subgroup of Lean methods (e.g. Lean, Lean Six Sigma) used.

• Setting (e.g. Emergency Department, Laboratory, Pharmacy) in which the intervention was implemented.

To conduct the subgroup analysis, we will use all outcomes that are common to at least three studies. We will perform a separate analysis for each outcome.

Sensitivity analysis

We will perform sensitivity analysis defined a priori to assess the robustness of our conclusions and to explore the impact of this analysis on effect size. This will involve:

• restricting the analysis to published studies;

• restricting the analysis to studies with low overall risk of bias; and

• imputing missing data.