Types of studies

We will include randomized controlled trials (RCTs) and quasi‐experimental studies that compare Alcoholics Anonymous or treatments designed to facilitate engagement in Alcoholics Anonymous with other interventions such as MET or CBT, 12‐step program variants, or no treatment. Given the potential healthcare cost savings of using freely available community‐based recovery resources such as Alcoholics Anonymous and TSF/12‐step treatments, we will consider cost‐effectiveness studies that use controlled clinical trial (CCT), quasi‐experimental, or observational designs as well as RCTs.

Types of participants

We will include adults (18 years or older) with AUD, alcohol abuse, or alcohol dependence, as defined using commonly used standardized criteria (i.e. theDiagnostic and Statistical Manual of Mental Disorders, 4th and 5th editions (APA 1994; APA 2013); the 9th and 10th revisions of the International Statistical Classification of Diseases and Related Health Problems (WHO 2010); validated screening or diagnostic tools), attending Alcoholics Anonymous meetings and/or participating in TSF/12‐step treatments. We will exclude studies involving participants coerced to attend Alcoholics Anonymous meetings by legal order.

Types of interventions

Experimental interventions will include Alcoholics Anonymous participation and TSF. Control interventions will include other psychological clinical interventions (e.g. MET, CBT, etc.), other 12‐step program variants (e.g. studies comparing different 12‐step interventions), and no treatment (e.g. wait list control).

Types of outcome measures

We will describe variations in outcome measures of the same domain (e.g. in the time window assessed) in the narrative, as appropriate, along with any potential limitations to inferences and generalizability that may result from such variation.

Electronic searches

We will identify published, unpublished, and ongoing studies by searching the following databases from their inception.

• Cochrane Drugs and Alcohol Group Specialised Register.

• Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library.

• MEDLINE PubMed (from 1946 onwards).

• Embase Ovid (from 1974 onwards).

• CINAHL EBSCO (Cumulative Index to Nursing and Allied Health Literature; from 1982 onwards).

• PsycINFO (from 1880s onwards).

We will model the subject strategies for databases on the search strategy designed for CENTRAL (Appendix 1).

We will search the following trial registries.

• World Health Organization International Clinical Trials Registry Platform (www.who.int/trialsearch).

• ClinicalTrials.gov (www.clinicaltrials.gov).

We will also search the above databases for health economics evidence.

Searching other resources

We will attempt to identify other potentially eligible studies by searching the reference lists of retrieved included studies, systematic reviews, and meta‐analyses.

Selection of studies

Two review authors (JK, KH) will independently scan the abstract, title, or both, of every record retrieved to determine which studies should be further evaluated for inclusion. We will investigate all potentially relevant articles as full text, and resolve any discrepancies between the two review authors through consultation and discussion with the third author (MF). We will add studies that remain questionable for review inclusion even after discussion among the authors to the list of articles awaiting assessment. We will contact study authors for clarification when necessary. We will delineate the study selection process in a PRISMA flow chart (Liberati 2009; Moher 2009).

Data extraction and management

Using a standardized data extraction form, two authors (JK and KH) will independently abstract the relevant elements of the study, including study design, sample characteristics, description of the experimental and control interventions, outcomes, study funding, and conflicts of interest. Any disagreements regarding these details will be resolved among all review authors by discussion. We will contact study authors for clarification when necessary.

Assessment of risk of bias in included studies

Two review authors (JK and MF) will independently assess the risk of bias in the included studies using the criteria recommended in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a). The recommended approach for assessing risk of bias in studies comprises the assessment of seven domains: sequence generation, allocation concealment (selection bias), blinding of participants and providers (performance bias), blinding of outcome assessor (detection bias), incomplete outcome data (attrition bias), selective outcome reporting (reporting bias) and other sources of bias. The first part of the assessment process involves describing what was reported to have happened in the study. The second part involves assigning a judgment relating to the risk of bias for that entry, in terms of low, high, or unclear risk. To make these judgments we will use the criteria indicated by the Cochrane Handbook for Systematic Reviews of Interventions adapted to the addiction field (Higgins 2011b) (see Appendix 2 for details).

We will address the domains of sequence generation and allocation concealment (avoidance of selection bias) using a single entry for each study.

We will consider the blinding of participants, personnel, and outcome assessor (avoidance of performance bias and detection bias) separately for objective outcomes (e.g. drop out from treatment, use of substance measured by urinalysis, participants relapsed at the end of follow‐up, participants engaged in further treatments) and subjective outcomes (e.g. duration and severity of signs and symptoms of withdrawal, participant self‐reported use of substance, side effects, social functioning as integration at school or at work, family relationship).

We will consider incomplete outcome data (while also taking due note of any observed attrition bias) for all outcomes except for drop‐out from treatment, which is very often the primary outcome measure in trials on addiction.

We will operationalize 'Risk of bias' tables to be used for the assessment of RCTs, CCTs, and prospective observational studies, according to the criteria recommended by the Cochrane Drugs and Alcohol Review Group (see Appendix 2 for details).

We will evaluate any cost‐effectiveness studies using the appropriate Cochrane 'Risk of bias' criteria, according to the specific design of each study (e.g. if the cost‐effectiveness study is conducted using an RCT or CCT design, then we will use the appropriate 'Risk of bias' criteria).

Measures of treatment effect

Where applicable, we will calculate the standardized mean difference for continuous variables (e.g. percent days abstinent) or the relative risk for dichotomous variables (e.g. proportion of participants completely abstinent) with the uncertainty of the estimate expressed using 95% confidence intervals. If the degree of heterogeneity across studies is such that it precludes us from conducting a meta‐analysis, we will describe the findings and effects using a narrative review. Where applicable (e.g. in multiarm studies), we will report outcomes separately.

Unit of analysis issues

If we include multiarm studies, wherein there are more than two intervention conditions being compared, we will combine all the relevant experimental groups into a single group where appropriate and compare this with the control group to avoid double‐counting participants in the control groups.

Dealing with missing data

All the review authors will appraise the presence and impact of missing data on study findings and detail in the narrative as appropriate. If necessary, we will contact the original study authors to attempt to obtain any missing data and information on their potential impact.

Assessment of heterogeneity

We will analyse heterogeneity by means of the I² statistic and the Chi² test. We will regard heterogeneity as substantial if the I² statistic is greater than 50% or the P value lower than 0.10 for the Chi² test for heterogeneity. Following the guidance in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011) we will consider the following values to denote no important, moderate, substantial, or considerable heterogeneity, respectively: 0% to 40%, 30% to 60%, 50% to 90%, and 75% to 100%. If we find considerable levels of heterogeneity (i.e. ≥ 75%), we will explore possible reasons by visually inspecting the forest plot to identify studies that might be contributing to heterogeneity.

Assessment of reporting biases

We will report whether studies obtained ethical approval, or were registered, or both, where applicable (e.g. for RCTs). We will obtain and extract the registered outcomes and compare them with the reported outcomes to assess reporting bias. We also will visually inspect funnel plots (plots of the effect estimate from each study against the sample size or effect standard error) to identify possible publication bias. We acknowledge that the funnel plot should be seen as a generic means of displaying small‐study effects (the tendency for the intervention effects estimated in smaller studies to differ from those estimated in larger studies, so that asymmetry could be due to publication bias), but small‐study effects may be due to reasons other than publication bias, such as a greater risk of bias in smaller studies, and the inclusion of a more‐restricted and thus more‐responsive population, or merely due to the role of chance. We will inspect funnel plot symmetry when there are at least 10 studies included in a meta‐analysis.

Data synthesis

We will combine the outcomes from individual trials through meta‐analysis where possible (comparability of intervention and outcomes between trials) using a random‐effects model because we expect a certain degree of heterogeneity among trials.

As detailed in the Cochrane Handbook for Systematic Reviews of Interventions (section 15.6.3; Higgins 2011c) there is currently no consensus regarding the appropriate methods for pooling combined estimates of cost‐effectiveness studies, as well as potential issues concerning the validity of such methods when combining metrics across cost‐effectiveness studies. For these reasons, rather than conduct a meta‐analysis, we plan to summarize results from any cost‐effectiveness studies in the narrative.

Subgroup analysis and investigation of heterogeneity

We will perform subgroup analyses according to AUD severity, where appropriate (e.g. according to the DSM IV criteria 'abuse' versus 'dependence').

Sensitivity analysis

We will perform sensitivity analyses to assess the robustness of the results, including examining how the pattern of findings is affected by excluding studies at high risk of selection or attrition bias. We will also conduct sensitivity analyses across types of study designs, as appropriate (e.g. RCTs versus quasi‐experimental studies).