Types of studies

Randomised controlled trials (RCTs) and quasi‐RCTs (in which methods of allocation to groups are not truly random, e.g. day of week, case number).

Types of participants

Families with children from birth to age of completion of compulsory education from all ethnic backgrounds and family sizes, however defined by the trialists.

Types of interventions

FAST programmes compared with waiting list, usual services, alternate service or no treatment.

Types of outcome measures

We will assess outcome measures in the short term (up to one year follow‐up), the medium term (between one and two years' follow‐up), and long term (over two years post FASTWORKS). We will record the timing of outcome assessment as presented in studies.

Electronic searches

We will search the electronic resources listed below.

1. Cochrane Central Register of Controlled Trials (CENTRAL; current issue) in the Cochrane Library, which includes the Cochrane Developmental, Psychosocial and Learning Problems Specialised Register.

2. MEDLINE Ovid (1946 onwards).

3. Embase Ovid (1980 onwards).

4. PsycINFO Ovid (1806 onwards).

5. ERIC EBSCOhost (Education Resources Information Center; 1966 onwards).

6. British Education Index EBSCOhost (BEI; 1950 onwards).

7. ProQuest Education Database (1988 onwards).

8. Education Abstracts (HW Wilson) EBSCOhost (1983 onwards).

9. Social Science Citation Index Web of Science (1970 onwards).

10. Conference Proceedings Citation Index Social Science and Humanities Web of Science (1990 onwards).

11. EPPI‐Centre Database of Education Research (eppi.ioe.ac.uk/webdatabases/Search.aspx).

12. Campbell Library of Systematic Reviews (www.campbellcollaboration.org/library.html).

13. Cochrane Database of Systematic Reviews (CDSR; current issue), part of the Cochrane Library.

14. Database of Abstracts of Reviews of Effectiveness (DARE; current issue), part of the Cochrane Library.

15. Epistemonikos (www.epistemonikos.org).

16. UK Clinical Trials Gateway (www.ukctg.nihr.ac.uk/clinical‐trials).

17. ClinicalTrials.gov (clinicaltrials.gov).

18. World Health Organisation International Clinical Trials Registry Platform (WHO ICTRP; www.who.int/ictrp/en).

Searching other resources

We will examine reference lists of reports, reviews and primary studies, and will contact the FAST programme developers, FAST practitioners and independent researchers to identify studies not retrieved by the electronic searches. In addition, we will search the Families and Schools Together website (familiesandschoolstogether.com), What Works (ies.ed.gov/ncee/wwc/FWW), and other government, education, health and social services websites, as well as NGOs (nongovernmental organisations) with an education, child or family remit in which FAST is, or has been, employed. We will also search using Google Scholar.

Selection of studies

Two reviewers will independently review all titles and abstracts to determine all potentially relevant studies. Any citations deemed potentially relevant by at least one reviewer will be retrieved in full text. The same two reviewers will then independently read all retrieved papers of potentially relevant studies to determine whether they satisfy the inclusion criteria (Criteria for considering studies for this review). The reviewers will resolve disagreements by discussion with GM and JV. When the reviewers exclude a retrieved study, they will document the reasons for its exclusion. We will record our decisions in a PRISMA diagram (Moher 2009).

Data extraction and management

For each included study, two review author pairs will independently extract and record all relevant data on a specifically designed and piloted data collection form. The review authors will resolve any disagreements in discussion with a GM or JV. The reviewers will extract the following data.

1. Study characteristics: study author(s), year of publication, journal or source, contact details, study design, study duration, attrition details, language.

2. Child characteristics: age, gender, ethnicity, special educational needs or disability.

3. Parent characteristics: age, gender, ethnicity, educational attainment or qualifications or both, employment status.

4. Family characteristics: family size, marital status, annual income.

5. School characteristics: population served, size, other interventions, location.

6. Outcomes and measures used: details on all primary and secondary outcome measures, including measures used, length of follow‐up, summary data, means and standard deviations.

7. Cost incurred by the intervention.

We will collect information on study design and implementation in a format suited to completion of the 'Risk of bias' tables to appear in the completed review (Higgins 2011a). We will collect raw (unadjusted) results in preference to adjusted results, for reasons of consistency of interpretation across studies and because this choice of analysis appears to be less susceptible to selective reporting bias (for example, it prevents potentially biased selection of covariates for inclusion in the model). This decision may, however, increase the risk of bias that may be attributable to baseline differences, such as those arising from differential dropout.

Assessment of risk of bias in included studies

We will assess the risk of bias of included studies using Cochrane's 'Risk of bias' tool (Higgins 2011b). For each included study, two review author pairs will independently assess the risk of bias within each included study based on the seven domains listed below, with review authors' judgements presented as 'high', 'low' or 'unclear' risk of bias (see Table 1). Where any disagreements occur between the judgement of the authors, they will seek resolution in discussion with the Cochrane Developmental, Psychosocial and Learning Problems Editorial Team.

1. Random sequence generation. We will describe the methods used to generate the allocation sequence in detail, in order to assess whether it was likely to produce comparable groups of participants. The question: was the allocation sequence adequately generated?

2. Allocation concealment. We will describe the methods used to conceal the allocation sequence in detail, in order to determine whether intervention allocation has been concealed before and during the allocation process. The question: was the allocation adequately concealed?

3. Blinding of participants and personnel. Given the nature of the intervention, it is not possible to blind participants or personnel to knowledge of the allocated intervention, and we will examine the extent to which this may have introduced a high risk of bias. The question: was performance biased due to knowledge of the allocated interventions by participants and personnel during the study?

4. Blinding of outcome assessment. We will provide a description of the methods used to blind outcome assessors to knowledge of the allocated intervention to ascertain whether adequate protection of concealment was maintained throughout the study. The question: was knowledge of the allocated intervention adequately prevented during the study?

5. Incomplete outcome data. We will describe the completeness of outcome data for each main outcome, noting reported attrition and exclusions in each intervention group, the reasons for attrition or exclusions, and any reinclusions in analysis employed by the review authors. The question: were incomplete outcome data adequately addressed?

6. Selective outcome reporting. We will examine the comprehensiveness of outcome reporting in relation to published reports or available study protocols to ascertain whether selective outcome reporting was employed. The question: are reports of the study free from selective outcome reporting?

7. Other sources of bias not addressed under the preceding domains. We will examine study protocols in sufficient detail to ascertain whether other sources of bias are present. The question: are reports of the study free from the sources of bias listed in Table 4 below.

Measures of treatment effect

Where possible, we will calculate intervention effects using Cochrane's software: Review Manager 2014.

Unit of analysis issues

Dealing with missing data

Where necessary, we will contact the corresponding authors of included studies to secure any unreported data (e.g. group means and standard deviations, details of dropouts, and reasons for attrition). We will contact other authors as necessary. Where a study reports outcomes only for those participants completing the trial, or only for those who followed the protocol, we will endeavour to obtain the additional information necessary to facilitate analyses according to intention‐to‐treat (ITT) principles. We will describe missing data and attrition/dropout rates for each included study in the 'Risk of bias' tables and discuss the extent to which missing data could affect the results of the review or the conclusions drawn.

Where we are certain that missing data are 'missing at random' and unlikely to be related to the characteristics of the participants or study design, we will analyse the available data ignoring the missing data (Higgins 2011c). Conversely, where there is no reason to believe that data are missing at random — that is, as a result of publication or selective reporting bias — we will work with a statistician to select replacement values using imputed mean values or multiple imputation methods.

Assessment of heterogeneity

We will assess and describe clinical variation across included studies (variability in participants, the FAST programme), and methodological diversity (randomisation, randomisation concealment, blinding of outcomes assessment, losses to follow‐up, etc.). We will describe statistical heterogeneity by computing the I² (Deeks 2011, section 9.5), a quantity which broadly describes the proportion of variation in point estimates that is due to heterogeneity rather than sampling error. In addition, we will use a Chi² test of homogeneity to determine the strength of evidence that heterogeneity is genuine. Inconsistency between studies may be ambiguous and depend upon several factors; therefore the results of the I² test may be roughly interpreted as follows.

1. 0% to 40% might not be important.

2. 30% to 60% may represent moderate heterogeneity.

3. 50% to 90% may represent substantial heterogeneity.

4. 75% to 100% represents considerable heterogeneity.

Assessment of reporting biases

Where 10 or more studies provide data on a particular outcome, we will draw funnel plots (estimated differences in treatment effects against their standard error). Symmetrical funnel plots are associated with low levels of bias. Asymmetric funnel plots may be due to publication bias, but they can also reflect a real relationships between trial size and effect size, such as when larger trials have lower compliance, and compliance is positively related to effect size. If we have reason to think that this is happening, we will look for a possible explanation in clinical variation across studies.

To test directly for publication bias, we will conduct a Sensitivity analysis to compare results from published data with unpublished data and data from other sources.

Data synthesis

We will synthesise the data using RevMan 5 (Review Manager 2014). We will use both a fixed‐effect model and a random‐effects model and compare the two to assess the impact of statistical heterogeneity. Unless contraindicated by the presence of funnel plot asymmetry, we will present the results from the fixed‐effect model, given the focus of the review is on one intervention. Where we encounter serious funnel plot asymmetry, we will assume that neither model is appropriate and present the results of both the fixed‐effect and random‐effects analyses. Where both indicate the presence or absence of an effect, we will assume that we can have some confidence in the results. Where they disagree, we will report this.

We will calculate all overall effects using inverse variance methods. If some included studies report an outcome using dichotomous outcome measures and others use a continuous measure, we will convert the results from the former, from an OR to an SMD, as long as there is reason to assume that the underlying continuous measure approximates a normal or logistic distribution. Where this is not the case we will conduct separate analyses.

Subgroup analysis and investigation of heterogeneity

As the overuse of subgroup analysis is problematic (Deeks 2011), we will only use subgroup analyses to determine a small number of effect modifiers.

We will conduct the following four subgroup analyses.

1. Differences in treatment effect between each of the FAST variants (correlated to ages of child participants) namely:

   1. Baby FAST;

   2. Pre‐K FAST;

   3. Kids' FAST;

   4. Middle School FAST; and

   5. Teen FAST.

2. Programmes evaluated by teams independent of the programme developer, versus those involving the programme developer, as there is evidence to suggest the effect sizes reported in studies involving the programme developer are larger than those conducted entirely independently.

3. Location, exploring the possible impacts of FAST in countries of differing stages of economic development.

4. Ethnicity. Since Moberg 2007 noted that Latino families are 12% more likely to graduate from FAST, and more than twice as likely to attend FASTWORKS than African Americans, we will consider a subgroup analysis of the ethnicity of participants or cultural adaptation of the programme (or both) and implementation of FASTWORKS.

As no family size effects are noted in McDonald 2009, McDonald 2010 and Crozier 2010, despite reported differing average family sizes, we will not include family size in the subgroup analysis.

Sensitivity analysis

We will use sensitivity analyses to explore the impact of studies at high risk of bias on the robustness of the results of the review, restricting the analyses to (a) studies or outcomes with low risk of assessment bias, (b) studies with low risk of attrition bias, and (c) studies with low risk of reporting bias. In addition:

1. where RCTs and quasi‐RCTs are included in a meta‐analysis, we will explore the impact of removing the quasi‐RCT studies;

2. where one or two studies appear to be 'outliers' (have results very different from the remainder), we will examine the impact of excluding these from the meta‐analysis;

3. where the results of a meta‐analysis appear to be heavily dependent on one particular trial, we will repeat the analysis excluding this trial (which may be the largest, or the earliest); and

4. we may examine the effect of different ICCs for cluster‐randomised trials.