Scolaris Content Display Scolaris Content Display

Cochrane Database of Systematic Reviews Protocol - Intervention

Pivotal Response Treatment for autism spectrum disorder (ASD)

Editorial note

This protocol will not be progressed to the review stage due to a lack of progress.

Collapse all Expand all

Abstract

Objectives

This is a protocol for a Cochrane Review (intervention). The objectives are as follows:

To systematically review evidence on the effectiveness of PRT in individuals with ASD. Our specific objectives are:

  1. to assess the effect of PRT compared with treatment as usual or wait‐list control on social communication skills in individuals with ASD based on: 1. direct observation; and 2. parent/caregiver or teacher report (or both);

  2. to assess the effect of PRT compared to treatment as usual or wait‐list control on: 1. a. intelligence; b. restricted and repetitive behaviour; c. internalising and externalising behaviour; d. global clinical improvement; e. the dose‐response of PRT; and f. possible reported adverse effects; and 2. parenting stress;

  3. to examine differential effects of age, gender, IQ measure before treatment, and severity of ASD symptoms on treatment outcomes; and

  4. to examine differential effects of fidelity of PRT implementation, duration of intervention, intensity of intervention, implementation setting and therapist involvement on treatment outcomes.

Background

Description of the condition

Autism spectrum disorder (ASD) is characterised by persistent deficits in social communication and interaction, and by restricted, repetitive patterns of behaviour, interests, or activities (APA 2013). As a neurodevelopmental disorder, ASD emerges at a young age (Ibanez 2014), and the diagnosis is stable throughout life in most individuals (Woolfenden 2012). Both heritability and environmental factors contribute to the aetiology of ASD (Sandin 2014; Sealy 2016). ASD affects boys more often than girls, with a ratio of 5:1 among individuals with average intellectual functioning and a ratio of 2:1 among those with an intellectual disability (Fombonne 2005). A meta‐analysis that focused on ASD symptoms in boys versus girls found that girls showed fewer restricted interests and behaviours and stereotypes than did boys, but only in children older than six years, which suggests that the ASD phenotype differs by age and gender (Van Wijngaarden‐Cremers 2014).

ASD is typically diagnosed using criteria from the Diagnostic and Statistical Manual of Mental Disorders, currently in its fifth edition (DSM‐5; APA 2013). In the previous version of the DSM (DSM‐Fourth Edition‐Text Revision; APA 2000), ASD was separated into four distinct disorders: autistic disorder, Asperger's disorder, childhood disintegrative disorder and pervasive developmental disorder not otherwise specified (PDD‐NOS). Although these subtypes still exist in the current version of the International Statistical Classification of Diseases and Related Health Problems (ICD‐10) (WHO 1992), they have been subsumed into a single diagnosis of ASD in the DSM‐5 (APA 2013), since no valid differences as regards clinical and demographic characteristics, neuropsychological profiles, comorbidity and prognosis have been found (Volkmar 2014; Witwer 2008). In addition, three severity classifications have been included in the DSM‐5, to indicate the level of support that an individual needs in the areas of both social communication and restricted and repetitive behaviors (Weitlauf 2014).

The prevalence of ASD in the general population is estimated to be about 1% (Baird 2006; Elsabbagh 2012). There is heterogeneity in the severity of ASD symptoms, developmental trajectories, and cognitive, language and adaptive functioning (Georgiades 2013). Although ASD can affect individuals with any level of intelligence, a significant proportion of affected individuals have an intellectual disability (varying from around 40% to 70%) (CDC 2012; Fombonne 2006; Yeargin‐Allsopp 2003). Some studies have found the ASD symptom profile to be different in individuals with a lower (non‐verbal) intelligence quotient (IQ) (Bishop 2006; Kjellmer 2012), but other studies have not found evidence in support of this (Charman 2011). Most individuals with ASD (70%) have a comorbid psychiatric disorder, such as anxiety or attention‐deficit hyperactivity disorder (ADHD), with 41% having two or more comorbid conditions (Simonoff 2008).

Although the long‐term outcomes for most children with ASD are poor (Billstedt 2005; Van Heijst 2015), behavioural and social outcomes can vary highly between individuals with ASD. Some individuals improve markedly in social and behavioural functioning, while others experience deterioration in functioning or show a stable course (Levy 2011). Individuals with ASD who have a higher IQ and fewer comorbid conditions are more likely to experience better outcomes in adolescence or early adulthood (Levy 2011). A minority of people with ASD are able to find suitable employment, to develop meaningful relationships and to live independently (Howlin 2004), but the majority rely on family, caregivers and professional services throughout life. Estimated costs due to productivity loss for an individual with ASD can range from about USD 27,000 to USD 32,000 per year in the UK and can be about USD 11,000 in the USA (Buescher 2014). Additionally, costs due to productivity loss for parents of children with ASD can range from about USD 800 to USD 7000 per year in the UK and can be about USD 24,000 per year in the USA (Buescher 2014). Total societal costs for caring for and treating a person with ASD in the USA are estimated to be USD 3.2 million per year (Ganz 2007).

Treatments

Pharmacological treatments target symptoms that may co‐occur with ASD, such as symptoms of ADHD, aggression, irritability, self‐injurious behaviour and anxiety (Myers 2007), because there are currently no treatments for the core symptoms of ASD (Ghosh 2013). Interventions designed to address the core symptoms of ASD are non‐pharmacological interventions, such as communication‐focused interventions, social skill development interventions, sensory motor interventions and milieu therapy (Ospina 2008). These interventions vary considerably in their underlying theoretical frameworks, intensity, modes of delivery and cost‐effectiveness (Seida 2009). Most of the 30 treatment models available for ASD have clearly defined procedures and a detailed curriculum, or both (Odom 2010). The evidence for most of these interventions is weak (Odom 2010), but promising treatment methods are characterised by a focus on the core symptoms of ASD, a manualised approach, systematic measurement of effect (after treatment and follow‐up), monitoring of fidelity of implementation, and being embedded in daily activities in the individual's natural environment (Ospina 2008).

Interventions for ASD based on the principles of applied behaviour analysis (ABA) cover a number of these elements (Kodak 2011; Odom 2010), and a meta‐analysis of the effectiveness of ABA for young children with ASD has reported positive medium‐to‐large effects on intellectual functioning, language development and adaptive behaviour, regardless of the specific ABA method used (Virués‐Ortega 2010). Traditional ABA‐based interventions involve discrete trial training in a structured one‐to‐one situation (Ghezzi 2007). This is highly labour intensive (Smith 2001), and so attention has been paid to how to limit the intensity of ABA‐based treatments for ASD and to implement ABA procedures in the individual's natural environment (Foran 2015).

Description of the intervention

Pivotal Response Treatment (PRT) is an ABA‐based intervention that focuses on the generalisation of learned skills in the child's natural environment (Koegel 2016), and is used for the core symptoms of ASD (Koegel 1999; Koegel 2001; Koegel 2006). It was developed on the basis of the natural language paradigm (NLP) and represents a shift towards naturalistic procedures for language interventions in contrast to imitation and drill procedures outside the natural environment of the individual (Camarata 1996). PRT can be distinguished from other traditional ABA‐based treatments by focusing on 'pivotal' skills in contrast to one individual target skill, using child‐selected material versus adult‐selected material, being embedded in the child's natural environment versus therapy in a structured, intensive one‐to‐one setting, and using different people (e.g. parents, teachers and peers) to provide the intervention in contrast to therapist(s) only (Koegel 1999). PRT focuses on four pivotal (or 'core') areas, namely, motivation for social communication, self‐initiation, responding to multiple cues, and self‐management (Koegel 2001; National Autism Center 2009; National Autism Center 2015). Empathy has been proposed recently as a fifth pivotal area although scientific evidence to support this is limited (Koegel 2013; Koegel 2016). The hypothesis is that the acquisition of skills in these pivotal areas may promote widespread, collateral improvements in other skills.

In PRT, 'learning opportunities' are created according to the principles of ABC (i.e. antecedents, behaviour, consequences), with emphasis on techniques related to the antecedents and consequences of behaviour. For the 'antecedent' domain, three techniques are used to elicit target behaviour, namely: 1. following the child's interests (e.g. by child choice); 2. catching the child's attention (e.g. dividing material); and 3. providing a clear learning opportunity with appropriate help (e.g. the use of 'prompts'). The techniques used for the 'consequences' domain include: 1. consistent reinforcement; 2. use of natural reinforcements (i.e. that is directly and functionally related to a task); and 3. rewarding attempts (Koegel 1999; Koegel 2001; Koegel 2006; Koegel 2014a). As an additional technique, interspersing maintenance (i.e. previously learned) tasks with new tasks provides an appropriate and reliable sequence of learning opportunities. In Appendix 1, we provide two examples of PRT learning opportunities.

Specific goals for PRT vary depending on the developmental level and verbal abilities of the individual, ranging from teaching communicative intent and first words in very young children, to self‐management of social behaviour in adolescents with ASD (Koegel 1999; Koegel 2006). The education of parents, siblings, peers, and teachers is a crucial component of PRT because it improves learning opportunities in the individual’s natural environment (Koegel 2006; Koegel 2014a). Furthermore, training parents will improve family interactions and diminish parenting stress (Koegel 2012). For these reasons, PRT can be considered a comprehensive intervention that is implemented across people, settings and environments, with a view to achieving optimal outcomes in individuals with ASD.

PRT can be implemented at home, in a clinic, in classrooms or in play gardens, and by various implementers (Bozkus Genc 2013). Implementation in the natural environment by parents and other caregivers is a critical component of the intervention (Koegel 2006; Renshaw 2011). Clinicians that are familiar with basic theoretical as well as practical knowledge of ABA are trained in PRT techniques to become a certified PRT therapist. During intervention sessions, PRT therapists adapt the techniques to caregivers, give them opportunities to practice implementing the techniques, provide feedback and, at the same time, monitor the fidelity of treatment implementation (Koegel 2006; Koegel 2014a). Monitoring of the fidelity of treatment implementation, that is, the correct use of PRT techniques by clinicians, caregivers and teachers, is an important aspect in optimising the quality of PRT (Bryson 2007; Koegel 2012). This is often assessed over a 10‐minute period, with the criterion that PRT motivational techniques should be used for 80% of the time (i.e. child attending, clear opportunity to respond, interspersing maintenance and new tasks, using multiple cues, using child choice, contingent reinforcement, natural reinforcement and reinforcing attempts) (Koegel 2006).

How the intervention might work

PRT targets 'pivotal' or core areas of functioning that are impaired in individuals with ASD. For instance, young children with ASD take the initiative less often in social situations because they are not motivated to do so (Koegel 2012). This may be because they have limited skills for focusing on others, such as joint attention and emotion recognition, which might be due to a deficit in the development of theory of mind (Baron‐Cohen 2000; Fletcher‐Watson 2014). Because they are less motivated, young children with ASD have fewer opportunities for using language, making eye contact, taking turns, developing conversational skills, recognising emotions and learning social rules, etc. (Koegel 2012). They also have fewer opportunities for interacting with others, for developing language, for increasing their vocabulary and for learning about the outside world (Koegel 2014b). Increasing motivation to learn in children by using desired objects may benefit the acquisition of academic skills such as reading and calculating (Koegel 2006). Also, the lack of self‐management abilities in older children with ASD may result in their using requests and protests as almost the sole means of social interaction (Wetherby 1984). Thus, targeting pivotal areas impaired in individuals with ASD may lead to collateral improvement in other social and communicational skills.

PRT enables an individual with ASD to learn that social communication can be functional and rewarding (Koegel 2014a; Koegel 2014b). Prompts (i.e. levels of help) are used to help the individual make an appropriate response (Koegel 2012). These are often gradually phased out, going from a high level of help (e.g. tell prompt) to a low level of help (e.g. waiting prompt), as the new skill is acquired (Koegel 2012).

Considering the promising ingredients of PRT, the intervention has the potential to facilitate improvements in different areas of functioning of the individual with ASD and his/her relatives. However, qualitative and quantitative comparisons of the effects of PRT treatment with the effects of other treatments for individuals with ASD are needed to draw valid conclusions.

Why it is important to do this review

Lang 2009 reviewed studies of parents’ ability to implement communication interventions and the effects of PRT on children’s communication, but did not specifically compare PRT with other communication interventions for ASD. Bozkus Genc 2013 evaluated methods of PRT implementation, investigating participant characteristics, settings, which social skills were taught, characteristics of implementers, research model, and social validity; however, they only included information from single‐subject research. Bozkus‐Genc 2016 quantitatively evaluated these single‐subject studies but did not report on the quality of the studies, especially the risk of bias. One previous Cochrane Review focused on a parent‐mediated intervention for ASD but investigated young children only and did not specifically study the effects of PRT (Oono 2013).

A systematic review and meta‐analysis of the efficacy of PRT in children with ASD that adheres to Cochrane guidelines is needed for several reasons. First, the use of different methods in PRT studies makes it impossible to compare the effects of PRT with the effects of other ASD interventions. However, the use of the GRADE system to evaluate intervention studies in ASD can improve transparency and comparability across studies (Guyatt 2008). Second, it is important to present the results of randomised trials of PRT separately from those of non‐randomised studies, as the latter potentially have a higher risk of bias (Higgins 2017). In earlier reviews, evidence from randomised controlled trials (RCTs) was not evaluated (Cadogan 2015), or effects were only reviewed descriptively (Verschuur 2014). Third, no quantitative systematic review has compared studies using group designs to evaluate the effect of PRT and usual care on outcomes in individuals with ASD and their relatives. In addition to addressing these gaps in the evidence base, it is important to explore the role of individual characteristics and environmental factors as potential moderators of the impact of PRT, and the impact of 'dosage' factors such as frequency, intensity and implementation setting.

Objectives

To systematically review evidence on the effectiveness of PRT in individuals with ASD. Our specific objectives are:

  1. to assess the effect of PRT compared with treatment as usual or wait‐list control on social communication skills in individuals with ASD based on: 1. direct observation; and 2. parent/caregiver or teacher report (or both);

  2. to assess the effect of PRT compared to treatment as usual or wait‐list control on: 1. a. intelligence; b. restricted and repetitive behaviour; c. internalising and externalising behaviour; d. global clinical improvement; e. the dose‐response of PRT; and f. possible reported adverse effects; and 2. parenting stress;

  3. to examine differential effects of age, gender, IQ measure before treatment, and severity of ASD symptoms on treatment outcomes; and

  4. to examine differential effects of fidelity of PRT implementation, duration of intervention, intensity of intervention, implementation setting and therapist involvement on treatment outcomes.

Methods

Criteria for considering studies for this review

Types of studies

Randomised controlled trials (RCTs), quasi‐RCTs (trials using a quasi‐random method of allocation; for example, based on date of admission), non‐randomised studies (non‐RCTs) and controlled before‐and‐after studies (CBA) in which a PRT treatment group is compared with a control group.

Types of participants

Any person with a primary clinical diagnosis of ASD based on any version of the DSM (APA 1987; APA 1994; APA 2000; APA 2013) or the ICD‐10 (WHO 1992), with or without comorbid medical or psychiatric disorders, defined according to diagnostic terms relevant to ASD, such as autistic disorder, Asperger's disorder, Asperger's syndrome, PDD‐NOS, childhood autism, infantile autism, and atypical autism.

Types of interventions

Studies that explicitly state the use of PRT techniques in the treatment group compared with treatment as usual or wait‐list control (waiting for treatment).

Types of outcome measures

We will not exclude studies on the basis of the outcomes measured. The primary and secondary outcomes listed below are accompanied by examples of measures that are often used to evaluate treatment effects in individuals with ASD.

Primary outcomes

  1. Social communication skills based on direct observations, including observational measures such as the Social and Communication domains of the Autism Diagnostic Observation Scale (ADOS; Lord 2000), or observed expressive communication such as the Production Scale of the Reynell Developmental Language Scale (RDLS; Reynell 1985), the observed mean length of utterance (MLU), and observed functional verbalisations

  2. Social communication skills as measured by parent/caregiver or teacher report, or both, using scales such as the Social Responsiveness Scale (SRS; Constantino 2005), the Social Communication Questionnaire (SCQ; Rutter 2003), the Children's Communication Checklist (CCC; Bishop 1998) or the Socialization and Communication domains of the Vineland Adaptive Behaviour Scales (VABS; Sparrow 2005)

Secondary outcomes

  1. Intelligence (IQ), as measured with IQ scales such as the Wechsler Intelligence Scale for Children (WISC; Wechsler 1991), or the Mullen Scales of Early Learning (MSEL; Mullen 1995)

  2. Restricted and repetitive behaviour, as measured with scales such as the Restricted and Repetitive domain of the ADOS (Lord 2000) or the Repetitive Behaviour Scale (RBS; Bodfish 1999)

  3. Internalising and externalising behaviour, as measured with scales such as the Child Behaviour Checklist (CBCL; Achenbach 1991) or the Strengths and Difficulties Questionnaire (SDQ; Goodman 1997)

  4. Global improvement, as measured with scales such as the Clinical Global Impression ‐ Improvement (CGI‐I) scale (Guy 1976)

  5. Possible reported adverse effects, including reports of exhaustion

  6. Parenting stress, as measured with parent‐report scales such as the Parenting Stress Index (Abidin 1995), or the Parental Stress Scale (Berry 1995)

We will analyse patient‐, parent‐, and teacher‐reported information and data separately.

Search methods for identification of studies

We will not exclude studies on the basis of language, publication status or publication date.

Electronic searches

We will search the following electronic databases and trials registers.

  1. Cochrane Central Register of Controlled Trials (CENTRAL, current issue) in the Cochrane Library, which includes the Cochrane Developmental, Psychosocial and Learning Problems Specialised Register.

  2. MEDLINE Ovid (1946 onwards).

  3. MEDLINE In‐Process and Other Non‐Indexed Citations Ovid (current issue).

  4. MEDLINE Epub Ahead of Print Ovid (current issue).

  5. Embase Ovid (1974 onwards).

  6. CINAHL EBSCOhost (Cumulative Index to Nursing and Allied Health Literature; 1937 onwards).

  7. PsycINFO Ovid (1806 onwards).

  8. ERIC Ovid (Education Resources Information Center; 1965 onwards).

  9. Linguistics and Language Behaviour Abstracts ProQuest (1973 onwards).

  10. Science Citation Index ‐ EXPANDED Web of Science (SCI‐EXPANDED; 1945 onwards).

  11. Social Sciences Citation Index Web of Science (SSCI; 1956 onwards).

  12. Arts & Humanities Citation Index Web of Science (A&HCI; 1975 onwards).

  13. Emerging Sources Citation Index Web of Science (ESCI; 2015 onwards).

  14. Sociological Abstracts ProQuest (1952 onwards).

  15. Epistemonikos (www.epistemonikos.org/nl/advanced_search).

  16. National Autistic Society Library Catalogue (library.autism.org.uk/Portal/Default/en-GB/Search/AdvancedSearch).

  17. Networked Digital Libary of Theses and Dissertations (NDLTD; search.ndltd.org; 1990 onwards).

  18. ClinicalTrials.gov (www.clinicaltrials.gov).

  19. WorldCat OCLC (www.worldcat.org/default.jsp).

  20. World Health Organization International Clinical Trials Registry Platform (ICTRP; apps.who.int/trialsearch).

The search strategy for Ovid MEDLINE is reported in Appendix 2. We will adapt this search strategy for searches in other databases as appropriate.

Searching other resources

We will handsearch full‐text reports published in the scientific journals listed below, since PRT or NLP may be investigated in a study but not be mentioned in the title, abstract, or keywords.

  1. Autism (July 1997 onwards).

  2. Autism Research (2008 onwards).

  3. Developmental Disabilities Research Reviews (1995 onwards).

  4. Focus on Autism and Other Developmental Disabilities (June 1986 onwards).

  5. Good Autism Practice (May 2007 onwards).

  6. Journal of Applied Research in Intellectual Disabilities (JARID; 1988 onwards).

  7. Journal of Autism and Developmental Disorders (1971 onwards).

  8. Journal of Applied Behavior Analysis (1968 onwards).

  9. Conference abstracts of the Association for Behavior Analysis International (ABAI), including annual conventions, international conferences, autism conferences, other conferences (2001 onwards).

  10. European Journal of Behavior Analysis (2000 onwards).

  11. Conference abstracts of the European Association for Behavior Analysis (EABA; 2003 onwards).

For identifying other, possibly relevant studies, we will handsearch the following two websites.

  1. Research Autism (researchautism.net/interventions/88/pivotal-response-training-and-autism/Studies%20and%20Trials).

  2. Koegel Autism Centre (education.ucsb.edu/autism/pivotal-response-treatment).

In addition, we will ask experts in the field if they know of other studies not identified by the search. We will also search the reference lists of included studies, as well as the reference lists of earlier systematic reviews.

Data collection and analysis

Selection of studies

Two review authors (IS and IO) will independently assess the titles and abstracts yielded by the searches for relevance against the inclusion criteria (see Criteria for considering studies for this review). They will then retrieve and examine the full texts of potentially relevant reports, grouping together multiple reports related to the same study. If additional information on methodology is needed to determine whether the study should be included, the review authors will contact the corresponding authors of the original article. We will resolve any possible disagreements about eligibility by discussion and, if necessary, by consulting another review author (MVD‐B). The final decision about study inclusion will be made by all review authors.

Studies will be blinded before assessment of their eligibility to lessen the possibility of selection bias. We will assess studies for inclusion without information regarding authors, affiliations, year and journal of publication; we will only export this information from Endnote 2014. Before examining the full text of potentially relevant reports, an independent research assistant (Manon de Korte, MSc.), who will not be involved in study selection, will remove information regarding authors, affiliations, year and journal of publication.

Data extraction and management

Using a pre‐designed form, two review authors (IS and IO) will independently extract study data (see Appendix 3 for a description of the data to be extracted) and compare both sets of extracted data for accuracy, resolving any discrepancies by discussion. One review author (IS) will enter the extracted data into Review Manager 5 (RevMan 5) (RevMan 2014), and a second review author (IO) will check it.

Assessment of risk of bias in included studies

Two review authors (IS and IO) will independently assess the risk of bias of the included studies, using Cochrane's tool for assessing risk of bias for randomised trials (Higgins 2017). Specifically, for each included study, the review authors will assess the following sources of bias and assign judgements of low, high, or unclear risk of bias: random sequence generation; allocation concealment; blinding of participants and personnel; blinding of outcome assessment; incomplete outcome data; selective reporting; and other possible sources of bias (Higgins 2017). See Appendix 4 for a description of each possible source. For cluster‐randomised designs, we will assess the risk of bias in the following categories: 1. recruitment bias; 2. baseline imbalance; 3. loss of clusters; 4. incorrect analysis; and 5. comparability with individually randomised trials (Higgins 2011).

For assessing the risk of bias in non‐RCTs and CBAs we will use the 'Risk of Bias in Non‐Randomised Studies ‐ of Interventions' (ROBINS‐I) assessment tool (Sterne 2016). Potentially important confounding domains for these studies are differences between groups in severity of ASD symptoms, (verbal) IQ, presence of comorbid psychiatric disorders, and presence of one or more pharmacological or non‐pharmacological co‐intervention(s) besides PRT, treatment as usual or wait‐list control. Bias domains included in ROBINS‐I are: 1. bias due to confounding; 2. bias in selection of participants into the study; 3. bias in classification of interventions; 4. bias due to deviations from intended interventions; 5. bias due to missing data; 6. bias in measurement of outcomes; and 7. bias in selection of the reported result. Two review authors will assign a judgement of 'low risk of bias', 'moderate risk of bias', 'serious risk of bias', 'critical risk of bias', or 'no information' to each of the seven domains. See Appendix 5 for a detailed description of risk of bias judgements for each domain.

The two review authors will resolve disagreements by discussion and, if necessary, by consulting another review author (MVD‐B). We will present the 'Risk of bias' assessment and its justification in a table.

Measures of treatment effect

Dichotomous data

When dichotomous outcomes are reported (e.g. clinically significant response to a treatment versus no clinically significant response), we will use the risk ratio (RR) with a 95% confidence interval (CI). If study outcomes are not reported within a 2 x 2 table, we will contact the investigators for this information. To calculate the RR, we will use the algorithm described in the Cochrane Handbook for Systematic Reviews of Interventions (Deeks 2017). We have chosen the RR rather than the odds ratio (OR) because the OR can be misleading when event rates (i.e. rates of clinically significant improvement) are high, which is common in clinical trials and systematic reviews (Deeks 1998).

Continuous data

For those studies that report continuous outcomes assessed with the same instruments, we will use the mean difference (MD) as the summary statistic in the meta‐analyses and present these with a 95% CI. If continuous outcomes have been assessed with different instruments across studies, we will use the standardised mean difference (SMD) with a 95% CI for between‐subject designs, using the algorithm described in the Cochrane Handbook for Systematic Reviews of Interventions (Deeks 2017).

Since the SMD method does not correct for differences in the direction of the scale, we will multiply mean values by –1 if outcomes are presented such that a higher score means less severe problem severity. If there is insufficient information to enable calculation of the SMD, we will describe the study in the systematic review but will not include it in the meta‐analysis.

Unit of analysis issues

Cluster‐randomised designs

Studies with a cluster‐randomised design should report an intracluster correlation coefficient (ICC). If the ICC is not reported, we will obtain external estimates from similar studies (Higgins 2011), and then correct the analysis by reducing the size of each trial to its 'effective sample size'. We will enter the data into RevMan 5 (RevMan 2014), and use the generic inverse‐variance method for analysis. When studies report outcomes from individuals of the same family (e.g. in multilevel models), we will include effect estimates from analyses that properly account for the cluster design, if this information is available. However, since we expect that this information will not be available for each study, we can only account for the impact of including siblings on the effect estimates for studies that report this information properly.

Cross‐over designs

We will screen studies with a cross‐over design for information about the evaluation of the carry‐over or period effects. If neither carry‐over nor period effects are thought to be a problem, we will calculate the effect estimate of the difference measure in the meta‐analysis, using the generic inverse‐variance method in RevMan 5 (RevMan 2014). However, if carry‐over or period effects are present or insufficient information about these effects is available, we will base the analysis on the data from the first period only.

Studies with multiple intervention groups

If a study describes multiple treatment groups, we will combine all relevant treatment groups (i.e. using PRT techniques) into a single treatment group and combine all relevant control groups into a single control group, as recommended by Higgins 2011. The effect estimates that are used for both dichotomous and continuous data from between‐subject designs are described in earlier sections (Measures of treatment effect).

Controlled before‐and‐after studies (CBA)

We will quantitatively combine the data of CBA studies provided that there is little bias in the studies and that the studies are relatively homogeneous (Reeves 2011). We will perform a meta‐analysis of adjusted effect estimates as an inverse‐variance weighted average, using the generic inverse‐variance outcome type in RevMan 5 (RevMan 2014). For studies that report multiple estimates of effect, we will use the estimate from the model that adjusted for most covariates.

Dealing with missing data

If study details or numerical data are missing, we will contact the corresponding author to ask them to supply the missing information. If this fails and we do not obtain sufficient summary data, we will use only the data reported in the original article. We will include the study in the review and report that study data are missing (Higgins 2011). We will present these data in a 'Risk of bias' table. We will only analyse available data because using imputation to deal with missing data artificially inflates the precision of the effect estimate (Higgins 2011).

Assessment of heterogeneity

We will assess clinical heterogeneity by comparing participant, intervention and outcome characteristics across studies, and methodological heterogeneity by comparing differences in methods across studies, such as allocation procedures, blinding and measurement of outcomes. To assess statistical heterogeneity, we will perform the Chi2 test using RevMan 5 (RevMan 2014). A low P value (P < 0.10) provides evidence of heterogeneity of intervention effects. Additionally, we will estimate the amount of variation (tau2), and we will calculate the I2 statistic (Higgins 2003) to determine the percentage of variability in the effect estimates due to heterogeneity rather than sampling error: an I2 of 0% to 40% indicates that heterogeneity is not present or is not important, 30% to 60% that there is moderate heterogeneity, 50% to 90% that there is substantial heterogeneity, and 75% to 100% that there is considerable heterogeneity (Deeks 2017). If heterogeneity is present, we will investigate potential causes of heterogeneity by means of a subgroup analysis, provided that there are enough studies (see Subgroup analysis and investigation of heterogeneity).

Assessment of reporting biases

If at least 10 studies are included in the meta‐analysis, we will draw funnel plots (i.e. estimated differences in treatment effects against their standard error) and test for funnel plot asymmetry, to detect whether reporting bias is present. However, asymmetry in funnel plots can be caused by other sources, such as differences in methodological quality and true heterogeneity in intervention effects (Sterne 2017). We will also perform the Egger 1997 test for funnel plot asymmetry, as proposed by Sterne 2017.

Data synthesis

We will carry out a pair‐wise meta‐analysis if at least two relevant studies are available and if there is no substantial clinical or methodological heterogeneity (see Assessment of heterogeneity). As we expect heterogeneity in the clinical features of people with ASD, we will use the random‐effects model (DerSimonian 1986) for meta‐analysis (inverse variance approach in RevMan 5 (RevMan 2014)). By means of subgroup analysis, we will conduct separate meta‐analyses for the randomised and non‐randomised study designs. The results will be displayed in a forest plot (Reeves 2011).

If meta‐analysis is not appropriate (e.g. because of too few studies or considerable heterogeneity across studies), we will present a narrative description of the results.

We will prepare a 'Summary of findings' table in RevMan 5 (RevMan 2014), for the main comparison of the following outcomes:

  1. social communication skills based on direct observation; and

  2. social communication skills based on parent/caregiver or teacher report, or both.

We will provide information about the population, setting, intervention, and comparison in the table headings. For each outcome, we will describe (per column) the information listed below.

  1. Illustrative comparative risks.

  2. Number of participants and studies.

  3. Quality of evidence, using the GRADE approach for assessment (high, moderate, low, very low) (Guyatt 2008). Factors that may decrease the quality of evidence are: 1. limitations in the design and implementation (e.g. lack of allocation concealment, lack of blinding, loss to follow‐up); 2. indirectness of evidence (e.g. to what extent PRT and treatment as usual or wait‐list control are compared indirectly); 3. unexplained heterogeneity or inconsistency of results (e.g. widely differing effect estimates); 4. imprecision of results (e.g. few participants and wide CIs); and 5. high probability of publication bias (e.g. failing to report studies that show no effect or even worsening). Two independent review authors (IS and IO) will assess the quality of evidence. If there is disagreement, they will discuss their findings to try to reach consensus.

  4. Additional comments or differences from the standard methods.

Subgroup analysis and investigation of heterogeneity

We will explore possible causes of heterogeneity in study results by subgroup analysis, using meta‐regression, if there are at least 10 studies per outcome. Exploratory subgroup analysis will involve the following factors.

  1. Participant characteristics:

    1. age (pre‐school (under 6 years of age) versus school‐age (6 to 12 years of age) versus adolescents (over 12 years of age));

    2. gender (male versus female);

    3. IQ (below average IQ (less than 85) versus average IQ (85 to 115) versus above average IQ (more than 115)); and

    4. severity of ASD symptoms based on scores on standardised tests such as the ADOS (Lord 2000) and ADI‐R (Lord 1994) (low versus moderate versus high).

  2. Treatment characteristics:

    1. reported fidelity of implementation of PRT techniques (insufficient (less than 80%) versus sufficient (80% or more));

    2. duration of intervention;

    3. intensity or hours per week of intervention;

    4. implementation setting (at a clinic, at home, at school or at a community setting);

    5. therapist involvement (direct training versus indirect training (via parents, teachers, or peers) versus both direct and indirect training of individuals with ASD); and

    6. type of treatment used in the control group (ABA‐based versus non‐ABA‐based versus not known).

Sensitivity analysis

We will carry out sensitivity analyses to evaluate the robustness of the pooled effect sizes across the following components of methodological quality.

  1. Lack of blinding. We will re‐analyse data, excluding data from studies with a high risk of blinding bias (see Assessment of risk of bias in included studies). High risk of blinding bias is defined as a high or unclear risk of bias in both the blinding of participants and personnel, and blinding of outcome assessments, given that evidence supporting psychological interventions can be strongly influenced by these study characteristics (Sonuga‐Barke 2013).

  2. Incomplete outcome reporting. We will re‐analyse data, excluding data from studies with a high risk of bias due to incomplete outcome reporting.

  3. Dropout rate. We will re‐analyse data, excluding data from studies with a high dropout rate (30% or higher) or different distribution of dropout between groups within the study, or both.

Additionally, we will perform sensitivity analyses to evaluate the robustness of the pooled effect sizes across the following two components related to medication use.

  1. Lack of reporting on medication status. We will re‐analyse data, excluding data from studies that do not report on patients' medication status.

  2. Use of medication. We will re‐analyse data, excluding data from studies in which patients use medication.