Types of studies

We included relevant randomised controlled trials. Where trials are described as ‘double‐blind’ but are only implied as being randomised, we included these trials in a sensitivity analysis. If there were no substantive differences within primary outcomes (see Types of outcome measures) when these ‘implied randomisation’ studies were added, then we included these in the final analysis. If there were substantive differences, we only used clearly randomised trials and described the results of the sensitivity analysis in the text. We excluded quasi‐randomised studies such as those allocating by using alternate days of the week.

Types of participants

People with schizophrenia and other types of schizophrenia‐like psychoses (e.g. schizophreniform and schizoaffective disorders), irrespective of the diagnostic criteria used. There is no clear evidence that the schizophrenia‐like psychoses are caused by fundamentally different disease processes or require different treatment approaches (Carpenter 1994).

Types of interventions

1. Sulpiride in combination with any other antipsychotic treatment: any dose of oral administration.
2. Placebo (or no intervention) in combination with any other antipsychotic treatment: any dose of oral administration.

We predefined that ‘other antipsychotic treatment’ be in two categories. We will label all studies so that the drug being augmented is clear. The studies using a second‐line typical or atypical antipsychotic (such as clozapine) will be kept in separate subgroups for the primary outcomes.

Types of outcome measures

All outcomes were grouped by time ‐ short term (up to 12 weeks), medium term (13 to 26 weeks) and long term (over 26 weeks).

Electronic searches

1. Cochrane Schizophrenia Group Trials Register

The Cochrane Schizophrenia Group Trials Register was searched using the phrase:

[(ability * or championyl* or coolspan* or col‐sulpir* or digton* or dixibon* or dobren* or do?matil* or drominetas* or eglonyl* or equilid* or eusulpid* or guastil* or isnamid* or kapirid* or lavodina* or leboprid* or lusedan* or miradol* or mirbanil* or misulvan* or neuromyfar* or normum* or omperan* or psicocen* or quiridil* or sato * or sernevin* or sicofrenol* or sulp?ride* or sulpisedan* or suprium* or sursumid* or tepavil* or tonofit* or ulpir* or vipral*) in title, abstract and index fields in REFERENCE) OR (sulp?rid* in interventions field in STUDY)]

This register is compiled by systematic searches of major databases, hand searches and conference proceedings (see Group Module). The Cochrane Schizophrenia Group Trials Register is maintained on Meerkat 1.5. This version of Meerkat stores references as studies. When an individual reference is selected through a search, all references which have been identified as the same study are also selected.

2. Details of previous electronic searches
For details of the searches used in previous versions please see Appendix 1.

Searching other resources

1. Reference searching
We also searched reference lists of included studies for additional relevant trials.

2. Personal contact
We contacted the first author of each included study and known experts who had published reviews in the field for information regarding unpublished trials and extra data on the published trials.

3. Drug company
The manufacturers of sulpiride (Lorex Synthelabo Ltd, Bristol‐Myers Pharmaceuticals, Pharmacia and Upjohn) were contacted to provide relevant published and unpublished data.

Selection of studies

Two review authors (JW and IMO) independently inspected all study citations identified by the searches and full reports of the studies of agreed relevance were obtained. Where disputes arose, we acquired the full report for more detailed scrutiny. These articles were then inspected, independently, by two authors to assess their relevance to this review. Again, where disagreement occurred attempts were made to resolve this through discussion; if doubt still remained we added these trials to the list of those awaiting assessment pending acquisition of further information.

Data extraction and management

1. Data extraction
We independently extracted data from included studies. Again, any disagreement was discussed, decisions documented and, if necessary, authors of studies were contacted for clarification. When this was not possible and further information was necessary to resolve the dilemma, we did not enter data and added the trial to the list of those awaiting assessment.

2. Management
We extracted the data onto standard, simple forms. Where possible, data were entered into RevMan in such a way that the area to the left of the ‘line of no effect’ indicates a ‘favourable’ outcome for sulpiride augmentation. Where this was not possible (e.g. scales that calculate higher scores = improvement), the graphs in RevMan analyses were labelled accordingly so that the direction of effects were clear.

3. Scale‐derived data
3.1 Valid scales
A wide range of instruments are available to measure outcomes in mental health studies. These instruments vary in quality and many are not validated, or are even ad hoc. It is accepted generally that measuring instruments should have the properties of reliability (the extent to which a test effectively measures anything at all) and validity (the extent to which a test measures that which it is supposed to measure) (Rust 1989). Unpublished scales are known to be subject to bias in trials of treatments for schizophrenia (Marshall 2000). Therefore continuous data from rating scales were included only if the measuring instrument had been described in a peer reviewed journal. In addition, the following minimum standards for instruments were set: the instrument should either be (a) a self report or (b) completed by an independent rater or relative (not the therapist) and (c) the instrument should be a global assessment of an area of functioning.

3.2 Binary outcomes from scale data
Where possible, efforts were made to convert outcome measures to binary data. This can be done by identifying cut‐off points on rating scales and dividing participants accordingly into ‘clinically improved’ or ‘not clinically improved’. It was generally assumed that if there had been a 50% reduction in a scale‐derived score such as the Brief Psychiatric Rating Scale (BPRS, Overall 1962) or the Positive and Negative Syndrome Scale (PANSS, Kay 1986), this could be considered as a clinically significant response (Leucht 2005a, Leucht 2005b). It was recognised that for many people, especially those with chronic or severe illness, a less rigorous definition of important improvement (e.g. 25% on the BPRS) would be equally valid. If individual patient data were available, the 50% cut‐off was used for the definition in the case of non‐chronically ill people and 25% for those with chronic illness. If data based on these thresholds were not available, we used the primary cut‐off presented by the original authors.

Assessment of risk of bias in included studies

Again working independently, reviewers assessed risk of bias using the tool described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2008). This tool encourages consideration of how the sequence was generated, how allocation was concealed, the integrity of blinding at outcome, the completeness of outcome data, selective reporting and other biases. We would not have included studies where sequence generation was at high risk of bias or where allocation was clearly not concealed. If disputes arose as to which category a trial has to be allocated, again, resolution was made by discussion, after working with a third reviewer (BS).

Measures of treatment effect

1. Binary data
For binary outcomes we calculated a standard estimation of the fixed‐effect risk ratio (RR) and its 95% confidence interval (CI). For statistically significant results we calculated the number needed to treat/harm statistic (NNT/H), and its 95% confidence interval (CI) using Visual Rx (http://www.nntonline.net/) taking account of the event rate in the control group. It has been shown that RR is more intuitive (Boissel 1999) than odds ratios and that odds ratios tend to be interpreted as RR by clinicians (Deeks 2000). This misinterpretation then leads to an overestimate of the impression of the effect.

2. Continuous data
2.1 Summary statistic
For continuous outcomes we estimated a weighted mean difference (WMD) between groups. We did not calculate effect size measures.

2.3 Endpoint versus change data
Where both final endpoint data and change data were available for the same outcome category, only final endpoint data were presented. We acknowledge that by doing this much of the published change data may be excluded, but argue that endpoint data are more clinically relevant and that if change data were to be presented along with endpoint data it would be given undeserved equal prominence. Where studies reported only change data we contacted authors for endpoint figures but if endpoint data were unavailable, we reported change data.

2.3 Skewed data
Continuous data on clinical and social outcomes are often not normally distributed. To avoid the pitfall of applying parametric tests to non‐parametric data, we applied the following standards to all data before inclusion: (a) standard deviations and means are reported in the paper or obtainable from the authors; (b) when a scale starts from the finite number zero, the standard deviation, when multiplied by two, is less than the mean (as otherwise the mean is unlikely to be an appropriate measure of the centre of the distribution, (Altman 1996); (c) if a scale starts from a positive value (such as PANSS which can have values from 30 to 210) the calculation described above will be modified to take the scale starting point into account. In these cases skew is present if 2SD>(S‐Smin), where S is the mean score and S min is the minimum score. Endpoint scores on scales often have a finite start and end point and these rules can be applied. When continuous data are presented on a scale which includes a possibility of negative values (such as change data), it is difficult to tell whether data are skewed or not. Skewed data from studies of less than 200 participants were entered in additional tables rather than into an analysis. Skewed data pose less of a problem when looking at means if the sample size is large and were entered into syntheses.

Unit of analysis issues

1. Cluster trials
Studies increasingly employ cluster randomisation (such as randomisation by clinician or practice) but analysis and pooling of clustered data poses problems. Authors often fail to account for intraclass correlation in clustered studies, leading to a unit of analysis error (Divine 1992) whereby p values are spuriously low, confidence intervals unduly narrow and statistical significance overestimated. This can cause Type I errors (Bland 1997, Gulliford 1999).

Where clustering was not accounted for in primary studies, we presented the data in a table, with a (*) symbol to indicate the presence of a probable unit of analysis error. In subsequent versions of this review we will seek to contact first authors of studies to obtain intraclass correlation coefficients of their clustered data and to adjust for this using accepted methods (Gulliford 1999). Where clustering has been incorporated into the analysis of primary studies, we will also present these data as if from a non‐cluster randomised study, but adjusted for the clustering effect.

We have sought statistical advice and have been advised that the binary data as presented in a report should be divided by a design effect. This is calculated using the mean number of participants per cluster (m) and the intraclass correlation coefficient (ICC) [Design effect=1+(m‐1)*ICC] (Donner 2002). If the ICC was not reported it was assumed to be 0.1 (Ukoumunne 1999). If cluster studies had been appropriately analysed taking into account intraclass correlation coefficients and relevant data documented in the report, we synthesised these with other studies using the generic inverse variance technique.

2. Cross‐over trials
A major concern of cross‐over trials is the carry‐over effect. It occurs if an effect (e.g. pharmacological, physiological or psychological) of the treatment in the first phase is carried over to the second phase. As a consequence on entry to the second phase the participants can differ systematically from their initial state despite a wash‐out phase. For the same reason cross‐over trials are not appropriate if the condition of interest is unstable (Elbourne 2002). As both effects are very likely in schizophrenia, we will only use data of the first phase of cross‐over studies.

3. Studies with multiple treatment groups
Where a study involved more than two treatment arms, if relevant, the additional treatment arms were presented in comparisons. Where the additional treatment arms were not relevant, these data were not reproduced.

Dealing with missing data

1. Overall loss of credibility
At some degree of loss to follow‐up data must lose credibility (Xia 2007). We are forced to make a judgment where this is for the trials likely to be included in this review. Should more than 40% of data be unaccounted for by eight weeks we did not reproduce these data or use them within analyses.

2. Binary
Where attrition for a binary outcome is between 0 and 40%, and outcomes of these people are described, we included these data as reported. Where the outcomes of such people were not clearly described, we assumed the worst primary outcome, and rates of adverse effects similar to those who did continue to have their data recorded.

3. Continuous
In the case where attrition for a continuous outcome is between 0 and 40% and completer‐only data were reported, we have reproduced these.

Assessment of heterogeneity

1. Clinical heterogeneity
We considered all included studies without any comparison to judge clinical heterogeneity.

2. Statistical
2.1 Visual inspection
We visually inspected graphs to investigate the possibility of statistical heterogeneity.

2.2 Employing the I‐squared statistic
This provided an estimate of the percentage of inconsistency thought to be due to chance. I‐squared estimate greater than or equal to 50% was interpreted as evidence of high levels of heterogeneity when accompanied by a p value of <0.05 (Higgins 2003).

Assessment of reporting biases

Reporting biases arise when the dissemination of research findings is influenced by the nature and direction of results. These are described in section 10.1 of the Cochrane Handbook (Higgins 2008).We are aware that funnel plots may be useful in investigating reporting biases but are of limited power to detect small‐study effects (Egger 1997). We did not use funnel plots for outcomes where there were ten or fewer studies, or where all studies were of similar sizes. In other cases, where funnel plots were possible, we sought statistical advice in their interpretation.

Data synthesis

Where possible we employed a fixed‐effect model for analyses. We understand that there is no closed argument for preference for use of fixed or random‐effects models. The random‐effects method incorporates an assumption that the different studies are estimating different, yet related, intervention effects. This does seem true to us, however, random‐effects does put added weight onto the smaller of the studies ‐ those trials that are most vulnerable to bias. For this reason we favour using the fixed‐effect model.

Subgroup analysis and investigation of heterogeneity

1. Subgroup analysis
If possible, the groups of people with schizophrenia will be divided into two ‐ those that were designated by the trialists to have illnesses that were resistant to treatment (any definition) and the studies in which people were not stipulated to have treatment resistant illnesses. We propose to summate all data together but to present these subgroups separately for the primary outcomes.

2. Investigation of heterogeneity
If data are clearly heterogeneous we checked that data are correctly extracted and entered and that we had made no unit of analysis errors. If the high levels of heterogeneity remained we did not undertake a meta‐analysis at this point for if there is considerable variation in results, and particularly if there is inconsistency in the direction of effect, it may be misleading to quote an average value for the intervention effect. We would have wanted to explore heterogeneity. We pre‐specify no characteristics of studies that may be associated with heterogeneity except quality of trial method. If no clear association could be shown by sorting studies by quality of methods a random‐effects meta‐analysis was performed. Should another characteristic of the studies be highlighted by the investigation of heterogeneity, perhaps some clinical heterogeneity not hitherto predicted but plausible causes of heterogeneity these post hoc reasons will be discussed and the data analysed and presented. However, should the heterogeneity be substantially unaffected by use of random‐effects meta‐analysis and no other reasons for the heterogeneity be clear, the final data were presented without a meta‐analysis.

Sensitivity analysis

Trials comparing sulpiride plus clozapine with clozapine alone were taken out to investigate any significant effect of sulpiride augmentation on clozapine monotherapy. As there is concern regarding quality of trials from China (Wu 2006), a sensitivity analysis was performed to investigate whether the findings of these trials substantially differed from other trials.