Types of studies

In this review we will include RCTs only. We will exclude trials in which only one eye was treated with topical mast cell stabiliser and was compared with the other eye, because these studies cannot be used for the analysis of systemic outcomes, such as adverse events or quality of life. Additionally, the systemic absorption of the topical drugs may have an effect on the contralateral eye. If these designs are identified in the review, we will perform a sensitivity analysis in order to explore the impact of their inclusion, (see 'Sensitivity analysis'). 

Types of participants

We will include trials with participants of any age who have a clinical diagnosis of SAC or PAC of any duration. Any clinical criteria for the diagnosis of AC will be admitted. We will not include studies of patients with other types of AC, such as atopic keratoconjunctivitis, vernal keratoconjunctivitis, or giant papillary conjunctivitis in the review.

Types of interventions

We will consider studies using topical mast cell stabilisers combined with other treatments as eligible only if the specific effects of the topical mast cell stabilisers can be addressed, that is, the use of topical mast cell stabilisers is the only difference between the intervention and control arms. This review will include trials in which a topical mast cell stabiliser, alone or combined with topical steroids, topical cyclosporine A, or topical vasoconstrictors, has been compared with no treatment, placebo, or any other intervention (such as steroids, cyclosporine A or vasoconstrictors). This review will not include studies comparing different concentrations or routes of administration of mast cell stabilisers.

Types of outcome measures

Electronic searches

We will search the Cochrane Central Register of Controlled Trials (CENTRAL) (which contains the Cochrane Eyes and Vision Group Trials Register) (The Cochrane Library), Ovid MEDLINE, Ovid MEDLINE In‐Process and Other Non‐Indexed Citations, Ovid MEDLINE Daily, Ovid OLDMEDLINE, EMBASE, Latin American and Caribbean Health Sciences Literature Database (LILACS), Cumulative Index to Nursing and Allied Health Literature (CINAHL), OpenGrey (www.opengrey.eu), the metaRegister of Controlled Trials (mRCT) (www.controlled‐trials.com), ClinicalTrials.gov (www.clinicaltrials.gov), the International Clinical Trials Registry Platform (ICTRP) (www.who.int/ictrp/search/en/), the IFPMA Clinical Trials Portal (http://clinicaltrials.ifpma.org/no_cache/en/myportal/index.htm) and Web of Science Conference Proceedings Citation Index‐ Science (CPCI‐S). We will not use any date or language restrictions in the electronic search for trials.

See: Appendices for details of search strategies for CENTRAL (Appendix 1), MEDLINE (Appendix 2), EMBASE (Appendix 3), LILACS (Appendix 4), CINAHL (Appendix 5), OpenGrey, (Appendix 6), mRCT (Appendix 7), ClinicalTrials.gov (Appendix 8), ICTRP (Appendix 9), IFPMA Clinical Trials Portal (Appendix 10) and CPCI‐S (Appendix 11).

Searching other resources

We will handsearch the following conference proceedings:

• Association for Research in Vision and Ophthalmology (from 1962 to present at www.iovs.org/search.dtl)

• International Council of Ophthalmology (from 1965, when cromolyn sodium was first synthesized (Edwards 2005), to present).

We will check the reference lists of all included studies to identify further potentially relevant studies and search the ISI Science Citation Index to identify articles that have cited the studies that are included in the review.

We will contact experts who are active in the field and primary authors of included studies and relevant reviews to identify further published or non‐published studies eligible for inclusion. We will also contact pharmaceutical companies for unpublished trials.

Selection of studies

Two review authors (JJGL, RML) will independently check the titles and abstracts resulting from the searches. Using a form developed to document the process, the titles and abstracts will be divided into three groups: ‘exclude’, ‘unsure’ or ‘include’. All those titles classified as ‘unsure’ or ‘include’ will be marked and their full‐text versions retrieved for definitive assessment of eligibility. At this stage, we will only exclude those papers classified by both review authors as ‘exclude’.

We will try to obtain further information about any trial that has been published only as an abstract. If a full report is not available, and the information cannot be obtained from the trial investigators, we will exclude the abstract. If relevant, we will report the trial characteristics in the table of excluded studies.

Using another form developed to document the process, we will classify the full‐text copies into three groups (‘exclude’, ‘unsure’ or ‘include’), according to the 'Criteria for considering studies for this review' section. We will resolve any disagreements through discussion. If we cannot reach a consensus, we will consult a third review author (JLA) or the Cochrane Eyes and Vision Group (CEVG) editorial base . If relevant, we will document the disagreements. If there is insufficient information to determine the eligibility of a study (full‐text classified as ‘unsure’), we will ask the authors for clarification. We will detail in the table of excluded studies all relevant studies labelled as ‘excluded’ after the assessment of the full‐text, with the reasons for their exclusion.

Review authors will not be masked to trial results or publication details during the selection of the studies.

Data extraction and management

Two review authors will independently perform the extraction of data from trial reports, using pre‐designed data extraction forms. 

We will extract information about the methods used in the trial reports and the following details:

• participants (age, gender, setting, number in each group, comparability at baseline, etc);

• interventions (dosage, schedule, compliance, comparison group, timing, etc);

• outcomes (primary and secondary outcomes, adverse events);

• results;

• risk of bias; and

• other information (such as funding).

The review authors will resolve discrepancies on the data extraction through discussion. If there is no consensus, a third review author (JLA) or the CEVG editorial base will settle the discrepancies.

One review author will enter the data in to Review Manager 5 (RevMan 2011) (JJGL) and another review author will check the data entered manually (JLA).

Assessment of risk of bias in included studies

We will assess the risk of bias of each included study, according to the criteria of The Cochrane Collaboration’s tool for assessing risk of bias as given in Chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a). We will consider the following domains:

• random sequence generation;

• allocation concealment;

• blinding (masking) of participants and personnel*;

• masking of outcome assessment*;

• incomplete outcome data*; and

• selective outcome reporting.

*Assessments will be made for each main outcome (or class of outcomes). 

We will define adequate randomisation methods as those that are unpredictable, such as simple, blocked or stratified randomisation using published lists of random numbers or computer‐generated lists of random assignments. We will define inadequate randomisation methods as those that can be predicted, such as alternation, assignment based on date of birth, case record number or date of presentation (Higgins 2011a).

For each study, two review authors (JJGL, EdD), not masked to the study details, will independently assess each domain as ‘low risk of bias', ‘high risk of bias' or ‘unclear risk of bias'.

Disagreements will be resolved by discussion and consensus and, if necessary, with the involvement of a third author (JLA) or the CEVG editorial base.

We will summarise the overall risk of bias for each outcome (or class of similar outcomes) in two different ways (Higgins 2011a):

• within each study across domains: each outcome (or class of outcomes) will be defined as having a ‘low risk of bias’ only if it meets all the domains; as ‘high risk of bias’ if it demonstrates high risk of bias for one or more of domains; or an ‘unclear risk of bias’ if it demonstrates unclear risk of bias for at least one key domain without any of them described as ‘high risk of bias’;

• across studies: each outcome (or class of outcomes) will be defined as having a ‘low risk of bias’ if most information is from studies at low risk of bias; as ‘high risk of bias’ if the proportion of information from studies at high risk of bias is sufficient to affect the interpretation of the results; or an ‘unclear risk of bias’ if most information is from studies at low or unclear risk of bias.

We will try to obtain the information from the trial reports. If any domain is assessed as ‘unclear’, JJGL will write to the authors for clarification. If clarification is not obtained in less than 30 days, we will assign a grading to the outcome, based on the consensus between the review authors. 

Finally, we will incorporate the results of the 'Risk of bias' assessment in to the review through systematic narrative description and commentary and we will explore the effect of the risk of bias in the meta‐analysis (see 'Sensitivity analysis').

Measures of treatment effect

We will analyse data using Chapter 9 of the Cochrane Handbook for Systematic Reviews of Interventions (Deeks 2011) as a guide. We will report study results with their associated 95% confidence intervals (CI).

Measures of treatment effect will depend on the types of data presented in the individual studies:

• for dichotomous data (e.g. presence of adverse events), we will report the odds ratio (OR);

• for counts of rare events and rates we will report the OR;

• for counts of more common events we will report in the same way as continuous outcome data (see below);

• for ordinal data (e.g. signs and symptoms scales, quality of life scales) we will use the difference between means (DM) with its 95% CI . If different scales are used to measure continuous outcomes, we will calculate a standardised mean difference (SMD);

• for time‐to‐event data, we will report the hazard ratio (HR), if possible.

Unit of analysis issues

Seasonal and perennial AC usually present with symptoms in both eyes. For this reason, analysis of these studies can be difficult: people have two eyes and different studies deal with this in different ways. Moreover, it is not always evident from a trial report whether eyes or participants are being analysed and review authors may need to contact trial investigators in order to clarify what has been done (Cochrane Eyes and Vision Group module 2011).

We will not assume that two eyes in an individual are independent data. We will determine whether the data were correctly analysed by looking for potential 'unit of analysis errors' (Table 1). Comparisons that randomise participants but the unit of analysis is the eye do not account for clustering during analysis and have 'unit of analysis errors', resulting in potential artificially extreme P values and overly narrow CIs (Deeks 2011, Section 9.3; Ukoumunne 1999).

Data will be considered to have been analysed correctly if either:

• the analysis was conducted at the same level as the allocation; or

• the analysis was conducted at the level of the eye, but appropriate statistical correction for the clustering was performed.

If the data analysis was determined to have been performed incorrectly, we will extract the data and perform analyses using Chapter 9.3 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b) as a guide.

Dealing with missing data

We will describe missing outcomes of the included studies by reporting proportions of randomised participants for whom no outcome data were obtained (with reasons) by outcome and by randomised group. We will address the potential impact of the missing outcomes on the results of the included studies in the assessment of risk of bias and we will describe its impact on the findings of the review in the discussion section.

For all outcomes, we will try to carry out ‘analyses on an intention‐to‐treat’ principle (Table 1). We will contact the primary authors to request missing data and clarification of issues. If this information cannot be obtained, we will perform an ‘available case analysis’ (Table 1). If possible, we will ‘re‐include’ avoidable exclusions made by the authors of the included studies (Higgins 2011b, section 16.2). We will perform sensitivity analyses to assess how sensitive results are to changes in the assumptions made in the ‘available case analysis’ (see 'Sensitivity analysis').

Assessment of heterogeneity

We will check for heterogeneity by examining the following.

1. The characteristics of the included studies.

2. The forest plot of results of the studies. We will display graphically the results of the studies and we will check the symmetry of their results visually.

3. The results of the Chi² test for statistical heterogeneity (a significant P value will be considered for P < 0.10).

4. The results of the I² statistic for the quantification of the heterogeneity. The I² statistic describes the percentage total variation across studies that is due to heterogeneity rather than chance. We will judge the importance of the observed value of I² depending on the magnitude and direction of effects and the strength of evidence for heterogeneity (moderate to high heterogeneity will be defined as I² ≥ 50%) (Deeks 2011).

Assessment of reporting biases

We will attempt to minimise reporting bias:

• by including both published and unpublished studies;

• in the case of studies with multiple publications, by extracting data on outcomes from the publication with the most mature data;

• by not excluding studies solely on the basis of the publication language.

We will look for evidence of publication bias for each outcome. We will assess publication bias in two different ways:

• graphically, by visual assessment of funnel plots (Table 1); and

• statistically, provided there are at least 10 studies in the meta‐analysis; we will follow the guidance provided by Sterne 2011 for statistical testing for funnel plot asymmetry.

Visual and statistical assessment of funnel plot asymmetry will be interpreted cautiously. If there is evidence of ‘small‐study effects’ (Table 1), we will consider publication bias as only one of a number of possible explanations (Sterne 2011).

Data synthesis

We will try to combine all the outcome measures from the individual trials in a meta‐analysis to provide a pooled effect estimate for each outcome only if the studies are clinically and methodologically comparable.

If the number of trials for each outcome measure is greater than three and no heterogeneity is detected in their results statistically or by observation, we will use a random‐effects model to pool data. If the number of trials is three or less, we will use the fixed‐effect model, provided that heterogeneity has not been detected either statistically or by observation (Deeks 2011). We will assess the influence of the statistical model in the effects being evaluated ('Sensitivity analysis'). We will use the generic inverse variance to meta‐analyse trials that have used paired data with trials that have not.

If heterogeneity is detected, it may be inappropriate to combine results to produce a single summary measure. In this case, we will not combine results but undertake a narrative analysis of studies, providing a descriptive presentation of the results, with supporting tables.

We will perform the analyses using the Review Manager 5 (RevMan 2011) statistical package provided by The Cochrane Collaboration and we will present the results with 95% CIs.

Subgroup analysis and investigation of heterogeneity

We will perform the following subgroup analysis based on the study design used:

• paired studies (trials in which only one eye was treated with topical mast cell stabilisers and compared with the other eye) versus unpaired studies.

Sensitivity analysis

If there are sufficient studies, we will undertake the following sensitivity analyses:

• we will explore the effect of the risk of bias in the meta‐analysis by excluding studies with any domain assessed as 'high risk of bias' or 'unclear risk of bias';

• we will explore the impact of the assumptions taken in the ‘available case analysis’ by performing a sensitivity analysis with imputation of missing data:

  • for dichotomous outcomes, we will consider the ‘best‐case’ and ‘worst‐case’ scenarios (Gamble 2005). We will define the ‘best‐case’ scenario as all participants with missing outcomes in the experimental intervention group having good outcomes, and all those with missing outcomes in the control intervention group having poor outcomes; the ‘worst‐case’ scenario will be the converse (Higgins 2011b);

  • for continuous data, we will conduct a sensitivity analysis assuming a fixed difference between the actual mean for the missing data and the mean assumed by the analysis (Higgins 2011b);

• we will assess the effect of the statistical model chosen for meta‐analysis: we will use a fixed‐effect model for those meta‐analyses performed with a random‐effects model, and a random‐effects model will be used for meta‐analysis where a fixed‐effect model was used in the first place.