Types of studies

We included studies with the following design characteristics:

• Randomised controlled trials (including cluster‐randomised trials where the unit of randomisation is the setting or operator) and quasi‐randomised trials.

• Patients were followed up for at least one week.

We excluded studies with the following design characteristics:

• Cross‐over trials, because CSOM is not expected to be a stable chronic condition. Unless data from the first phase were available, we excluded such studies.

Types of participants

We included studies with patients (adults and children) who had:

• chronic ear discharge of unknown cause; or

• chronic suppurative otitis media.

We defined patients with chronic ear discharge as patients with at least two weeks of ear discharge, where the cause of the discharge was unknown.

We defined patients with chronic suppurative otitis media (CSOM) as patients with:

• chronic or persistent ear discharge for at least two weeks; and

• a perforated tympanic membrane.

We did not exclude any populations based on age, risk factors (cleft palate, Down syndrome), ethnicity (e.g. Australian Aboriginal or Torres Strait Islanders) or the presence of ventilation tubes (grommets). Where available, we recorded these factors in the patient characteristics section during data extraction from the studies. If any of the included studies recruited these patients as a majority (80% or more), we analysed them in a subgroup analysis (see Subgroup analysis and investigation of heterogeneity).

We excluded studies where the majority (more than 50%) of participants:

• had an alternative diagnosis to CSOM (e.g. otitis externa);

• had underlying cholesteatoma;

• had ear surgery within the last six weeks.

We did not include studies designed to evaluate interventions in the immediate peri‐surgical period, which were focused on assessing the impact of the intervention on the surgical procedure or outcomes.

Types of interventions

Types of outcome measures

We analysed the following outcomes in the review, but we did not use them as a basis for including or excluding studies.

We extracted and reported data from the longest available follow‐up for all outcomes.

Electronic searches

The Information Specialist searched:

• the Cochrane ENT Register (searched via the Cochrane Register of Studies to 16 March 2020);

• the Cochrane Central Register of Controlled Trials (CENTRAL) (searched via the Cochrane Register of Studies Web to 16 March 2020);

• Ovid MEDLINE(R) Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, Ovid MEDLINE(R) Daily and Ovid MEDLINE(R) (1946 to 16 March 2020);

• Ovid EMBASE (1974 to 16 March 2020);

• EBSCO CINAHL (1982 to 16 March 2020);

• LILACS (Latin American and Caribbean Health Science Information database), lilacs.bvsalud.org (search to 16 March 2020);

• Web of Knowledge, Web of Science (1945 to 16 March 2020);

• ClinicalTrials.gov, www.clinicaltrials.gov (search via the Cochrane Register of Studies to 16 March 2020);

• World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) (search to 16 March 2020).

We also searched:

• IndMed (search to 22 March 2018);

• African Index Medicus (search to 22 March 2018).

The search strategies for major databases are detailed in Appendix 1. The Information Specialist modelled subject strategies for databases on the search strategy designed for CENTRAL. The strategies were designed to identify all relevant studies for a suite of reviews on various interventions for chronic suppurative otitis media (Bhutta 2018; Brennan‐Jones 2020; Brennan‐Jones 2018b; Chong 2018a; Chong 2018b; Head 2020a; Head 2020b). A supplementary search of the major databases was performed for this review. Where appropriate, they were combined with subject strategy adaptations of the highly sensitive search strategy designed by Cochrane for identifying randomised controlled trials and controlled clinical trials (as described in the Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0, Box 6.4.b. (Handbook 2011).

Searching other resources

We scanned the reference lists of identified publications for additional trials and contacted trial authors where necessary. In addition, the Information Specialist searched Ovid MEDLINE to retrieve existing systematic reviews relevant to this systematic review, so that we could scan their reference lists for additional trials. The Information Specialist also ran non‐systematic searches of Google Scholar to retrieve grey literature and other sources of potential trials.

We did not perform a separate search for adverse effects. We considered adverse effects described in the included studies only.

We contacted original authors for clarification and further data if trial reports were unclear and we arranged translations of papers where necessary.

Selection of studies

At least two review authors (KH/LYC) independently screened all titles and abstracts of the references obtained from the database searches to identify potentially relevant studies. At least two review authors (KH/LYC) evaluated the full text of each potentially relevant study to determine whether it met the inclusion and exclusion criteria for this review.

We resolved any differences by discussion and consensus, with the involvement of a third author for clinical and methodological input where necessary.

Data extraction and management

At least two review authors (KH/LYC/CBJ/MB) independently extracted data from each study using a standardised data collection form (see Appendix 2). Whenever a study had more than one publication, we retrieved all publications to ensure complete extraction of data. Where there were discrepancies in the data extracted by different review authors, we checked these against the original reports and resolved any differences by discussion and consensus, with the involvement of a third author or a methodologist where appropriate. We contacted the original study authors for clarification or for missing data whenever possible. If differences were found between publications of a study, we contacted the original authors for clarification. We used data from the main paper(s) if no further information was found.

We included key characteristics of the included studies, such as study design, setting (including location), year of study, sample size, age and sex of participants, and how outcomes were defined or collected in the studies. In addition, we have also collected baseline information on prognostic factors or effect modifiers (see Appendix 2). For this review, this included the following information whenever available:

• duration of ear discharge at entry to the study;

• diagnosis of ear discharge (where known);

• number people who may have been at higher risk of CSOM, including those with cleft palate or Down syndrome;

• ethnicity of participants including the number who were from Indigenous populations;

• number who had previously had ventilation tubes (grommets) inserted (and, where known, the number who had tubes still in place);

• number who had previous ear surgery;

• number who had previous treatments for CSOM (non‐responders, recurrent versus new cases).

We recorded concurrent treatments alongside the details of the interventions used. See the 'Data extraction form' in Appendix 2 for more details.

For the outcomes of interest to the review, we extracted the findings of the studies on an available case analysis basis, i.e. we included data from all patients available at the time points based on the treatment randomised whenever possible, irrespective of compliance or whether patients had received the treatment as planned.

In addition to extracting pre‐specified information about study characteristics and aspects of methodology relevant to risk of bias, we extracted the following summary statistics for each trial and each outcome:

• For continuous data: the mean values, standard deviations and number of patients for each treatment group. Where endpoint data were not available, we extracted the values for change from baseline. We analysed data from disease‐specific quality of life scales such as COMOT‐12, COMOT‐15 and CES as continuous data.

• For binary data: the number of participants who experienced an event and the number of patients assessed at the time point.

• For ordinal scale data: if the data appeared to be approximately normally distributed or if the analysis that the investigators performed suggested parametric tests were appropriate, then we treated the outcome measures as continuous data. Alternatively, if data were available, we converted it into binary data.

• Time‐to‐event outcomes: we did not expect any outcomes to be measured as time‐to‐event data. However, if outcomes such as resolution of ear discharge were measured in this way, we reported the hazard ratios.

For resolution of ear discharge, we extracted the longest available data within the time frame of interest, defined as from one week up to (and including) two weeks (7 days to 14 days), from two weeks up to (and including) four weeks (15 to 28 days), and after four weeks (28 days or one month).

For other outcomes, we reported the results from the longest available follow‐up period.

Assessment of risk of bias in included studies

At least two review authors (KH/LYC/CBJ/MB) independently assessed the risk of bias of each included study. We followed the guidance in the Cochrane Handbook for Systematic Reviews of Interventions (Handbook 2011), using the Cochrane 'Risk of bias' tool. With this tool we assessed the risk of bias as 'low', 'high' or 'unclear' for each of the following six domains:

• sequence generation;

• allocation concealment;

• blinding of participants, personnel and outcome assessment;

• incomplete outcome data;

• selective reporting;

• other sources of bias.

Measures of treatment effect

We summarised the effects of dichotomous outcomes (e.g. proportion of patients with complete resolution of ear discharge) as risk ratios (RR) with confidence intervals (CIs). For the key outcomes that are presented in the 'Summary of findings' table, we expressed the results as absolute numbers based on the pooled results and compared to the assumed risk. We also had planned to calculate the number needed to treat to benefit (NNTB) using the pooled results, where the results were moderate or high certainty. The assumed baseline risk would have been either (a) the median of the risks of the control groups in the included studies, this being used to represent a 'medium‐risk population' or, alternatively, (b) the average risk of the control groups in the included studies, which is used as the 'study population' (Handbook 2011). If a large number of studies were available, and where appropriate, we would have also presented additional data based on the assumed baseline risk in (c) a low‐risk population and (d) a high‐risk population.

Had we had any continuous outcomes, we would have expressed treatment effects as a mean difference (MD) with standard deviation (SD). If different scales were used to measure the same outcome we would have used the standardised mean difference (SMD) and provided a clinical interpretation of the SMD values.

Unit of analysis issues

Dealing with missing data

We attempted to contact study authors via email whenever the outcome of interest was not reported but the methods of the study had suggested that the outcome had been measured. We did the same if not all of the data required for the meta‐analysis were reported, unless the missing data were standard deviations. If standard deviation data were not available, we approximated these using the standard estimation methods from P values, standard errors or 95% CIs if these were reported, as detailed in the Cochrane Handbook for Systematic Reviews of Interventions (Handbook 2011). Where it was impossible to estimate these, we contacted the study authors.

Apart from imputations for missing standard deviations, we did not conduct any other imputations. We extracted and analysed data for all outcomes using the available case analysis method.

Assessment of heterogeneity

We assessed clinical heterogeneity (which may be present even in the absence of statistical heterogeneity) by examining the included studies for potential differences in the types of participants recruited, interventions or controls used, and the outcomes measured. We did not pool studies where the clinical heterogeneity made it unreasonable to do so.

We assessed statistical heterogeneity by visually inspecting the forest plots and by considering the Chi² test (with a significance level set at P value < 0.10) and the I² statistic, which calculated the percentage of variability that was due to heterogeneity rather than chance, with I² values over 50% suggesting substantial heterogeneity (Handbook 2011).

Assessment of reporting biases

We assessed reporting bias as within‐study outcome reporting bias and between‐study publication bias.

Data synthesis

We conducted all meta‐analyses using Review Manager 5.3 (RevMan 2014). For dichotomous data, we analysed treatment differences as a risk ratio (RR) calculated using the Mantel‐Haenszel methods. We analysed time‐to‐event data using the generic inverse variance method.

For continuous outcomes, if all the data were from the same scale, we would have pooled the mean values obtained at follow‐up with change outcomes and reported this as a MD. However, if the SMD had to be used as an effect measurement, we would not have not pooled change and endpoint data.

When statistical heterogeneity is low, random‐effects versus fixed‐effect methods yield trivial differences in treatment effects. However, when statistical heterogeneity is high, the random‐effects method provides a more conservative estimate of the difference.

Subgroup analysis and investigation of heterogeneity

We subgrouped studies where most participants (80% or more) met the criteria stated below in order to determine whether the effect of the intervention was different compared to other patients. Due to the risks of reporting and publication bias with unplanned subgroup analyses of trials, we only analysed subgroups reported in studies if these were prespecified and stratified at randomisation.

We planned to conduct subgroup analyses regardless of whether statistical heterogeneity was observed for studies that included patients identified as high risk (i.e. thought to be less responsive to treatment and more likely to develop CSOM, recurrence or complications) and patients with ventilation tubes (grommets). 'High risk' patients include Indigenous populations (e.g. Australian Aboriginal and Torres Strait Islanders, Native Americans and Inuit populations of Alaska, Canada and Greenland), people with craniofacial malformation (e.g. cleft palate), Down syndrome and people with known immunodeficiency.

We planned to present the main analyses of this review in the form of forest plots based on this main subgroup analysis.

• For the high‐risk group, this applied to the outcomes resolution of ear discharge (dry ear), quality of life, pain/discomfort, development of complications and hearing loss.

For patients with ventilation tubes, this applied to the outcome resolution of ear discharge (dry ear) for the time point of four weeks or more because this group was perceived to be at lower risk of treatment failure and recurrence than other patient groups. If statistical heterogeneity was observed, we also conducted subgroup analysis for the effect modifiers below. If there were statistically significant subgroup effects, we presented these subgroup analysis results as forest plots.

For this review, effect modifiers include:

• Diagnosis of CSOM: it was likely that some studies would include patients with chronic ear discharge but who had not had a diagnosis of CSOM. Therefore, we subgrouped studies where most patients (80% or more) met the criteria for CSOM diagnosis in order to determine whether the effect of the intervention was different compared to patients where the precise diagnosis was unknown and inclusion into the study was based purely on chronic ear discharge symptoms.

• Duration of ear discharge: there is uncertainty about whether the duration of ear discharge prior to treatment has an impact on the effectiveness of treatment and whether more established disease (i.e. discharge for more than six weeks) is more refractory to treatment compared with discharge of a shorter duration (i.e. less than six weeks).

• Patient age: patients who were younger than two years old versus patients up to six years old versus adults. Patients under two years are widely considered to be more difficult to treat.

We presented the results as subgroups regardless of the presence of statistical heterogeneity based on the main types of aural toileting methods as follows:

• dry mopping;

• irrigation;

• microsuction.

This was because the different methods of aural toileting were expected to have different treatment effects and possible adverse effects due to their intensity (e.g. microsuction is thought to be a more intense method than dry mopping).

Sensitivity analysis

We planned to carry out sensitivity analyses to determine whether the findings were robust to the decisions made in the course of identifying, screening and analysing the trials. We planned to conduct sensitivity analysis for the following factors, whenever possible:

• Impact of model chosen: fixed‐effect versus random‐effects model.

• Risk of bias of included studies: excluding studies with high risk of bias (we defined these as studies that have a high risk of allocation concealment bias and a high risk of attrition bias (overall loss to follow‐up of 20%, differential follow‐up observed)).

• Where there was statistical heterogeneity, studies that only recruited patients who had previously not responded to one of the treatments under investigation in the RCT. Studies that specifically recruited patients who did not respond to a treatment could potentially have reduced the relative effectiveness of an agent.

If any of these investigations found a difference in the size of the effect or heterogeneity, we would have mentioned this in the 'Effects of interventions' section and/or presented the findings in a table.

Summary of findings and assessment of the certainty of the evidence

Using the GRADE approach, at least two review authors (KH/LYC) independently rated the overall certainty of evidence using the GDT tool (http://www.guidelinedevelopment.org/) for the main comparison pairs listed in the Types of interventions section. The certainty of evidence reflects the extent to which we were confident that an estimate of effect was correct and we applied this in the interpretation of results. There are four possible ratings: 'high', 'moderate', 'low' and 'very low' (Handbook 2011). A rating of 'high' certainty evidence implies that we were confident in our estimate of effect and that further research is very unlikely to change our confidence in the estimate of effect. A rating of 'very low' certainty implies that any estimate of effect obtained is very uncertain.

The GRADE approach rates evidence from RCTs that do not have serious limitations as high certainty. However, several factors could lead to the downgrading of the evidence to moderate, low or very low. The degree of downgrading was determined by the seriousness of these factors:

• study limitations (risk of bias);

• inconsistency;

• indirectness of evidence;

• imprecision;

• publication bias.

The 'Summary of findings' table presents the following outcomes:

• resolution of ear discharge or 'dry ear':

  • at between one week and up to two weeks;

  • after four weeks;

• health‐related quality of life;

• ear pain (otalgia) or discomfort or local irritation;

• hearing;

• serious complications;

• adverse events: dizziness/vertigo/balance problems.