Types of studies

Published or unpublished randomised controlled trials (RCTs) that evaluated the effects of any brand of NPWT in the treatment of foot wounds in people with DM, irrespective of publication status or language of publication.

Types of participants

Trials recruiting people with type 1 or type 2 DM (as defined by the study authors), with foot wounds below the ankle, regardless of underlying aetiology (i.e. ischaemic, neuropathic or neuroischaemic). This included diabetic foot ulcers, or wounds resulting from amputation or other surgical treatment, or both. We included trials involving people of any age and from any setting.

Where trials with broad inclusion criteria recruited people with DM with foot wounds as part of a larger chronic wound study population (e.g. alongside participants with pressure ulcers or leg ulcers), we excluded these trials unless the results for the subgroup of people with DM with foot wounds were reported separately or were available from authors on request.

Types of interventions

Any brand of NPWT (including studies that investigated homemade or ad hoc negative pressure devices) compared with standard care (such as advanced wound dressings and gauze) or other treatments, so that the primary intervention of interest was NPWT (both commercial and non‐commercial treatments). We included RCTs in which the use of a specific NPWT intervention during the treatment period was the only systematic difference between treatment groups. We anticipated that likely comparisons would include the use of NPWT during the care pathway compared with no use of NPWT or comparison of different types/brands of NPWT used during the care pathway.

Types of outcome measures

We listed primary and secondary outcomes below. If a study was otherwise eligible (i.e. correct study design, population and intervention/comparator) but did not report a listed outcome, then we contacted the study authors where possible to establish whether an outcome of interest here was measured but not reported. If we remained unsure whether an outcome was measured or not, the study was included. We reported outcome measures at the latest time point available (assumed to be length of follow‐up if not specified) and the time point specified in the methods as being of primary interest (if this was different from latest time point available). For all outcomes, we planned to class assessment of outcome measures from:

• one week or less to eight weeks as short term;

• eight weeks to 16 weeks as medium term;

• more than 16 weeks as long term.

Electronic searches

In January 2018, we searched the following electronic databases to identify reports of relevant clinical trials:

• Cochrane Wounds Group Specialised Register (searched 10 January 2018);

• Cochrane Central Register of Controlled Trials (CENTRAL; 2017, Issue 12) in the Cochrane Library (searched 10 January 2018);

• Ovid MEDLINE including In‐Process & Other Non‐Indexed Citations (1946 to 10 January 2018);

• Ovid Embase (1974 to 10 January 2018);

• EBSCO CINAHL Plus (Cumulative Index to Nursing and Allied Health Literature; 1937 to 10 January 2018).

Appendix 1 shows the search strategies for the Cochrane Wounds Specialised Register, CENTRAL, Ovid MEDLINE, Ovid Embase and EBSCO CINAHL Plus. We combined the Ovid MEDLINE search with the Cochrane Highly Sensitive Search Strategy for identifying randomised trials in MEDLINE: sensitivity‐ and precision‐maximising version (2008 revision) (Lefebvre 2011). We combined the Embase search with the Ovid Embase filter developed by the UK Cochrane Centre (Lefebvre 2011). We combined the CINAHL Plus searches with the trial filters developed by the Scottish Intercollegiate Guidelines Network (SIGN 2018). There were no restrictions with respect to language, date of publication or study setting.

We also searched the following clinical trials registries for unpublished and ongoing studies in the area. We searched for trials evaluating NPWT and explored these records for those pertaining to foot wounds in people with DM as defined above:

• ClinicalTrials.gov (www.clinicaltrials.gov) (28 February 2018);

• WHO International Clinical Trials Registry Platform (apps.who.int/trialsearch/Default.aspx) (28 February 2018);

• EU Clinical Trials Register (www.clinicaltrialsregister.eu/ctr‐search/search) (28 February 2018).

Appendix 1 shows the search strategies for clinical trial registries.

Searching other resources

We aimed to identify other potentially eligible trials or ancillary publications by searching the reference lists of retrieved included trials, as well as relevant systematic reviews, meta‐analyses and health technology assessment reports.

When necessary, we contacted authors of key papers and abstracts to request further information about their trials.

We also examined the content of the European Wound Management Association conference proceedings (2012 to 2017) and systematic reviews in the field that might have referred to data we had not found, and contacted key manufacturers (KCI, and Smith & Nephew) to ask about unpublished (as well as ongoing) work.

Selection of studies

Two review authors independently assessed the titles and abstracts of retrieved studies for relevance. After this initial assessment, we obtained full copies of all studies considered to be potentially relevant. Two review authors independently checked the full papers for eligibility; we resolved disagreements by discussion and, where required, the input of a third review author. We recorded all reasons for exclusion of studies for which we had obtained full copies in the Characteristics of excluded studies table. We completed a PRISMA flowchart to summarise this process (Liberati 2009).

Data extraction and management

We extracted and summarised details of the eligible studies using a data extraction sheet. Two review authors extracted data independently and resolved disagreements by discussion, drawing on a third review author where required. Where data were missing from reports, we attempted to contact the study authors to obtain this information. We included studies published in duplicate once, but extracted the maximal amount of data. We extracted the following data, where possible:

• country of origin;

• participants' type of DM;

• wound aetiology (e.g. PAD);

• type of wound, including site on foot;

• unit of investigation (per participant) (i.e. single wound, or foot, or patient, or multiple wounds on the same participant);

• care setting;

• number of participants randomised to each trial arm;

• eligibility criteria and key baseline participant data;

• details of the dressing/treatment regimen received by each group;

• details of any co interventions;

• number of postamputation/debridement wounds closed surgically;

• primary and secondary outcome(s) (with definitions);

• outcome data for primary and secondary outcomes (by group);

• duration of follow‐up;

• number of withdrawals (by group);

• adverse events;

• publication status of study; and

• source of funding for trial.

Assessment of risk of bias in included studies

Two review authors independently assessed each included study using the Cochrane tool for assessing risk of bias (Higgins 2011a) (Appendix 2). This tool addresses six specific domains, namely, sequence generation, allocation concealment, blinding, incomplete outcome data, selective outcome reporting and other issues (e.g. extreme baseline imbalance, issues with unit of investigation). We assessed blinding of participants and health professionals, and blinded outcome assessment separately. Blinding to reduce the risk of performance bias is often not possible in device trials but it can be minimised, for example, in some cases using blinded panels to make care decisions. To avoid detection bias, blinded outcome assessment is key in open trials. Hróbjartsson 2012 argued that the estimated effects of experimental interventions in RCTs tended to be considerably more optimistic when they were based on non‐blinded assessment of subjective outcomes compared with blinded assessment.

For our assessment, we were aware that blinding of participants and health professionals to treatment received would not be possible, but it was important to understand if, and how, studies had compensated for this where required. We completed a 'Risk of bias' table for each eligible study and resolved disagreements about risk of bias assessment by discussion. Where possible, when a lack of reported information resulted in an unclear decision, we contacted authors for clarification.

We classed studies with an assessment of high risk of bias for the randomisation sequence domain or the allocation concealment domain or the blinded outcome assessment domain (for specified outcome) (or a combination of these) as being at overall high risk of bias. We also considered the potential for performance and measurement bias for each primary and secondary outcome extracted.

Measures of treatment effect

Where possible, we grouped studies according to wound type. Where possible, we presented the outcome results for each trial with 95% confidence intervals (CI). We reported estimates for dichotomous outcomes (e.g. ulcers healed during a particular time period) as risk ratios (RR). We used the RR rather than odds ratio (OR), since, when event rates are high, as is the case for many trials reporting wound healing, ORs (when interpreted as RR) can give an inflated impression of the effect size (Deeks 2002). We planned to report outcomes relating to continuous data (e.g. percentage change in ulcer area) as mean differences (MD) and overall effect size (with 95% CI). Where a study reported data on time‐to‐healing (the probability of healing over a consecutive time period) we planned to report and plot these data (where possible) using hazard ratio (HR) estimates. However, where the HR was not reported, but data regarding the number of events and the P value for a log rank test (reported to at least two significant figures) were reported, we employed methods proposed by Parmar 1998 to calculate the HR indirectly. Where log rank test P values were published to only one significant figure, the robustness of the calculated HR for the highest possible P value was investigated to test robustness of estimates. HRs and associated 95% CIs were then calculated using the inverse variance option in Review Manager 5 (Review Manager 2014).

Unit of analysis issues

We recorded whether trials presented outcomes in relation to a wound, a foot, a participant or as multiple wounds on the same participant. We also recorded occasions where multiple wounds on a participant were (incorrectly) treated as independent within a study, rather than having within‐participant analysis methods applied. This was recorded as part of the risk of bias assessment. For wound healing and amputation, unless otherwise stated, where the number of wounds appeared to equal the number of participants, we treated the participant as the unit of analysis. For other adverse event outcomes, in order to facilitate further analyses, we aimed to establish whether data were presented at the level of the participant, because in this area there is potential for data to refer to multiple events occurring to a single person (or wound per person), which means that data cannot be analysed further without violating the assumption of independence.

Where studies randomised at the participant level and measured outcomes at the wound level (e.g. wound healing), we treated the participant as the unit of analysis when the number of wounds assessed appeared equal to the number of participants (e.g. one wound per person).

Where there were instances of clustered data, that is where a proportion of individually randomised trial participants had outcome data collected and reported on multiple wounds, this was not treated as a cluster trial since not all participants would have multiple wounds. Rather this was a trial that incorrectly included a mixture of individual and clustered data. We noted these trials and recorded the issue in the risk of bias assessment. Data were extracted and presented but not the subject of any further analyses.

We planned only to incorporate clearly conducted fully clustered trials into meta‐analyses if the trial was analysed correctly. Where a cluster trial had been conducted but incorrectly analysed, we recorded this as part of the 'Risk of bias' assessment. If possible we planned to approximate the correct analyses based on Cochrane Handbook for Systematic Reviews of Interventions guidance (Higgins 2011b) using information on:

• the number of clusters (or groups) randomised to each intervention group; or the average (mean) size of each cluster;

• the outcome data ignoring the cluster design for the total number of participants (e.g. number or proportion of participants with events, or means and standard deviations (SD)); and

• an estimate of the intracluster (or intraclass) correlation coefficient (ICC).

Where multiple trial arms were reported in a single trial, we planned to include only the relevant arms. If two interventions or more interventions were compared with control and eligible for the same meta‐analysis, we planned to pool the intervention arms and compare them with control. If the study data could not be analysed correctly, we extracted outcome data and presented them but did not analysed them further.

Dealing with missing data

It is common to have data missing from trial reports. Excluding participants post randomisation from the analysis, or ignoring those participants who are lost to follow‐up compromises the randomisation, and potentially introduces bias into the trial. In individual studies, where data on the proportion of ulcers healed were presented, we assumed that if randomised participants were not included in an analysis, their wound did not heal (i.e. they would be considered in the denominator but not the numerator). Where a trial did not specify participant group numbers prior to dropout, we presented only complete‐case data. In a time‐to‐healing analysis using survival analysis methods, dropouts should be accounted for as censored data. Hence all participants contributed to the analysis. Such analysis assumes that dropouts are missing at random (i.e. not associated with time to healing). We presented data for area change, and for all secondary outcomes, as a complete‐case analysis.

Assessment of heterogeneity

We considered both clinical and statistical heterogeneity. Wherever appropriate, that is, where studies appeared similar in terms of wound type, intervention type, duration and outcome type, we pooled data using meta‐analysis (conducted using Review Manager 5 (Review Manager 2014)). We planned to assess statistical heterogeneity using the Chi² test (a significance level of P less than 0.1 was considered to indicate heterogeneity) and the I² estimate (Higgins 2003). The I² estimate examines the percentage of total variation across studies due to heterogeneity rather than to chance. Values of I² higher than 50% indicate a high level of heterogeneity. In the absence of clinical heterogeneity and in the presence of statistical heterogeneity (I² over 50%), we envisioned using a random‐effects model; however, we did not anticipate pooling studies where heterogeneity was very high (I² over 75%) (Deeks 2011). Where there was no clinical or statistical heterogeneity, we used a fixed‐effect model.

Assessment of reporting biases

Reporting biases arise when the dissemination of research findings is influenced by the nature and direction of results. Publication bias is one of a number of possible causes of small‐study effects, that is, a tendency for estimates of the intervention effect to be more beneficial in smaller RCTs. Funnel plots allow a visual assessment of whether small‐study effects may be present in a meta‐analysis. A funnel plot is a simple scatter plot of the intervention effect estimates from individual RCTs against some measure of each trial's size or precision (Sterne 2011). We planned to present funnel plots for meta‐analyses comprising 10 RCTs or more using Review Manager 5 (Review Manager 2014).

Data synthesis

We were unable to pre specify the amount of clinical, methodological and statistical heterogeneity in the included studies. Thus, we used a random‐effects approach for meta‐analysis. Conducting meta‐analysis with a fixed‐effect model in the presence of even minor heterogeneity may provide overly narrow CIs. We would only have used a fixed‐effect approach when clinical and methodological heterogeneity was assessed to be minimal, and the assumption that a single underlying treatment effect was being estimated held. Chi² and I² statistics were used to quantify heterogeneity but were not used to guide choice of model for meta‐analysis (Kontopantelis 2012). We would have exercised caution when meta‐analysed data were at risk of small‐study effects because use of a random‐effects model may be unsuitable here. In this case, or where there were other reasons to question the selection of a fixed‐effect or random‐effects model, we planned to assess the impact of the approach using sensitivity analyses to compare results from alternate models, but this was not implemented (Thompson 1999).

We presented data using forest plots where possible. For dichotomous outcomes, we presented the summary estimate as an RR with 95% CI. Where continuous outcomes were measured, we presented an MD with 95% CI; we planned to pool standardised mean difference (SMD) estimates where studies measured the same outcome using different methods. For time‐to‐event data, we planned to use the inverse variance method on the estimated HR and standard error, when reported or calculated from available data. Unfortunately, it was not possible for us to plot (and, if appropriate, to pool) estimates of HRs and 95% CIs for time‐to‐event data, as there were insufficient data presented in the study reports. Where time to healing was analysed as a continuous measure, but it was not clear if all wounds had healed, we documented use of the outcome in the study, but did not summarise or use these data in any meta‐analysis.

We obtained pooled estimates of the treatment effect using Review Manager 5 (Review Manager 2014).

Subgroup analysis and investigation of heterogeneity

We considered whether there was potential heterogeneity between wound dressings used in control groups (i.e. advanced dressings (non‐antimicrobial), antimicrobial dressings or basic contact dressings) as there is no single dressing to suit all scenarios (Wounds International 2013). Where there was evidence of between‐trial heterogeneity in trial‐level co interventions, especially off‐loading, we envisaged a subgroup analysis being conducted based on variations in co interventions (e.g. all trial participants reported to receive adequate off‐loading protocol/advice being compared with trial participants who received unclear advice about off‐loading); however, this was not required. Finally, depending on the number and heterogeneity of included studies, we considered using meta‐regression to investigate wound aetiology as a possible explanatory variable but this analysis also was not possible.

Sensitivity analysis

We planned to perform sensitivity analyses to explore the effect of the removal of studies classed at high risk of bias for any domain, but this was not possible due to lack of available data.