Appendix 1. Methods for network meta‐analysis if we find this is possible in the future

Measures of treatment effect

Relative treatment effects

For dichotomous variables (e.g. proportion of participants with serious adverse events or any adverse events), we will calculate the odds ratio with 95% credible interval (or Bayesian confidence interval) (Severini 1993). For continuous variables (e.g. quality of life reported on the same scale), we will calculate the mean difference with 95% credible interval. We will use standardised mean difference values with 95% credible interval for quality of life if included trials use different scales. For count outcomes (e.g. numbers of adverse events and serious adverse events), we will calculate the rate ratio with 95% credible interval. For time‐to‐event data (e.g. mortality at maximal follow‐up), we will calculate hazard ratio with 95% credible interval.

Relative ranking

We will estimate ranking probabilities for all treatments of being at each possible rank for each intervention. Then, we will obtain the surface under the cumulative ranking curve (SUCRA) (cumulative probability) and rankogram (Salanti 2011; Chaimani 2013).

Unit of analysis issues

We will collect data for all trial treatment groups that meet the inclusion criteria. The codes for analysis that we will use account for the correlation between effect sizes from trials with more than two groups.

Assessment of heterogeneity

We will assess clinical and methodological heterogeneity by carefully examining the characteristics and design of included trials. We will assess the presence of clinical heterogeneity by comparing effect estimates under different categories of potential effect modifiers. Different study designs and risk of bias may contribute to methodological heterogeneity.

We will assess the statistical heterogeneity by comparing results of the fixed‐effect model meta‐analysis and the random‐effects model meta‐analysis, and between‐study standard deviation (tau² and comparing this with values reported in the study of the distribution of between‐study heterogeneity (Turner 2012)), and by calculating I² (using Stata/SE 14.2). If we identify substantial heterogeneity ‐ clinical, methodological, or statistical ‐ we will explore and address heterogeneity in a subgroup analysis (see ‘Subgroup analysis and investigation of heterogeneity for network meta‐analysis’ section).

Assessment of transitivity across treatment comparisons

We will evaluate the plausibility of the transitivity assumption (the assumption that participants included in different studies with different immunosuppressive regimens can be considered part of a multi‐arm randomised clinical trial and could potentially have been randomised to any treatment) (Salanti 2012). In other words, any participant who meets the inclusion criteria is, in principle, equally likely to be randomised to any of the above eligible interventions. If we have any concern that clinical safety and effectiveness are dependent upon effect modifiers, we will continue to do traditional Cochrane pair‐wise comparisons and will not perform a network meta‐analysis on all participant subgroups.

Assessment of reporting biases

For the network meta‐analysis, we will judge reporting bias by completeness of the search (i.e. searching various databases and including conference abstracts), as we do not currently find any meaningful order to performing a comparison‐adjusted funnel plot, as suggested by Chaimani 2012. However, if we find any meaningful order, for example, the control group used depended upon the year of conduct of the trial, we will use a comparison‐adjusted funnel plot, as suggested by Chaimani 2012.

Data synthesis

Methods for indirect and mixed comparisons

We will conduct network meta‐analyses to compare multiple interventions simultaneously for each of the primary and secondary outcomes. Network meta‐analysis combines direct evidence within trials and indirect evidence across trials (Mills 2012). We will obtain a network plot to ensure that trials were connected by treatments using Stata/SE 14.2 (Chaimani 2013). The network plot for mortality at maximal follow‐up for this review is presented in Figure 5. We will exclude any trials that were not connected to the network. We will conduct a Bayesian network meta‐analysis using the Markov chain Monte Carlo method in OpenBUGS 3.2.3, as per guidance from the National Institute for Health and Care Excellence (NICE) Decision Support Unit (DSU) documents (Dias 2014a). We will model treatment contrast (i.e. log odds ratio for binary outcomes, mean difference or standardised mean difference for continuous outcomes, log rate ratio for count outcomes, and log hazard ratio for time‐to‐event outcomes) for any two interventions ('functional parameters') as a function of comparisons between each individual intervention and an arbitrarily selected reference group ('basic parameters') (Lu 2006) using appropriate likelihood functions and links. We will use binomial likelihood and logit link for binary outcomes, Poisson likelihood and log link for count outcomes, binomial likelihood and complementary log‐log link for time‐to‐event outcomes, and normal likelihood and identity link for continuous outcomes. We will apply a fixed‐effect model and a random‐effects model for the network meta‐analysis. We will report both models for comparison with the reference group in a forest plot. For pair‐wise comparison, we will report the fixed‐effect model if the two models reported similar results; otherwise, we will report the more conservative model.

Figure 5

---

---

Network plot for mortality at maximal follow‐up. The size of the node (circle) provides a measure of the number of trials in which the particular treatment was included in one of the arms. The thickness of the line provides a measure of the number of direct comparisons between two nodes (treatments).

We will use a hierarchical Bayesian model using three different initial values and codes provided by NICE DSU (Dias 2014a). We will use a normal distribution with large variance (10,000) for treatment effect priors (vague or flat priors). For the random‐effects model, we will use a prior distributed uniformly (limits: 0 to 5) for between‐trial standard deviation but assumed similar between‐trial standard deviation across treatment comparisons (Dias 2014a). We will use a 'burn‐in' of 5000 simulations, check for convergence visually, and run models for another 10,000 simulations to obtain effect estimates. If we did not obtain convergence, we will increase the number of simulations for 'burn‐in'. If we do not obtain convergence still, we will use alternate initial values and priors according to methods suggested by van Valkenhoef 2012. We will also estimate the probability that each intervention ranks at one of the possible positions using the NICE DSU codes (Dias 2014a).

Assessment of inconsistency

We will assess inconsistency (statistical evidence of violation of the transitivity assumption) by fitting both an inconsistency model and a consistency model. We will use inconsistency models described in the NICE DSU manual, as we plan to use a common between‐study deviation for comparisons (Dias 2014b). In addition, we will use the design‐by‐treatment full interaction model (Higgins 2012) and IF (inconsistency factor) plots (Chaimani 2013) to assess inconsistency. In the presence of inconsistency, we will assess whether it is due to clinical or methodological heterogeneity by performing separate analyses for each of the different subgroups mentioned in the ‘Subgroup analysis and investigation of heterogeneity for network meta‐analysis’ section below.

If we find evidence of inconsistency, we will identify areas in the network where substantial inconsistency might be present in terms of clinical and methodological diversities between trials and, when appropriate, will limit network meta‐analysis to a more compatible subset of trials.

Direct comparison

We will perform direct comparisons using the same codes and the same technical details.

Sample size calculations

To control for risk of random errors, we will interpret information with caution when the accrued sample size in the network meta‐analysis (i.e. across all treatment comparisons) was less than the required sample size (required information size). For calculation of the required information size, see Appendix 3.

Subgroup analysis and investigation of heterogeneity for network meta‐analysis

We will assess differences in effect estimates between subgroups listed in subgroup analysis and investigation of heterogeneity using meta‐regression with the help of the OpenBUGS code (Dias 2012a) if we include a sufficient number of trials. We will use potential modifiers as study level co‐variates for meta‐regression. We will calculate a single common interaction term (Dias 2012a). If 95% credible intervals of the interaction term do not overlap zero, we will consider this as evidence of difference in subgroups.

Presentation of results

We will present effect estimates with 95% CrI for each pair‐wise comparison calculated from direct comparisons and network meta‐analysis. We will present the cumulative probability of treatment ranks (i.e. the probability that the treatment is within the top two, the probability that the treatment is within the top three, etc.) in graphs (surface under the cumulative ranking curve, or SUCRA) (Salanti 2011). We will plot the probability that each treatment is best, second best, third best, etc., for each of the different outcomes (rankograms), which generally are considered more informative (Salanti 2011; Dias 2012b).

We will present 'Summary of findings' tables for mortality. In summary of findings Table for the main comparison, we will follow the approach suggested by Puhan et al. (Puhan 2014). First, we will calculate direct and indirect effect estimates and 95% credible intervals using the node‐splitting approach (Dias 2010) (i.e. calculate the direct estimate for each comparison by including only trials that performed direct comparisons of treatments, and the indirect estimate for each comparison by excluding trials that performed direct comparisons of treatments). Then we will rate the quality of direct and indirect effect estimates using GRADE, which takes into account risk of bias, inconsistency, directness of evidence, imprecision, and publication bias (Guyatt 2011). We will present estimates of the network meta‐analysis and will rate the quality of network meta‐analysis effect estimates as the best quality of evidence between direct and indirect estimates (Puhan 2014). In addition, in the same table, we will present illustrations and information on numbers of trials and participants, as per the standard 'Summary of findings' table.

Appendix 2. Search strategies

Appendix 3. Sample size calculation

Five‐year mortality in patients with primary sclerosing cholangitis is 18% (Talwalkar 2001). The required information size is based on a control group proportion of 18%, a relative risk reduction of 20% in the experimental group, type I error of 5%, and type II error of 20% in 3396 participants. Network analyses may be more prone to risk of random error than direct comparisons (Del Re 2013). Accordingly, a larger sample size is required in indirect comparisons than in direct comparisons (Thorlund 2012). Power and precision in indirect comparisons depend upon various factors, such as the number of participants included under each comparison and heterogeneity between the trials (Thorlund 2012). If no heterogeneity is evident across trials, the sample size in indirect comparisons would be equivalent to the sample size in direct comparisons. The effective indirect sample size can be calculated using the number of participants included in each direct comparison (Thorlund 2012). For example, a sample size of 2500 participants in the direct comparison A versus C (n_AC) and a sample size of 7500 participants in the direct comparison B versus C (n_BC) result in an effective indirect sample size of 1876 participants. However, in the presence of heterogeneity within comparisons, the sample size required is greater. In the above scenario, for an I² statistic for each of the comparisons A versus C (I_AC²) and B versus C (I_BC²) of 25%, the effective indirect sample size is 1407 participants. For an I² statistic for each of the comparisons A versus C and B versus C of 50%, the effective indirect sample size is 938 participants (Thorlund 2012). If the study includes only three groups and sample size is greater than required information size, we will calculate the effective indirect sample size using the following generic formula (Thorlund 2012):

((n_AC × (1 ‐ I_AC²)) × (n_BC × (1 ‐ I_BC²))/((n_AC × (1 ‐ I_AC²)) + (n_BC × (1 ‐ I_BC²)).

No method is currently known to calculate the effective indirect sample size for a network analysis involving more than three intervention groups.