All‐cause mortality

The data from all five included trials were analysed by meta‐analysis as well as by trial sequential analysis. The meta‐analysis showed that pentoxifylline reduced mortality in participants with alcoholic hepatitis; this result was statistically significant (Analysis 1.1). However, the results of the trial sequential analysis did not support the traditional meta‐analysis (Figure 3). Trial sequential analysis showed that the intervention effect (z‐curve, blue line) did not cross the trial sequential monitoring boundary (red curve) indicating lack of information to detect or reject a statistically significant intervention effect of 20% risk ratio reduction (RRR) (Figure 3). Trial sequential analysis is a statistical analysis that is adjusted for multiple testing on accumulating data and, therefore, is a more robust analysis than cumulative meta‐analysis. As such, we lack firm evidence on the intervention effect of pentoxifylline and cannot conclude that pentoxifylline has a positive intervention effect on mortality in participants with alcoholic hepatitis. The trial sequential analysis result also indicates that in order to detect or reject with firm evidence an intervention effect of 20% risk ratio reduction, an information size of 1169 may be needed (Figure 3).

Risk of bias is known to impact on the estimated intervention effect, with trials of a high risk of bias tending to overestimate the intervention effect (Schulz 1995; Moher 1998; Kjaergard 2001; Wood 2008). To assess the effect of risk of bias on the analysis of all‐cause mortality, the included trials were categorised as having either a high or a low risk of bias. Four of the five included trials were judged to be of high risk of bias (Figure 1). Therefore, the estimated intervention effect calculated using data from all five trials is at high risk of bias and is not a robust result.

Separate meta‐analyses were conducted on the trial with a high risk of bias and on those with a low risk of bias. The meta‐analysis on the high risk of bias trials showed that pentoxifylline had a non‐statistically significant positive intervention effect on all‐cause mortality (Analysis 1.3). Interestingly, the meta‐analysis on the single trial with a low risk of bias (Akriviadis 2000) showed greatest intervention effect, a risk ratio of 0.52, and hence, the result is statistically significant (Analysis 1.3).

The trials with a high risk of bias also have very small information sizes, so their results could also be affected by random error. It is also worth noting that the control event rate in the Akriviadis 2000 trial is 48%, whereas the weighted average control event rate in all trials is 39%. A control event rate, which is close to 50%, makes it easier to detect an intervention effect. This may contribute to the explanation that the low risk of bias trial and not the high risk of bias trial estimates a greater intervention effect.

A test of interaction between high risk and low risk of bias trials did not show a significant effect between these two groups, suggesting that risk of bias did not impact on the estimated intervention effect for all‐cause mortality. However, this result is limited because of the small sample sizes in each group (Altman 2003).

Basing recommendations on data from only one trial is not advisable, so the analysis of one trial with 102 participants and a low risk of bias is not sufficient evidence on which to draw conclusions (Akriviadis 2000). This is in part because the information size is too small, which means that there is a risk of random error, ie, that the result is actually a false positive (type I error) or a false negative (type II error). As such we need at least one other trial with a low risk of bias demonstrating benefit, before any positive intervention effect can be determined.

Hepatic‐related mortality

Meta‐analysis showed that hepatic‐related mortality (caused by hepatorenal syndrome) was reduced in the pentoxifylline group compared to the control group, and this result was statistically significant (Analysis 2.1). The result can be expressed as a reduction in the risk of hepatic‐related mortality estimated to be 60%. Although the statistical heterogeneity of the analysis was high, I² = 22%, there was no discrepancy in the results using either the fixed‐effect or random‐effects model (Analysis 2.2). A fourth included trial reported on hepatorenal syndrome, but did not give any data; therefore, this trial has not been included in the meta‐analysis (McHutchison 1991). The trial reports a statistically significant decrease in hepatorenal syndrome in the pentoxifylline group compared to control (McHutchison 1991), agreeing with the meta‐analysis here.

The biochemical parameters, which indicate kidney function (blood urea nitrogen and serum creatinine), are both reduced in the pentoxifylline group compared to the control group (Table 2 and Analysis 5.1) and both results are statistically significant. The trial sequential analysis of the data on serum creatinine shows firm evidence of a decrease in serum creatinine in the pentoxifylline group compared to control. Although the prognostic role of serum creatinine for hepatorenal syndrome is widely accepted, the data presented in our review are not sufficient firm evidence to support that pentoxifylline reduces hepatorenal syndrome. The data can, however, generate the hypothesis for a randomised clinical trial to investigate the effect of pentoxifylline on hepatorenal syndrome and examine the validity of serum creatinine as a surrogate marker for the potential effect of pentoxifylline on hepatorenal syndrome.

Trial sequential analysis was also performed on the data for hepatic‐related mortality (all occurrences caused by hepatorenal syndrome) (Figure 4). The trial sequential analysis showed that the z‐curve (blue curve) does not cross the trial sequential monitoring boundary (red curve), implying that there is no firm evidence for an effect of 20% risk ratio reduction (RRR). The results of the trial sequential analysis does not support that there is firm evidence for the intervention effect suggested by the meta‐analysis; however, it is a more robust result because it is adjusted for multiple testing on accumulating data, so reduces the risk of random error.

The trial sequential analysis was adjusted for heterogeneity, resulting in a larger required information size than currently accrued. The total number of participants contributing to these analyses is only 182, whereas the required information size is 1636 participants. This highlights the risk of random error in the results from the meta‐analysis, which only includes data from 182 participants. There is no firm evidence that pentoxifylline reduces hepatorenal syndrome in participants with alcoholic hepatitis (Figure 4).

Adverse events

Only one trial provided data on adverse events (Akriviadis 2000); there were a total of 33 occurrences of adverse events in the pentoxifylline group compared to 15 occurrences in the control group. The adverse events ranged in their seriousness (Table 3). No pre‐planned analysis could be performed on these data as they are count data. However, we did perform post‐hoc analysis on the number of participants who withdrew from the trial due to adverse events as these data are dichotomous (Table 1). The data show a statistically significant increase in the number of withdrawals due to adverse events in the pentoxifylline group than in the control group (Table 1). Post‐hoc analyses can be biased, as they may be driven by the statistical significance of the results. The reason for our post‐hoc analysis was that the data in the Akriviadis 2000 trial were dichotomous, related to adverse events, and lent themselves to our statistical analysis.

There were 13 reports of epigastric discomfort or pain with or without vomiting in the pentoxifylline group and five reports in the control group (Table 3). These data may suggest that pentoxifylline causes increased epigastric discomfort or pain without vomiting in participants with alcoholic hepatitis. This is in agreement with what is known regarding xanthine derivatives (of which pentoxifylline is one), ie, that they cause adverse effects such as epigastric pain and other gastric problems (Patient 2008).

The total number of occurrences of adverse events, the number of withdrawals due to adverse events, and the reports of epigastric pain, all indicate an increase of serious and non‐serious adverse events in the pentoxifylline group compared to the control group. The main caveat to be entered with respect to these adverse event data is that only one trial reported on adverse events. Only a trend in increased adverse events in the pentoxifylline group can be noted. Data on adverse events from the other trials are necessary to evaluate any possible harmful effect of pentoxifylline in participants with alcoholic hepatitis.

Tumour necrosis factor

A caveat should be entered here, as these analyses were not predefined in the protocol, ie, it was a post‐hoc analysis. We feel that analysing TNF levels is a valid outcome measure as pentoxifylline is proposed to be an inhibitor of TNF ligand gene transcription (Strieter 1988), although this is currently under debate (Lucey 2009).

Meta‐analysis of the data from McHutchison 1991 and Akriviadis 2000 shows a statistically significant increase in plasma TNF (tumour necrosis factor) concentration in the pentoxifylline group (Analysis 6.1). Trial sequential analysis on these data also indicates that TNF concentration is higher in the pentoxifylline group compared to control, but the trial sequential monitoring boundary is not crossed, so the evidence is not firm (Figure 6). The direction of these results seems to be in contrast to the proposed TNF‐inhibitory properties of pentoxifylline. An increase of 4 pg/ml of TNF indicates the activation of a stress response, but this may not be clinically significant and cannot be used as a clear surrogate outcome for either benefit nor harm. Furthermore, two caveats should be entered here, firstly, the data from the Akriviadis 2000 trial are not clearly reported. Neither the raw data, nor the unit of standard error were reported. Secondly, in the McHutchison 1991 trial report, there was a baseline imbalance in TNF levels, with a higher baseline level in the pentoxifylline group. No other laboratory parameters had a baseline imbalance, suggesting that randomisation had been successfully performed and that TNF levels may have been unequal due to chance alone. The difference in the TNF levels before and after the pentoxifylline regimen showed a greater increase in TNF levels in the control group than the pentoxifylline group which suggests that pentoxifylline actually suppressed an increase in TNF levels. In light of such inaccuracies, we do not make conclusions about the effect of pentoxifylline on TNF levels.

Limitations of this review

The limitations of this review include the high proportion of trials with a high risk of bias. Of the five included trials, four were assessed as having a high risk of bias (Figure 1). As such, the statistical analyses on these data are based primarily on high risk of bias trials (Figure 2). Assigning a risk of bias to a randomised clinical trial follows a judgement based on the information available from the publication. The four high risk of bias trials were published as abstracts only; therefore, the judgement is primarily on how well the trial was reported rather than a judgement of the design and conduct of the trial itself. A more detailed description of these trials (possibly a full text publication) may lead to future re‐classification of their risk of bias.

Another limitation of this review is that all five included trials had a small sample size, with an average of 67 participants with alcoholic hepatitis. Small trials have less power, meaning that there is less chance of detecting a small but true effect as statistically significant (Kjaergard 2001). Small trials are also prone to increased risk of random error, and meta‐analysis, which combines data from small trials may yield false results (Brok 2008). This is one reason that we also analysed the data using trial sequential analysis.

Trial sequential analysis

Random error is the probability of observing a result by chance. Type I error is the risk of observing false positive results and type II error is the risk of observing false negative results. Random error can lead to false conclusions about the intervention effect. Random error can occur in data from individual trials and in data from meta‐analysis. The risk of random error is higher when data come from small information sizes (or 'sample sizes' for individual trials), so information sizes need to be sufficiently large in order to reduce the risk of random error and increase the chance of observing a true intervention effect (Brok 2008; Wetterslev 2008).

The risk of random error in meta‐analysis is potentially further exacerbated when one performs, formally or informally, statistical analyses repeatedly as new trials provide additional data; this occurs whenever new trial data become available and when meta‐analyses are updated. The benefit of using trial sequential analysis is that it controls for such an exacerbation of random error, which a traditional cumulative meta‐analysis does not (Brok 2008a).

Elements of a trial sequential analysis include an information size, trial sequential monitoring boundaries (red curves), and a cumulative z‐curve (blue curve). The information size is the number of participants needed in the analysis to detect or reject a certain intervention effect and is calculated based on an a priori, realistic intervention effect (here 20% risk ratio reduction) and a risk of type I error of 5% and a power of 80%. The trial sequential monitoring boundaries are based on the required information size and on the a priori intervention effect (Thorlund 2008). The cumulative z‐curve (blue curve) represents the estimated intervention effect generated from the data from the trials. If the cumulative z‐curve (blue curve) crosses the trial sequential monitoring boundary (red curve), then a sufficient level of evidence is reached and no further trials are needed. However, if the z‐curve does not cross the boundary, then there is insufficient evidence to reach a conclusion, which is what we see in this review for the outcome measures all‐cause mortality and hepatorenal syndrome (Brok 2008a). Additionally, trial sequential analysis provides us with important information regarding the need for additional trials and the required information size.

In this Cochrane review we used trial sequential analysis to analyse data on all‐cause mortality and on hepatorenal syndrome. We applied trial sequential monitoring boundaries according to an information size suggested by an a priori risk ratio reduction (RRR) of 20% (Thorlund 2008). The current information size is 336 participants from all five trials, with 102 participants coming from the single low risk of bias trial. At least 800 participants may be needed in order to detect or reject a risk ratio reduction (RRR) of 20% in the analyses. Therefore, our current analyses, being mere interim‐analyses in this perspective, cannot reach firm conclusions.

The main findings of this Cochrane systematic review are that there is a trend in reduction of all‐cause mortality, reduction of hepatorenal syndrome, and an increase in adverse events in participants who received pentoxifylline; but firm evidence for this is still absent. Data from low risk of bias trials, providing a large information size, are needed in order to draw firm conclusions from the data. Such data can then be added to the trial sequential analyses. Trial sequential analysis controls for the worst case scenario of increased risk of random error whenever you add new trial data to the existing data for an updated analysis.

One trial that was identified in our search of clinicaltrials.gov is classified in this review as an ongoing study (NCT00205049). The final collection of primary outcome measures was July 2008; our search strategy was performed in August 2009 and did not identify a corresponding publication. We await the publication of these results in order to update our analysis.