Summary of main results

We identified two RCTs for iMRI (Kubben 2014; Senft 2011), one for fluorescent‐guided surgery with 5‐ALA (Stummer 2006), and one assessing neuronavigation using standard preoperative MRI sequences (Willems 2006). Formal meta‐analysis was not possible due to the different comparisons and variability in the control arm population between trials. We were therefore limited to performing a narrative analysis of the included trials.

Two trials demonstrated a benefit for intraoperative imaging technology (iMRI and 5‐ALA, respectively) in terms of extent of resection (the primary outcome) (Senft 2011; Stummer 2006). Overall survival data were available for 5‐ALA only; there was no clear evidence that 5‐ALA improved overall survival. Data for PFS were only available for two trials, and were not available in the format specified (hazard ratios and their variance). Nevertheless, there was a suggestion that 5‐ALA increased PFS compared with standard surgery. Quality of life data were only reported in a single trial, and there was significant attrition and reporting bias. Adverse event reporting varied considerably between trials but in general was poorly performed. With 5‐ALA, it appears that neurological deterioration is more common after fluorescence‐guided surgery. The studies that reported on this effect noted that it occurred mainly among those with fixed deficits and early after surgery, but there was subsequently a trend towards recovery (Stummer 2006). Other adverse events appeared to be rare and similar in frequency between study arms.

To supplement the main systematic review of effects, we sought to identify cost analyses and economic evaluations that compared the interventions with each other or between different variants of the same intervention. A search of MEDLINE and Embase identified six such studies (Eljamel 2016; Esteves 2015; Hall 2003; Kowalik 2000; Makary 2011; Schulder 2003; Slof 2015) (one study was reported in two papers ‐ Hall 2003; Kowalik 2000).

Of the six studies, three studies compared iMRI to conventional surgery (Kowalik 2000; Makary 2011; Schulder 2003); two compared 5‐ALA with white light surgery (Esteves 2015; Slof 2015); and one compared conventional, 5‐ALA, fluorescein, ultrasound, and iMRI surgery (Eljamel 2016). Three studies were conducted in the USA (Kowalik 2000; Makary 2011; Schulder 2003), one in Portugal (Esteves 2015), and one in Spain (Slof 2015), and for one it was unclear (but was probably the USA) (Eljamel 2016). Four studies were based on non‐randomised retrospective comparative cohorts (Kowalik 2000; Makary 2011; Schulder 2003; Slof 2015); one was based on a review and pairwise meta‐analyses (Eljamel 2016), and one used data from a trial and retrospective cohorts (Esteves 2015). The cohort studies all involved fewer than 100 participants, except for the study by Slof 2015, which included 254 participants who received 5‐ALA and 120 who received white light surgery. All the studies except one (Esteves 2015), which integrated data using a Markov model, were based on comparisons of individual patient level data.

In terms of costs, what costs were included and over what time horizon varied markedly. Only one study considered costs over the patient lifetime (Esteves 2015), and one only considered the drug cost (Slof 2015). The other studies considered costs incurred in hospital for the index surgery. Costs were reported in US dollars in four studies (Esteves 2015; Kowalik 2000; Makary 2011; Schulder 2003), but the price year (2005/6) was stated only in one study (Makary 2011). The other two studies reported costs in Euros, and the price year was 2012 in one study, Esteves 2015, and not stated in the other (Slof 2015). Two studies were cost analyses only (Kowalik 2000; Schulder 2003). Effects were resection rates (Eljamel 2016), resection‐free years (Makary 2011), quality‐adjusted life years (Eljamel 2016; Esteves 2015; Slof 2015) PFS (Esteves 2015), and life years (Esteves 2015).

For the comparison of iMRI with conventional surgery, two studies reported a potential cost saving driven by reductions in length of stay (Kowalik 2000; Schulder 2003), and third study reported lower mean costs that were not statistically significant (Makary 2011). The one cost‐effectiveness analysis reported a longer interval to resection (20.1 versus 6.7 months; P = 0.02); further results suggested iMRI was more cost‐effective in terms of cost per resection‐free years. Another study reported that iMRI was the most costly of conventional, 5‐ALA, fluorescein, and ultrasound‐assisted surgery (Eljamel 2016). iMRI was the least cost‐effective, but the results could not be replicated from the data presented in the study. Estimates of cost‐effectiveness (and cost over a longer follow‐up) need to considered in the light of the very limited evidence for iMRI where there is a benefit shown in terms of extent of resection but no evidence in the review of clinical effectiveness on overall survival.

For the comparison of 5‐ALA and standard surgery, 5‐ALA was on average more costly in both studies, but results in more quality‐adjusted life years over the patient lifetime or over the time to progression of disease (Esteves 2015; Slof 2015). In both cases the study authors concluded that the extra costs were worth the extra quality‐adjusted life years and that these conclusions were consistent over all sensitivity analyses conducted. These findings of extra effectiveness in the economic studies need to be considered in context of the findings of the review of the best available clinical effectiveness data summarised above. National Institute for Health and Care Excellence (NICE) guidance on the cost‐effectiveness of 5‐ALA is expected to be published in June 2018.

Overall completeness and applicability of evidence

All the identified trials included highly selected participants in specialised centres, and the applicability of these findings to a more general population needs to be carefully considered. Participants included int he trials tended to be generally young and of good performance status. In addition, most trials also clearly specified the types of tumours that were to be included, and would not have randomised those patients with eloquent tumours or where a complete resection was not feasible. Potentially those enrolled in one of the iMRI trials (Senft 2011) were likely to have more resectable or less eloquent tumours than those in the 5‐ALA trial (Stummer 2006), given the far higher resection rates in both arms of the iMRI study (96% iMRI and 68% control versus 65% 5‐ALA and 36% control).

The majority of included trials only enrolled participants with probable high‐grade glioma. We identified no RCTs for ultrasound‐guided surgery, which may reflect the less widespread application of this particular technology. There are theoretical advantages to this technology, such as relative affordability, repeatability, and possibly better sensitivity in low‐grade tumours than the other included intraoperative imaging modalities. Nevertheless, it currently does not have the same evidence base as other intraoperative imaging modalities to recommend its use in routine clinical practice.

Quality of the evidence

It is clearly feasible to perform RCTs for new surgical interventions, and it appears now to have become standard practice to perform an RCT for assessing new intraoperative imaging technologies. The openness of major centres to enrolling participants in RCTs to provide clear outcome data is a major step forward in neuro‐oncology. Some aspects of the included trials were at low risk of bias, such as randomisation methods and blinded, objective reporting for extent of resection. However, the overall the risk of bias was high, and there were consistent concerns with stopping trials early and the role of industry involvement (summary of findings Table 1; summary of findings Table 2; summary of findings Table 3).

Extent of resection was the primary outcome for all of the included trials. This has the advantage of being the outcome most directly influenced by intraoperative imaging. However, there is still no evidence from RCTs that resection (either total or less than total) improves outcomes for high‐grade glioma over biopsy alone (Hart 2011). Subgroup analyses, particularly for the 5‐ALA trial (Stummer 2006), have shown that those participants that have a complete resection of all contrast‐enhancing tumour survive longer than those with residual tumour (Pichlmeier 2008). Studies of chemotherapy have also found that those without residual tumour survive longer (Stupp 2005). While this is not direct evidence in favour of complete resection, but rather a post hoc non‐randomised subgroup analysis, it is becoming increasingly apparent that a complete tumour resection is desirable, particularly when it can be achieved safely. Precisely how much a complete resection contributes towards the overall outcome is unclear. New methods of imaging (e.g. amino acid positron emission tomography) have found that tumours frequently extend out from the contrast‐enhancing margin on MRI (Miwa 2004). However, validation of this approach has yet to be established, and the need for a cyclotron makes widespread application and testing a challenge in the UK, therefore MRI in assessing residual tumour remains the current standard of care.

After extent of resection, studies tended to focus on PFS rather than overall survival. There are certain advantages to this in that possibly fewer participants are required and the results may be available sooner. Additionally, it may provide a more direct assessment of the effect of the primary intervention that is not confounded by subsequent therapy. However, it can be argued that overall survival should still remain the main outcome of interest. Firstly, survival is so short in high‐grade glioma that the practical benefits of assessing PFS are less relevant. Secondly, assessment of PFS can be more subjective, and is critically dependent on the timing and interpretation of imaging, which can often be complicated (Wen 2010).

Quality control for surgical neuro‐oncology trials is an emerging area (GNOSIS 2007). Standardisation of reporting is required to allow clear comparisons between trials in meta‐analyses. Detailed reporting is required for tumour location with regard to eloquent brain; operative technique used; postoperative imaging protocol; assessment of extent of resection; and recording of adverse events (including total numbers of events, total number of participants at risk, number of participants with multiple events; severity, timing, and outcome of events, i.e. resolution or persistence of neurological deficits).

Potential biases in the review process

We took multiple steps in the review process to minimise bias, including double independent literature sift and data extraction, not pooling results due to heterogeneity, and using strict inclusion criteria. Overall, these steps acted to minimise bias and restrict the review to the best available evidence. Notably this led to one trial that was titled as an RCT and included in an earlier version of this review being excluded. Specificially, the lead author stated that randomisation was not strict, surgeons were aware of allocations prior to enrolment, and that bias of participant allocation was inevitable. In a previous Cochrane Review (Barone 2014), this study was included to allow open discussion of its methodology, while for this review we felt it more appropriate to concentrate on the highest‐quality evidence in order to generate the most robust findings.

Notably, the majority of trials identified through the search strategy were not RCTs. It could be argued that excluding this volume of data biases our review and that it would be more appropriate to consider a Cochrane Review of non‐randomised studies (NRS). However, the issue of selection bias is critical, particularly in surgical trials. Participants enrolled in a NRS are likely to have a better prognosis than a control population, and it is impossible to accurately account for this bias without using randomisation. It would therefore not be clear what benefit intraoperative imaging had on the overall outcome. Meta‐analysis of RCTs remains the most reliable way of assessing the benefits of specific intraoperative imaging modalities. However, NRS may also have a role, particularly regarding technology development and reporting of adverse events.

This review included two specific groups of technologies, those that used imaging obtained intraoperatively and those that used imaging obtained preoperatively for use in an intraoperative manner. We felt that both methods were suitable for comparison, as the goals are similar: namely, to achieve maximal safe resection via the application of surgical technology. A major concern with preoperative imaging is intraoperative brain shift, whereby anatomical localisation is affected by events that occur during surgery (e.g. anaesthesia, brain retraction, tumour resection, dural opening, and cerebrospinal fluid drainage). Imaging obtained intraoperatively can theoretically account for brain shift and allow more accurate navigation than imaging obtained preoperatively. In this review we found that a single trial did not demonstrate an effect for intraoperative imaging utilising preoperatively acquired data (Willems 2006).

Another technique that is commonly used in neuro‐oncology surgery is awake craniotomy. This is often perceived as a technology to make surgery safer by allowing intraoperative mapping of eloquent brain. It is not typically regarded as a technique to maximise extent of resection and was therefore not included in this review.

We did not subject these studies to critical appraisal, and we do not attempt to draw any firm or general conclusions regarding the relative costs or efficiency of the interventions being compared. For the comparison of iMRI surgery with conventional surgery, it is clear that the available economic evidence is, at best, equivocal. For the comparison of 5‐ALA with white light surgery, the available economic evidence indicates that, from an economic perspective, use of 5‐ALA could be a promising strategy but effectiveness data used in the economic studies is not consistent with the findings of the review of effectiveness.

Agreements and disagreements with other studies or reviews

We are not aware of any other similar reviews that compare all the different types of intraoperative imaging or other interventions to maximise the extent of resection in neuro‐oncology. Currently, there are no national guidelines appraising the use of any of the technologies, for example by NICE, but guidance on 5‐ALA and iMRI is expected in 2018 when the NICE guidelines for the management of primary brain tumours are published. Many of the trials are relatively recent and appraisal is often limited to a linked editorial. In addition, many of the techniques have only been used in specialised trial centres, and real‐world experience is limited. Further prospective data reporting such real‐world experience would help inform future clinical guidelines and NHS (National Health Service) policy by reporting data on patients who are unsuitable for RCTs, for example due to co‐morbidities.

An interim analysis of an on‐going trial of iMRI is broadly in agreement with the findings of this review (Wu 2014). This reported outcomes on 114 out of a projected 304 participants. Complete resection was achieved in 86% of the iMRI arm versus 53% of the control arm. There was no difference in AEs or PFS while OS was not reported.