Outcome 

Gain 3+ lines

Loss 3+ lines

Gain 3+ lines

Loss 3+ lines

Study

antiangiogenic drug

6 to 12 months

2 years

Soheilian 2007

bevacizumab

yes

yes

yes

yes

Ahmadieh 2008

bevacizumab

E

E

E

E

BOLT

bevacizumab

yes

yes

NA

NA

Macugen 2005

pegaptanib

yes

E

NA

NA

Macugen 2011

pegaptanib

yes

yes

yes

yes

DRCRnet

ranibizumab

yes

yes

yes

yes

READ2

ranibizumab

yes

E

NA

NA

RESOLVE

ranibizumab

yes

yes

NA

NA

RESTORE

ranibizumab

yes

yes

NA

NA

RISE‐RIDE

ranibizumab

E

E

yes

yes

DA VINCI

aflibercept

yes

yes

NA

NA

study

follow‐up

sham

ranibizumab

bevacizumab

pegaptanib

laser

antiVEGF

+ laser (prompt)

ranibizumab

+ laser (deferred)

aflibercept 2 mg PRN§

Macugen 2005

36 weeks (reported at 30 weeks)

4.5 (1.5)

5 (1.2)

Soheilian 2007

2 years

3.1 (1.6)

1 (0.1)

Ahmadieh 2008

24 weeks

3

3

DRCRnet

1 year

8 (7,11)*

9 (7,11)*

DRCRnet

year 2 only

2 (0,4)*

3 (1,7)*

RESTORE

1 year

7 (2.81)

7.3 (3.22)

6.8 (2.95)

RESOLVE

1 year

8.9 (3.5)

10.2 (2.5)

READ2

1.5 years

5.3

4.4

2.9

BOLT

1 year

9 (8,9)*

3 (2, 4)*

Macugen 2011

1 year

8.4 (1.4)

8.3 (1.7) 

Macugen 2011

2 years

12.9 (4.4)

12.7 (4.6)

RISE‐RIDE (two studies)

2 years

20 (7.5)

20.8 (7.1) 

20.9 (6.3) 

21.9 (5.8)

DA VINCI

1 year

7.4 (3.19)

 Cut‐off

Studies

Patients

Control

Absolute effect

RR

NNT

pegaptanib versus sham (1 year)

Gain 3+

2

346

9%

+7% (0, +20%)

1.79 (1.01, 3.16)

13 (5, 1056)

Gain 2+

2

346

17%

+19% (+7%, +36%)

2.11 (1.43, 3.09)

5 (3, 14)

Stable

2

346

78%

Loss 2+

2

346

12%

‐7% (‐1%, ‐9%)

0.44 (0.21, 0.92)

15 (11, 106)

Loss 3+

1

260

9%

‐5% (+2%, ‐7%)

0.43 (0.16, 1.21)

20 (14, 55)

ranibizumab versus sham (1 year)

Gain 3+

1

151

10%

+22% (3%, +68%)

3.17 (1.32, 7.62)

5 (1, 31)

Gain 2+

1

151

18%

+42% (+15%, +93%)

3.31 (1.80, 6.09)

2 (1, 7)

Stable

1

151

57%

Loss 2+

1

151

24%

‐20% (‐11%, ‐23%)

0.20 (0.07, 0.54)

5 (4, 9)

Loss 3+

1

151

20%

‐18% (‐10%, ‐20%)

0.14 (0.04, 0.50)

6 (5, 10)

bevacizumab versus laser (1 year)

Gain 3+

2

167

10%

+15% (2%, +42%)

2.52 (1.20, 5.29)

7 (2, 51)

Gain 2+

2

167

11%

+22% (+6%, +56%)

3.06 (1.54, 6.05)

4 (2, 17)

Stable

2

167

64%

Loss 2+

2

167

25%

‐20% (‐11%, ‐23%)

0.19 (0.07, 0.52)

5 (4, 9)

Loss 3+

2

167

22%

‐17% (‐11%, ‐21%)

0.16 (0.05, 0.51)

5 (5, 9)

ranibizumab versus laser (1 year)

Gain 3+

1

225

8%

+21% (6%, +48%)

3.51 (1.78, 6.92)

7 (3, 34)

Gain 2+

1

225

15%

+32% (+14%, +60%)

3.06 (1.93, 4.87)

5 (2, 14)

Stable

1

225

72%

Loss 2+

1

225

13%

‐9% (‐3%, ‐12%)

0.27 (0.09, 0.80)

11 (9, 39)

Loss 3+

1

225

8%

‐7% (‐1%, ‐8%)

0.11 (0.01, 0.83)

14 (12, 72)

ranibizumab plus prompt laser versus laser (1 year)

Gain 3+

3

786

12%

+15% (8%, +25%)

2.29 [1.67, 3.13)

7 (4, 13)

Gain 2+

3

786

23%

+26% (+17%, +37%)

2.14 [1.73, 2.64)

4 (3, 6)

Stable

2

708

63%

Loss 2+

2

708

13%

‐10% (‐6%, ‐11%)

0.27 (0.14, 0.51)

10 (9, 16)

Loss 3+

2

708

8%

‐6% (‐3%, ‐7%)

0.29 [0.13, 0.67)

17 (14, 38)

aflibercept (2PRN§) versus laser (1 year)

Gain 3+

1

89

11%

31% (6%, 92%)

3.72 (1.52, 9.08)

3 (1, 17)

Gain 2+

1

89

30%

33% (8%, 74%)

2.11 (1.26, 3.51)

3 (1, 13)

Stable

1

89

52%

Loss 2+

1

89

18%

‐14% (‐17%, 2%)

0.24 (0.05, 1.09)

7 (6, 61)

Loss 3+

1

89

14%

‐13% (‐14%,4%)

0.08 (0.00, 1.30)

8 (7, 24)

study

treatment

gain 2+

gain 3+

total

BOLT

laser

3

2

38

BOLT

bevacizumab

13

5

42

DRCRnet

laser

81

43

293

DRCRnet

ranibizumab/laser

95

57

187

Macugen 2005

sham

4

3

42

Macugen 2005

pegaptanib

15

8

44

Macugen 2011

sham

25

13

127

Macugen 2011

pegaptanib

49

22

133

READ2

laser

2

0

38

READ2

ranibizumab

17

8

37

READ2

ranibizumab/laser

12

3

40

RESOLVE

sham

9

5

49

RESOLVE

ranibizumab

62

33

102

RESTORE

laser

17

9

110

RESTORE

ranibizumab

43

26

115

RESTORE

ranibizumab/laser

51

27

118

Soheilian 2007

laser

6

6

43

Soheilian 2007

bevacizumab

16

16

44

DA VINCI

laser

28

5

44

DA VINCI

aflibercept*

13

19

45

study

follow‐up

sham

ranibizumab

bevacizumab

pegaptanib

laser

ranibizumab + laser (prompt)

ranibizumab + laser (deferred)

aflibercept 2 mg PRN§

Macugen 2005

36 weeks

0/42

1/44

Soheilian 2007 *

2 years

0/48

0/48

Ahmadieh 2008 (#)

24 weeks

0

0

DRCRnet

2 years

1/293

2/187

2/188

RESTORE

1 year

0/115

0/110

0/120

RESOLVE

1 year

0/49

2/102

READ2

2 years

BOLT

1 year

0/42

0/40

Macugen 2011

2 years

0/127

0/133

RISE‐RIDE

2 years

0/250

3/250*

DA VINCI

1 year

0/44

1/45

study

follow‐up

sham

ranibizumab

bevacizumab

pegaptanib

laser

ranibizumab + laser (prompt)

ranibizumab + laser (deferred)

aflibercept 2 mg PRN§

Macugen 2005

36 weeks

0/42

0/44

Soheilian 2007

36 weeks

0/48

0/48

Ahmadieh 2008 (#)

24 weeks

0

0

DRCRnet

2 years

1/293

0/187

1/188

RESTORE

1 year

0/115

0/110

0/120

RESOLVE

1 year

1/49

0/102

READ2

2 years

BOLT

1 year

0/42

0/40

Macugen 2011

2 years

0/127

0/133

RISE‐RIDE

2 years

1/250

1/250*

DA VINCI

0/44

0/45

study

follow‐up

sham

ranibizumab

bevacizumab

pegaptanib

laser

ranibizumab + laser (prompt)

ranibizumab + laser (deferred)

aflibercept 2 mg PRN§

Macugen 2005

36 weeks

0/42

0/44

Soheilian 2007 (*)

36 weeks

0/48

0/48

Ahmadieh 2008 (*)

24 weeks

1

0

DRCRnet

2 years

 NA

 NA

 NA

 NA

RESTORE

1 year

23/115

15/110

17/120

RESOLVE

1 year

8/49

14/102

READ2

2 years

BOLT

1 year

3/42

7/38

Macugen 2011

2 years

 27/142

 28/144

RISE‐RIDE (#)

2 years

25/250

22/250*

DA VINCI

1 year

10/44

6/45

study

follow‐up

sham

ranibizumab

bevacizumab

pegaptanib

laser

ranibizumab + laser

aflibercept 2 mg PRN§

Macugen 2005

36 weeks

0/42

0/44

Soheilian 2007 (*)

36 weeks

0/44

2/43

Ahmadieh 2008 (*)

24 weeks

1/?

0/?

READ2

6 months

 0/39

 0/40

 1/38

DRCRnet

2 years

17/130

25/375

RESTORE

1 year

6/115

1/110

1/120

RESOLVE

1 year

2/49

3/102

BOLT

1 year

0/42

1/38

Macugen 2011 (#)

2 years

 9/142

 7/144

RISE‐RIDE (#)

2 years

13/250

18/250

DA VINCI

1 year

1/44

1/45

study

follow‐up

sham

ranibizumab

bevacizumab

pegaptanib

laser

ranibizumab + laser

aflibercept 2 mg PRN§

Macugen 2005

36 weeks

0/42

0/44

Soheilian 2007 (*)

36 weeks

0/44

2/43

Ahmadieh 2008 (*)

24 weeks

1/?

0/?

READ2

6 months

 0/39

 0/40

 1/38

RESTORE

1 year

2/115

2/110

2/120

RESOLVE

1 year

0/49

1/102

BOLT

1 year

0/42

0/38

DRCRnet

2 years

8/130

13/375

Macugen 2011 (#)

2 years

5/142

 4/144

RISE‐RIDE (#)

2 years

3/250

11/250

DA VINCI

1 year

1/44

0/45

Form of economic analysis

Cost‐effectiveness analysis (cost per unit line of vision saved).

Population

Patients with macular oedema due to diabetic retinopathy enrolled in phase I/III RCTs

Interventions

Laser grid photocoagulation, intravitreal triamcinolone, dexamethasone implant, pegaptanib, bevacizumab, ranibizumab

Comparators

Not defined a priori

Perspective, time horizon

6 to 12 months in included studies, life‐long horizon.

Modelling used and Key assumptions

Quote: “Most index studies offered 1‐year follow‐up data, but resource use data were conservatively extrapolated to 1 year when shorter, assuming a durable result at 1 year, and until death for line‐year calculations. […] By design, costs were generally underestimated and benefits were overestimated for more expensive modalities; the converse was applied to less expensive modalities.

Quote: “By using the estimate of lines saved, cost/line saved was calculated. The mean age of each study cohort was used to calculate a life expectancy, by consulting the actuarial table of the Social Security Administration, and to calculate cost/line years saved.”

Effectiveness data

Single randomised trial  comparing any listed intervention to any control. Non comparative studies also used (natural course of all conditions, outcome of bevacizumab treatment for DMO).

Health state valuations (utilities)

Cost‐benefit analysis, but [quote] “A cost utility analysis was performed by using the lines saved estimate and ascribing 0.03 marginal quality‐adjusted life years (QALYs) for each line gained, a reasonable estimate from previously published data in the VA range of the index study cohorts, albeit for a “better eye,” which was not usually the case in the index study cohorts.”

Unit cost data, price year, resource use data

Direct cost only, including follow‐up visits, OCT, fluorescein angiography, treatment. Quote: “The Medicare allowable amounts for hospital‐based use (in South Florida as of June 1, 2010) were applied as costs (including hospital facility usage allowable) for the hypothetic 1 year of treatment.

All costs were calculated in US dollars; present value adjustments were not included in cost calculations.”

Discounting

Not mentioned.

Results and sensitivity analyses

Not reported because we judged study quality was insufficient.

yes

no

quote/comment

Item

1. Is the study population clearly described?

Y

 Quote: "Purpose: To relate costs and treatment benefits for diabetic macular edema, branch retinal vein occlusion, and central retinal vein occlusion."

Comment: very general statement including macular oedema due to several diseases.

2. Are competing alternatives clearly described?

 N

Quote: "For each of the 3 diagnostic groups, an index study (usually a collaborative, randomized, controlled study with 6‐month to 1‐year follow‐up information) was chosen for natural history, focal laser, intravitreal triamcinolone acetonide, intravitreal dexamethasone implant, pegaptanib, bevacizumab, and ranibizumab, and a surgical series for DMO only".

Comment: treatments listed, but reasons for this choice are not reported. A discussion of the appropriate comparator was not made.

3. Is a well‐defined research question posed in answerable form?

 Y

 Quote: "Purpose: To relate costs and treatment benefits for diabetic macular edema, branch retinal vein occlusion, and central retinal vein occlusion."

Comment: same quotation as above, very general, broad purpose economic study.

4. Is the economic study design appropriate to the stated objective?

 N

Comment: calculation of cost per line of vision saved for each treatment, as well as of QALYs, made with no probabilistic modelling of transition probabilities between health states, including benefits and adverse events; non probabilistic sensitivity analyses were performed.

5. Is the chosen time horizon appropriate to include relevant costs and consequences?

 Y

Quote: ""Most index studies offered 1‐year follow‐up data, but resource use data were conservatively extrapolated to 1 year when shorter, assuming a durable result at 1 year, and until death for line‐year calculations".

Comment: life‐long time horizon appropriate.

6. Is the actual perspective chosen appropriate?

 ?

 ?

Comment: perspective not declared. Given the measurement of benefits and direct costs, the perspective should be that of the public health care provider.

7. Are all important and relevant costs for each alternative identified?

 N

Comment: Costs of treatment, follow‐up visits, OCT and fluorescein angiography considered, but not cost of adverse events.

8. Are all costs measured appropriately in physical units?

 Y

Quote: "The Medicare allowable amounts for hospital‐based use (in South Florida as of June 1, 2010) were applied as costs (including hospital facility usage allowable) for the hypothetic 1 year of treatment".

9. Are costs valued appropriately?

N

Quote: "By design, costs were generally underestimated and benefits were overestimated for more expensive modalities; the converse was applied to less expensive modalities."

"All costs were calculated in US dollars; present value adjustments were not included in cost calculations." 

Comment: cost assumptions neither justified nor checked in sensitivity analyses; no discount rate applied.

10. Are all important and relevant outcomes for each alternative identified?

 N

Comment: adverse events not considered.

11. Are all outcomes measured appropriately?

 N

 See above.

12. Are outcomes valued appropriately?

 N

 See above.

13. Is an incremental analysis of costs and outcomes of alternatives performed?

 Y

Quote: "By using the estimate of lines saved, cost/line saved was calculated."

Comment: cost and outcomes were combined, however the comparator generating the incremental cost‐effectiveness was insufficiently reported.

14. Are all future costs and outcomes discounted appropriately?

 N

 Comment: no discount rate applied.

15. Are all important variables, whose values are uncertain, appropriately subjected to sensitivity analysis?

 N

 Comment: no sensitivity analysis.

16. Do the conclusions follow from the data reported?

 Y

 Comment: conclusions are coherent with results, but the methodology is inappropriate.

17. Does the study discuss the generalizability of the results to other settings and patient/ client groups?

 N

Quote: "Certainly, selection biases and application differences may apply not only to the index study patient cohorts but also, more important, to the patients actually treated in the clinic on the basis of the results of these studies. Thus, the “real world” differential may be even less. Furthermore, most index studies report up to 1‐year results; the durability and amount of additional costs necessary to maximize longer‐term results are even more conjectural."

Comment: the study acknowledges that included RCTs may not be pragmatic and reflect routine clinical practice.

18. Does the article indicate that there is no potential conflict of interest of study researcher(s) and funder(s)?

 N

Comment: no conflict of interest explicitly declared.

19. Are ethical and distributional issues discussed appropriately?

 N

Quote: "Although physicians have historically been committed to making treatment decisions independently of treatment costs, at least for distinctly important medical problems, the 15‐fold cost differences, the high prevalence of these conditions, and the ongoing nature of treatment make the issue of costs inescapable. It would seem good stewardship for the physician, the traditional gatekeeper of medical care, to be cognizant of the magnitude of the cost differential and to consider it in some manner in formulating treatment regimens. A problem is that there is no policy basis or even short‐term motivation for the physician to act on these findings. This difficult ethical problem is even more difficult for an individual patient, payor, or health policy maker."

Comment: the perspective taken is that of the physician, which does not completely overlap with the perspective of policy‐maker.

Form of economic analysis

Cost‐utility analysis.  

Population

Patients with macular oedema due to diabetic retinopathy enrolled in phase III RCTs RESTORE and DRCRnet. READ2 was excluded because follow‐up was short and treatment schedule was inadequate.

Interventions

Ranibizumab alone or added to laser grid photocoagulation.

Comparators

Laser grid photocoagulation.

Perspective, time horizon

Perspective of health care provider; follow‐up 12‐24 months in included studies,15 years horizon in modelling.

Modelling used and Key assumptions

The key section of the original manufacturer's submission is presented here:

1) transition probabilities between three health states in year 1 were drew from changes in best corrected visual acuity (BCVA) in RESTORE for treated and control patients; treatment effect was modelled as stable in year 2 based on DRCRnet and no treatment effect was assumed thereafter, meaning that BCVA would remain constant and decline at the same rate in the two groups after year 1.

2) to estimate the HRQoL associated with each health state corresponding to vision, EQ‐5D data from RESTORE were transformed to utility values using standard social tariffs and then related to BCVA in the treated eye using linear regression.

The original manufacturer's submission was revised following the Evidence Review Group comments. Manufacturer's revised assumptions are presented here.

1) patients receiving ranibizumab would receive an average of two injections in a third year of treatment and one injection in a fourth year of treatment. In the laser photocoagulation arm, the model assumed once‐yearly treatments for years 3 and 4.

2) following criticisms, the manufacturer provided a scenario analysis, using its revised model, which simulated treatment in both eyes for 35% of people. This analysis assumed that, in people with bilateral disease, both eyes would be treated and monitored at the same visit, with ranibizumab drug and treatment costs doubled. The analysis applied reduced costs associated with severe visual impairment because fewer people would go blind in both eyes. The analysis assumed that treating the second eye would result in utility gains one quarter the magnitude of those achieved by treating the first eye; this is because the HRQoL of people who can see well with both eyes is only a little better than the HRQoL of people who can see well with one eye. The model calculated this figure by applying a 25% uplift to the QALYs generated by ranibizumab.

3) the revised model predicted that 43% of the cohort would remain alive after 15 whereas the original model had suggested that 65% of people would be alive at that time.

4) had concluded that, by assuming people whose BCVA rose to 76 letters or higher would stop receiving ranibizumab, the manufacturer's original model had not reflected likely clinical practice. Acknowledging this view, the manufacturer removed the stopping rule from the base case of its revised model.

5) [quote] "a further series of scenario analyses adopted alternative estimates of utility drawn from various published sources. When utility values from the better seeing eye study by Lloyd et al. were used, the ICER was £24,779 per QALY gained. When utility was estimated according to an equation published by Sharma et al., associating visual acuity in the better‐seeing eye with HRQoL, the ICER was between £12,312 and £12,610 per QALY gained, depending on the version of the equation used. A final analysis adopted utility values estimated in a study by Brazier et al., in which members of the general public valued levels of visual impairment that were simulated by custom‐made contact lenses, using the time trade‐off method. Participants wore the same lenses in both eyes, so the resulting utility values reflected bilateral impairment of vision. This was the source of utility values the Committee had judged most accurately reflected the HRQoL associated with visual impairment in NICE technology appraisal guidance 155. When these values were used in the revised ranibizumab model, the ICER was £23,664 per QALY gained."

Conclusive comments by the Appraisal Commitee are presented here.

Quote: “In summary, the Committee considered that amendments to the manufacturer's model made during the consultation process had resulted in a more robust analysis. However, the Committee believed that the manufacturer's revised base‐case model still provided an inaccurate reflection of likely clinical practice in at least six respects:

• By not accounting for the need to treat both eyes in a large proportion of people with diabetic macular oedema, the manufacturer's base‐case revised model underestimated the benefits and – to a greater degree – the costs of treatments. The manufacturer's scenario analysis simulating treatment in both eyes for 35% of people provided a more realistic reflection of likely clinical practice. This analysis generates an ICER that is almost 50% higher than the revised base case (see section 4.21).

• The range of utility values used in the manufacturer's revised base case was broader than would be expected according to the assumptions of the model. The Committee preferred the manufacturer's scenario analysis adjusting for factors that may influence the relationship between BCVA and HRQoL. This analysis generates an ICER that is more than 10% higher than the revised base case (see sections 4.22 and 4.23).

• The model underestimates the amount of ranibizumab that people with diabetic macular oedema are likely to need over time. Basing the number of injections for year 2 of the model's ranibizumab monotherapy arm on observed experience in DRCR.net overlooks the fact that the trial participants also received laser photocoagulation, which clinicians believe may have a ranibizumab‐sparing effect. The declining number of ranibizumab injections assumed in years 3 and 4 is not evidence‐based, and is unlikely to lead to stable vision during that period, as assumed. It may also be unrealistic to assume that ranibizumab treatment will not continue beyond 4 years. The full impact of these assumptions is uncertain, but a more realistic approach could be assumed to raise the ICER given in the revised base case (see section 4.25).

• The model's assumption that the relative benefit achieved during the treatment phase lasts indefinitely is unrealistic. If NICE technology appraisal guidance 155 is considered a precedent for this approach, then it should be noted that the model in that appraisal had a shorter time horizon, which limited the Committee's uncertainty about extrapolating treatment effects into the future. Shortening the time horizon in the present appraisal to limit uncertainty similarly would result in an ICER that could be expected to be up to 50% higher than the ICER in the revised base case (see section 4.26).

• The model applies unequal assumptions about treatment visits and monitoring visits for people treated with ranibizumab and those treated with laser photocoagulation. When this imbalance is corrected, the ICER in the revised base case rises by 10–25% (see section 4.27).

• If glycaemic control in the modelled population reflected that in people treated in clinical practice, it is very likely that the ICER in the revised base case would rise (see section 4.28)."

Effectiveness data

RESTORE, DRCRnet

Health state valuations (utilities)

See above.

Unit cost data, price year, resource use data

Treatment costs:

• Cost of ranibizumab: £761.20 per injection

• Treatment with both ranibizumab and laser photocoagulation on an outpatient basis: costs £150 per visit.

• Combination therapy: ranibizumab injections and laser photocoagulation at the same visit, cost £184 per visit.

• Visits to monitor patients: £126 each. Patients receiving ranibizumab monotherapy had 12 visits in the first year and 10 visits in the second year; patients receiving combination therapy had 12 visits in the first year and eight visits in the second year; those receiving laser photocoagulation alone in the first and second years, and all patients from the third year onwards, had four visits per year. For the ranibizumab‐containing arms, a visit for treatment was assumed to include monitoring as well. The same assumption was not applied for patients receiving laser photocoagulation alone; that is, patients receiving laser photocoagulation required separate visits for treatment and monitoring.

Costs of low vision:

• Costs associated  with severe vision loss for patients with the lowest BCVA in the treated eye (0–25 or 26–35 letters), regardless of vision in the nontreated eye: £6067 in the first year and  £5936 in subsequent years. These accounted for a range of tems including low‐vision aids, rehabilitation, residential care, district nursing, community care and the cost of treating complications including depression and falls. The manufacturer drew cost data largely from a published costing study of blindness in the UK that focused on people with age‐related macular degeneration (Meads and Hyde 2003), with costs updated or adjusted for inflation as appropriate.

Discounting

Not mentioned.

Results and sensitivity analyses

Results, description of ICER for base‐case scenario and sensitivity analyses:

• Base‐case scenario on the comparison of ranibizumab monotherapy and laser photocoagulation: ICER £30,277 per QALY gained. Subgroup of subgroup of people with central foveal thickness greater than 400 micrometres: ICER £21,418 per QALY gained.

• Treatment in both eyes for 35% of people: ICER £44,400 per QALY gained. Subgroup of people with central foveal thickness greater than 400 micrometres: ICER £35,719 per QALY gained

• Utilities re‐estimated from RESTORE data using an extended model with additional covariates reflecting patient characteristics and risk factors for diabetic complications: ICER £33,857 per QALY gained.

• Utility values from the better seeing eye study by other sources:ICER £12,312 ‐ £12,610 (Sharma), ICER £24,779 (Lloyd), ICER £23,664 (Brazier et al., recommended in NICE guidance 155)

Quote: “The Committee considered that the manufacturer's model, as revised during consultation, resulted in an ICER for ranibizumab monotherapy compared with laser photocoagulation of £30,277 per QALY gained, which was at the upper limit of the range it could consider to represent an effective use of NHS resources. However, the Committee concluded that this revised model underestimated the ICER because, despite improvements made in response to the Committee's comments in the appraisal consultation document, the model remained reliant on several implausible assumptions. Where the effect of using individual alternative assumptions was quantifiable, the ICER rose by an appreciable amount; for example the ICER increased to £44,400 per QALY gained when 35% of people were assumed to be treated in both eyes. The Committee concluded that a model that relied on a combined set of plausible assumptions would be certain to produce an ICER that substantially exceeded the acceptable range. Therefore, ranibizumab could not be recommended as a treatment for people with diabetic macular oedema.

[...]

The Committee heard conflicting evidence about the extent to which bevacizumab is currently used to treat diabetic macular oedema in England and Wales for this indication. It concluded that, although bevacizumab is not in routine use throughout the NHS, it is among the treatments adopted by some clinicians and is funded by some NHS trusts. The Committee was mindful that some consultees and commentators supported the evaluation of bevacizumab as a comparator and others opposed it. Because of this, the Committee agreed that, in order to evaluate bevacizumab as a comparator consider a cost‐effectiveness analysis of ranibizumab compared with bevacizumab. However, because it had not been provided with plausible evidence that ranibizumab represents an effective use of NHS resources when compared with laser photocoagulation, the Committee did not believe that considering evidence about the costs and effects of bevacizumab would alter its decision. For this reason, given the current model and its assumptions, the Committee concluded that it could make recommendations on the use of ranibizumab for diabetic macular oedema without requiring additional evidence comparing it with bevacizumab."

Form of economic analysis

Cost‐utility analysis.  

Population

Patients with macular oedema due to diabetic retinopathy enrolled in phase III RCTs RESTORE and DRCRnet.

Interventions

Ranibizumab alone or added to laser grid photocoagulation.

Comparators

Laser grid photocoagulation.

Perspective, time horizon

12‐24 months in included studies, 15‐year horizon.

Modelling used and Key assumptions

Quote: “The model also assumed the same baseline BCVA distribution [asRESTORE], but excluded patients with BCVA >75 letters, consistent with guidance from the ranibizumab summary of product characteristics that such patients may benefit less from treatment than those with baseline BCVA ≤75 letters. The model framework allocated eight linear health states defined by BCVA in the treated eye using a set of 10‐letter (two‐line) categories. Movement of patients from one health state to another was determined by transition probabilities that depended on the effectiveness of treatment and natural BCVA changes over time

Costs and outcomes were accrued over 3‐month cycles, applying half‐cycle corrections. The time horizon in the base case was 15 years; although a lifetime horizon could be justified given the chronic nature of the condition, we selected a more conservative approach for the base case because of the lack of evidence on long‐term prognosis.

[…] Patients were assumed to receive ranibizumab treatment in year 1 at the frequency observed in RESTORE. In year 2, patients were assumed to need fewer injections, as observed in the Diabetic Retinopathy Clinical Research Network (DRCR.net) protocol I study (which included patients with comparable baseline demographics to RESTORE).1 A proportionately smaller number of monitoring visits was therefore assumed in year 2. After year 2, laser therapy was assumed to be administered as required in all arms, with no further need for ranibizumab; the assumed number of monitoring visits was further reduced accordingly.

The average BCVA achieved in year 1 was assumed to be maintained during year 2, as was observed in the DRCR.net protocol I study.1 After year 2, all arms of the model followed natural disease history based on 4‐year health state transition outcomes modelled from the Wisconsin Epidemiologic Study of Diabetic Retinopathy (WESDR) reports, the Diabetes Control and Complications Trial and UK Prospective Diabetes Study. Transition probabilities were calibrated to adjust for the improvement in diabetes management since the WESDR reports, and predicted that around 30% of patients would be expected to exhibit a worsening in BCVA of at least 10 letters and 20% of patients would show an improvement of at least 10 letters over a 4‐year time horizon. […]

Mortality was estimated by adjusting general UK population death rates according to the increased RR of death in patients with DME. Mulnier et al estimated an increased mortality (HR 1.93) in a UK type 2 diabetes population relative to patients without diabetes, while Hirai et al estimated an HR of 1.27 for death in patients with clinically significant macular oedema (CSME) and diabetes relative to diabetic patients without CSME. We calculated a 2.45 RR of death in a DME population by multiplying these two ratios.

Utility scores were calculated based on patient‐reported outcomes data from RESTORE […], in which patients completed the EuroQoL (EQ‐5D) questionnaire at baseline and months 3, 6 and 12. Individual EQ‐5D health scores were converted into utility scores using preferences from a UK population survey; mean utility scores were calculated for each health state […]. As these states were defined by BCVA in the treated eye, of which 67.2% were the worse‐seeing eye at baseline, this method established an association between utility and BCVA changes in the treated eye.

Health state costs included the costs of treatment and monitoring (Supplementary tables 3–6), and the costs associated with blindness (Supplementary table 2). Treatment costs included the costs of ranibizumab (Novartis UK, personal communication) and its administration, laser therapy and investigative procedures. Monitoring costs, including consultation and procedure costs, were estimated from the UK National Health Service Reference Costs. Costs of blindness included those incurred by the UK National Health Service for items such as low‐vision aids, low‐vision rehabilitation, residential or home care, depression and hip fracture/replacement as listed in the costing study by Meads and Hyde. Where older cost estimates were used, these were inflated to 2010 prices using the Hospital and Community Health Services index.26 The cost of blindness would be incurred only in patients reaching health states with BCVA ≤35 letters (Snellen ≤6/60) in the better‐seeing eye. However, as the study assesses treatment response according to the enrolled eye, the proportion of patients reaching this level within the time horizon of the model is therefore uncertain. As such, the base case model adjusts for the cost of blindness on the basis of treated eyes reaching the BCVA ≤35‐letter threshold. As with other model parameters subject to uncertainty, deviations from this assumption were explored in sensitivity analyses (Supplementary table 7).

[…] An annual 3.5% discount rate was applied for future costs and utilities, consistent with the standard UK approach.

Effectiveness data

RESTORE, DRCRnet

Health state valuations (utilities)

See above.

Unit cost data, price year, resource use data

Cost of treatment

Ranibizumab injection (0.5 mg vial [x1]) ) £742.17, 7 injections year 1, 3 in year 2.  Novartis UK, personal communication.

Laser treatment per session (weighted average of day cases and outpatient procedures for vitreous retinal procedures category 1) £274.19, 2 treatments year1, 1 in year 2. NHS Reference Costs 2008–09 – NHS Trusts and PCTs  combined (unless otherwise stated).

Ophthalmologist visit (weighted first  attendance and follow‐up attendance)  £84.42, 12 visits ranibizumab, 5 visits laser both in year 1 and 2. NHS Reference Costs 2008–09 – NHS Trusts and PCTs  combined (unless otherwise stated)  Additional ophthalmologist visit £73.16.  

Pre‐injection VA and BCVA assessment  (first attendance for ophthalomology non‐consultant‐led, non‐admitted visit)   £83.97 NHS Reference Costs 2008–09 – NHS Trusts and PCTs  combined (unless otherwise stated)

Optometrist visit (follow‐up attendance for  ophthalomology non‐consultant‐led, non‐admitted visit) £60.92, 12 visits ranibizumab, 5 visits laser. NHS Reference Costs 2008–09 – NHS Trusts and PCTs  combined (unless otherwise stated).

VA and BCVA checks £55.59 NHS Reference Costs 2008–09 – NHS Trusts and PCTs.

combined (unless otherwise stated).

Cost of low vision

Low vision aids annual cost £194.16 £ in 33% of blind patients Inflated to base year 2008–09.

Low vision rehabilitation (occupational health therapist): annual cost £221.00 in 11% of blind patients. Section 7.2: NHS community  occupational therapist.

Residential care (homecare) – 30% private payer: annual cost £16,998.80 in 30% blind patients. Section 1.2: Private residential  care for older people: fees (A) only.

Community care £12,064.00 annual cost in 6% of blind patients. £723.84 Section 9.5: Local authority  home care worker.

Depression annual cost: £558.24in 39% blind patients. Inflated to base year 2008–09.

Hip replacement annual cost: £6952.93 in 5% blind patients. Weighted average of major hip.

procedures category – 12B and  12C TPCTEI.

Discounting

3.5%

Results and sensitivity analyses

Base Case: ICER £24,028.

Discount rate 0‐5% (vs 3.5%): ICER £17,051‐27,042.

Time horizon 10‐20 years (vs 15 years): ICER £33,139‐21,343.

Cost of blindness (base £6477) ‐25%‐+25%: ICER £27,907‐20,150.

Long term progression of visual acuity (vs declining) stable or improved: ICER £26,198‐28,4131.

Total n. ranibizumab injections (vs 10) 6‐14: ICER £12,446‐38,836.

Baseline age (vs 63 years) 58 years: ICER £19,259.

Source of utilities (vs RESTORE) Brazier: ICER £21,953.

QUALITY CRITERION: question

Study

Response

Comments

STRUCTURE: Statement of decision problem/objective

Is there a clear statement of the decision problem?

NICE 2011

Yes

Clinical effectiveness and cost effectiveness of ranibizumab monotherapy and ranibizumab in combination with laser photocoagulation compared with laser photocoagulation alone.

Mitchell 2012

Yes

Is the objective of the evaluation and model specified and consistent with the stated decision problem?

NICE 2011

Yes

See above.

Mitchell 2012

Yes

Is the primary decision‐maker specified?

NICE 2011

Yes

Not reported, but models submitted to NICE should take the perspective of the health service provider.

Mitchell 2012

Yes

UK healthcare payer perspective.

STRUCTURE: Statement of scope/perspective

Is the perspective of the model stated clearly?

NICE 2011

Yes

Ranibizumab manufacturer’s submission to NICE including an effectiveness review of two RCTs (RESTORE and DRCRnet), costs of treatment and visits and of some of its complications, as well as costs of low vision. Bevacizumab was not included in the submission but was considered a comparator by the Appraisal Committee.

Mitchell 2012

Yes

Manufacturer’s sponsored study on cost‐effectiveness of ranibizumab adopting similar model boundaries as NICE 2011. Bevacizumab not mentioned as a potential comparator.

Are the model inputs consistent with the stated perspective?

NICE 2011

Yes

See above.

Mitchell 2012

Yes

See above.

Has the scope of the model been stated and justified?

NICE 2011

Yes

See above.

Mitchell 2012

Yes

See above.

Are the outcomes of the model consistent with the perspective, scope and overall objective of the model?

NICE 2011

Yes

Utility‐based health‐related quality‐of‐life associated to visual acuity data from RCTs and costs of disease and treatment.

Mitchell 2012

Yes

STRUCTURE: Rationale for structure

Is the structure of the model consistent with a coherent theory of the health condition under evaluation?

NICE 2011

Yes

Health states and transition probabilities based on changes of best corrected visual acuity changes, although treatment of better‐seeing vs. worse eye or both eyes is a problem.

Mitchell 2012

Yes

Are the sources of data used to develop the structure of the model specified?

NICE 2011

Yes

EQ‐5D data from RESTORE related to visual acuity, NHS reference cost of treatment and visits. Cost of low vision from a published cost study.

Mitchell 2012

Yes

Same as above, plus Wisconsin study (WESDR) data for natural disease history.

Are the causal relationships described by the model structure justified appropriately?

NICE 2011

Unclear

Treatment effect and adverse events are based on RCTs, but costs, and not utility decrement, related to adverse events considered. Although adverse events associated with the treatments were low, unclear if they impact on cost‐effectiveness.

Mitchell 2012

No

Adverse events assumed to have negligible impact on cost‐effectiveness, but no demonstration given.

STRUCTURE: Structural assumptions

Are the structural assumptions transparent and justified?

NICE 2011

No

See Table 12. We accept the criticism of the Appraisal committee, including the fact that treatment in better‐seeing, worse‐seeing eye or both eyes s not considered.

Mitchell 2012

No

Same as for NICE 2011.

Are the structural assumptions reasonable given the overall objective, perspective and scope of the model?

NICE 2011

No

See above.

Mitchell 2012

No

STRUCTURE: Strategies/comparators

Is there a clear definition of the options under evaluation?

NICE 2011

Yes

The manufacturer considered bevacizumab as a potential comparator but concluded that significant methodological and clinical differences between studies precluded a valid analysis.

Mitchell 2012

No

Options under evaluation not discussed.

Have all feasible and practical options been evaluated?

NICE 2011

No

See above. Bevacizumab is used off‐label for treatment of DMO in several countries.

Mitchell 2012

No

See above.

Is there justification for the exclusion of feasible options?

NICE 2011

No

See above.

Mitchell 2012

No

See above.

STRUCTURE: Model type

Is the chosen model type appropriate given the decision problem and specified causal relationships within the model?

NICE 2011

Yes

Appropriate given what reported above.

Mitchell 2012

Yes

STRUCTURE: Time horizon

Is the time horizon of the model sufficient to reflect all important differences between options?

NICE 2011

Yes

15‐year horizon.

Mitchell 2012

Yes

Are the time horizon of the model, the duration of treatment and  the duration of treatment effect described and justified?

NICE 2011

No

The model’s assumption that the relative benefit achieved during the treatment phase lasts indefinitely is unrealistic.

Mitchell 2012

No

STRUCTURE: Disease states/pathways

Do the disease states (state transition model) or the pathways (decision tree model) reflect the underlying biological process of the disease in question and the impact of interventions?

NICE 2011

Yes

Course of visual acuity change after treatment for diabetic macular oedema.

Mitchell 2012

Yes

STRUCTURE: Cycle length

Is the cycle length defined and justified in terms of the natural history of disease?

NICE 2011

Yes

3‐monthly cycles.

Mitchell 2012

Yes

DATA: Data identification

Are the data identification methods transparent and appropriate given the objectives of the model?

NICE 2011

Unclear

A discussion of the choices of the sources is given but a systematic search is not mentioned.

Mitchell 2012

No

No discussion of the choices of the sources.

Where choices have been made between data sources, are these justified appropriately?

NICE 2011

Yes

RESTORE and DRCRnet data used. The choice of excluding RESOLVE and READ2 is discussed.

Mitchell 2012

No

No discussion of the choices of the sources.

Has particular attention been paid to identifying data for the important parameters in the model?

NICE 2011

Yes

Model revised following ERG comments considered all relevant parameters.

Mitchell 2012

Yes

We believe all the important parameters have been considered.

Has the quality of the data been assessed appropriately?

NICE 2011

Yes

Unclear, but NICE comment, quoted above, uses standard quality assessments methods, and assessments correspond to our review.

Mitchell 2012

No

No discussion of data quality.

Where expert opinion has been used, are the methods described and justified?

NICE 2011

NA

Mitchell 2012

NA

DATA: Data modelling

Is the data modelling methodology based on justifiable statistical and epidemiological techniques?

NICE 2011

Yes

Markov model using RCT data.

Mitchell 2012

 Yes

DATA: Data modelling ‐ Baseline data

Is the choice of baseline data described and justified?

NICE 2011

Yes

See above: data from two RCTs.

Mitchell 2012

Yes

Are transition probabilities calculated appropriately?

NICE 2011

Yes

Extraction of transition probabilities were revised and agreed with NICE, see Table 12.

Mitchell 2012

Unclear

Insufficient details.

Has a half‐cycle correction been applied to both cost and outcome?

NICE 2011

Unclear

Not specified.

Mitchell 2012

Yes

If not, has this omission been justified?

NICE 2011

NA

Mitchell 2012

NA

DATA: Data modelling ‐ Treatment effect

If relative treatment effects have been derived from trial data, have they been synthesised using appropriate techniques? 

NICE 2011

NA

Effects from individual studies.

Mitchell 2012

NA

Have the methods and assumptions used to extrapolate short‐term results to final outcomes been documented and justified?

NICE 2011

Yes

Explanation given: no change in treatment effect during year 2, according to DRCRnet, and from year 2 to 15. Mortality data from UK‐based literature (2.45 RR for diabetics with DMO).

Mitchell 2012

Yes

Same as NICE 2011, plus decline of visual acuity after year 2 modelled according to Wisconsin study (WESDR), the Diabetes Control and Complications Trial and UK Prospective Diabetes Study.

Have alternative assumptions been explored through sensitivity analysis?

NICE 2011

Yes

The manufacturer presented a series of deterministic sensitivity analyses in which parameters were varied across plausible ranges, including: number of ranibizumab injections during year 3 and 4;  retreatment need and treatment of both eyes in 35% of patients.

Mitchell 2012

No

Sensitivity analyses on number of injections performed. Sensitivity analysis regarding treatment need after 2 years and treatment in better vs. worse seeing eye or both eyes not presented.

Have assumptions regarding the continuing effect of treatment once treatment is complete been documented and justified? Have alternative assumptions been explored through sensitivity analysis?

NICE 2011

No

See previous comments and Table 12.

Mitchell 2012

No

See previous comments and Table 12.

DATA: Data modelling ‐ Costs

Are the costs incorporated into the model justified?

NICE 2011

Yes

All sources of costs considered, UK setting. Cost of adverse events not considered but suggested to be minimal due to the very low rate.

Mitchell 2012

Yes

Has the source for all costs been described?

NICE 2011

Yes

See Table 12.

Mitchell 2012

Yes

See Table 10.

Have discount rates been described and justified given the target decision‐maker?

NICE 2011

Unclear

Cannot find it in the report.

Mitchell 2012

Yes

3.5% discount rate.

DATA: Data modelling ‐ Quality of life weights (utilities)

Are the utilities incorporated into the model appropriate?

NICE 2011

No

Utility gain for treatment in the better‐seeing vs. worse‐seeing eye or in both eyes still a problem in the revised model.

Mitchell 2012

No

Utility gain for treatment in the better‐seeing vs. worse‐seeing eye or in both eyes still not considered.

Is the source for the utility weights referenced?

NICE 2011

Yes

EQ‐5D data from RESTORE were transformed to utility values using standard social tariffs and then related to BCVA in the treated eye using linear regression. Covariate adjustment used in revised submission. sensitivity analyses based on other sources available.

Mitchell 2012

Yes

EQ‐5D data from RESTORE were transformed to utility values using standard social tariffs and then related to BCVA in the treated eye using linear regression.

Are the methods of derivation for the utility weights justified?

NICE 2011

No

See above.

Mitchell 2012

No

See above.

DATA: Data incorporation

Have all data incorporated into the model been described and referenced in sufficient detail?

NICE 2011

Yes

Mitchell 2012

Yes

Has the use of mutually inconsistent data been justified (i.e. are assumptions and choices appropriate)?

NICE 2011

NA

Sources were specific to each type of data and no calibration was possible.

Mitchell 2012

NA

Is the process of data incorporation transparent?

NICE 2011

Yes

Use of data from RCT described.

Mitchell 2012

Yes

Use of data from RCT  and population‐based study described.

If data have been incorporated as distributions, has the choice of distribution for each parameter been described and justified?

NICE 2011

NA

Individual data from RCTs used to estimate parameters.

Mitchell 2012

NA

If data have been incorporated as distributions, is it clear that second order uncertainty is reflected?

NICE 2011

NA

Mitchell 2012

NA

DATA: Assessment of uncertainty

Have the four principal types of uncertainty been addressed? 

NICE 2011

No

See below

Mitchell 2012

No

If not, has the omission of particular forms of uncertainty been justified?

NICE 2011

No

Mitchell 2012

No

DATA: Assessment of uncertainty: Methodological

Have methodological uncertainties been addressed by running alternative versions of the model with different methodological assumptions?

NICE 2011

No

Single model structure as far as reported in NICE document

Mitchell 2012

No

Single model structure.

DATA: Assessment of uncertainty: Structural

Is there evidence that structural uncertainties have been addressed via sensitivity analysis?

NICE 2011

Yes

Sensitivity analyses presented.

Mitchell 2012

Yes

DATA: Assessment of uncertainty: Heterogeneity

Has heterogeneity been dealt with by running the model separately for different subgroups?

NICE 2011

Yes/Unclear

The manufacturer’s subgroup analysis suggesting that ranibizumab has a favourable cost‐effectiveness profile in people with a thicker retina (central foveal thickness greater than 400 micrometres), but The Appraisal Committee concluded that it could not consider this subgroup analysis sufficiently robust to support separate recommendations to the NHS because of small sample sizes.

Mitchell 2012

No

No subgroup analyses reported.

DATA: Assessment of uncertainty: Parameter

Are the methods of assessment of parameter uncertainty appropriate?

NICE 2011

Yes

One‐way sensitivity analyses of point estimates of parameters.

Mitchell 2012

Yes

If data are incorporated as point estimates, are the ranges used for sensitivity analysis stated clearly and justified?

NICE 2011

Yes

Mitchell 2012

Yes

CONSISTENCY: Internal consistency

Is there evidence that the mathematical logic of the model has been tested thoroughly before use?

NICE 2011

Unclear

Not reported.

Mitchell 2012

Unclear

Not reported.

CONSISTENCY: External consistency

Are any counterintuitive results from the model explained and justified?

NICE 2011

Unclear

No such results.

Mitchell 2012

Unclear

If the model has been calibrated against independent data, have any differences been explained and justified?

NICE 2011

NA

No such calibration.

Mitchell 2012

NA

Have the results of the model been compared with those of previous models and any differences in results explained?

NICE 2011

NA

No previous models.

Mitchell 2012

No

No comparison with NICE 2011, published in July 2011 and available to manufacturer.

NICE 2011

 Mitchell 2012

Cost of resources

Ranibizumab– drug cost

£761.20

£742.17

Ranibizumab injection – procedure cost

£150

Injections year 1‐2

10

10

Injections after year 2

3

0

Laser photocoagulation – procedure cost

£150

£274.19

Procedures year 1‐2

3

3

Procedures after year 2

2

0

Ophthalmologist visit for ranibizumab plus any  injection

£126

£84.42

Visits year 1‐2

22

22

Ophthalmologist visit for laser

£126

£84.42

Visits year 1‐2

8

8

Visits after year 2

4

4

Cost of disease consequences

Low vision total cost ‐  1st year

£6067

Low vision total cost ‐ 2nd year

£5936

Low‐vision aids (annual, 33% blind patients)

£194.16

Residential care (private, 30% blind patients)

£16,998.80

Community Care (annual, 6% blind patients)

£12,064

Depression (annual, 39% blind patients)

£558.24

Hip replacement (5% blind patients)

£6952.93

Mortality of patients with DMO (vs. age‐sex matched)

RR: 2.45

RR: 2.45

Utility source

RESTORE

RESTORE

Time horizon

15 years

15 years

Discount rate

unclear

3.5%

ICER (cost per QALY) ‐ base‐case and main sensitivity analyses

Base‐case

£30,277

£24,028

Bilateral treatment 35%

£44,400

Covariate adjusted utility estimation

£33,857

Utility source: Lloyd

£24,779

£19,238

Uitlity source: Brazier

£23,664

Utility source: Brown

£21,953

Discount rate 0‐5% (vs 3.5%): ICER £17,051‐27,042

Total n. ranibizumab injections (vs 10) 14 ‐ 6

£38,836 ‐12,446

Cost of blindness (base £6477) ‐25%‐+25%: ICER

£27,907‐20,150

Long term progression of visual acuity (vs declining) stable or improved

£26,198‐28,4131

Baseline age (vs 63 years) 58 years

£19,259

Number of visits with laser equal to ranibizumab

£37,673

Number of visits with ranibizumab equal to laser

£33,074