Types of studies

We included all randomised controlled trials (RCTs) or controlled clinical trials (CCTs) comparing bisphosphonate versus placebo or other active treatment for Paget's disease of bone in adults, including those that compared regimens of bisphosphonates aimed to normalise biochemical markers of bone turnover with those that did not.

Types of participants

Participants aged 18 years and over with Paget's disease of bone confirmed by plain radiographs or isotope bone scintigraphy (Selby 2002).

Types of interventions

The experimental intervention was defined as the use of bisphosphonates including non‐aminobisphosphonates (etidronate, clodronate or tiludronate) or aminobisphosphonates (alendronate, neridronate, pamidronate, olpadronate, ibandronate, risedronate or zoledronate).

The comparators were defined as placebo or other interventions including calcitonin, comparisons between non‐aminobisphosphonates and aminobisphosphonates; comparisons between different aminobisphosphonates, or comparisons among different treatment strategies using bisphosphonates.

We compared:

• bisphosphonates versus placebo;

• bisphosphonates versus calcitonin;

• bisphosphonates versus bisphosphonates:

  • non‐aminobisphosphonates;

  • aminobisphosphonates and non‐aminobisphosphonates;

  • different aminobisphosphonates;

• bisphosphonates regimens that aimed to normalise elevated bone turnover with those that did not (intensive versus non‐intensive treatment); and

• bisphosphonates versus bisphosphonates plus calcitonin.

We considered calcium, vitamin D supplementation and analgesics as co‐interventions if use was equally available to all treatment groups.

The planned analysis according time points (3 or 6 months from the start of treatment; 1, 2, 3, 4 or more years from the start of treatment; and at the end of the trial) were not performed because nearly all included trials had only six months follow‐up.

Types of outcome measures

Electronic searches

We searched:

1. MEDLINE (1946 to 3 March 2017) (Appendix 1);

2. Embase (1980 to 3 March 2017) (Appendix 2);

3. Cochrane Central Register of Controlled Trials (CENTRAL) (all issues to 3 March 2017) (Appendix 3);

4. ISI Web of Knowledge (all years to 8 March 2017) (Appendix 4).

We used the 'Cochrane Highly Sensitive Search Strategy for identifying randomised trials in MEDLINE: sensitivity‐maximising version (2008 revision); Ovid format' for identifying randomised trials as proposed in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). For Embase, we used a combination of the search filters for identifying randomised trials listed in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). We set no year, language or country of publication restrictions.

Searching other resources

We screened the reference lists of all included articles to identify additional eligible studies. We focused on systematic reviews and meta‐analyses and reference lists from identified RCTs to identify further relevant studies.

We searched the ISRCTN registry on metaRegister of Controlled Trials (mRCT) and ClinicalTrials.gov to identify ongoing trials (Date of last search: 20 February 2017). The mRCT includes the International Clinical Trials Registry Platform (ICTRP) of the World Health Organization (WHO) and the United Kingdom Clinical Trials Gateway (UKCTG).

We also handsearched for abstracts presented from 2010 to 2016 at scientific meetings from the following societies and included those with sufficient information in the body of the abstract:

• American Society for Bone and Mineral Research;

• International Bone and Mineral Society;

• International Osteoporosis Foundation/European Society for Clinical and Economic Aspects of Osteoporosis and Osteoarthritis; and

• European Calcified Tissue Society.

In accordance with Cochrane Musculoskeletal Group recommendations we searched the websites of the regulatory agencies to broaden our search for specific and rare adverse events: USA Food and Drug Administration MedWatch (FDA), European Medicines Agency (EMA), and the UK Medicines and Healthcare products Regulatory Agency pharmacovigilance and drug safety updates (MHRA) using the terms 'Paget's' and 'bisphosphonate' (Date of last search for the regulatory agencies websites: 22 February 2017). We also searched in the Australian Adverse Drug Reactions Bulletin (AADRB) [published from 1995 to 2009 and available at https://www.tga.gov.au/publication/australian‐adverse‐drug‐reactions‐bulletin] and in the Australian Therapeutic Goods Administration website (Date of last search: 22 February 2017).

Selection of studies

Three review authors (AT, LCG, JdPM) independently assessed results from searches of electronic databases to identify potentially‐relevant articles based on title or title and abstract. We retrieved the full manuscripts of potentially‐relevant articles for further assessment. Review authors (AT and LCG; JdPM and LCG) screened the selected articles independently against the inclusion criteria. We resolved any differences of opinion during the selection process by discussion and consensus, or by consulting a third review author (SHR) if needed.

Data extraction and management

Two review authors (AT, LCG) extracted relevant data using a pre‐defined data collection form designed according to guidelines from the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). One review author (AT) who was involved in data extraction was also an author of a primary study (PRISM‐EZ trial Tan 2017). Because authors of primary studies should not extract data from their own studies, we recruited an additional reviewer (JdPM) who extracted trial data from Tan 2017.

We extracted the following data:

• Study identification: author, year of publication, journal.

• Characteristics of the trial: study design, calculation of sample size before the study, use of intention‐to‐treat analysis, setting or location, number of centres, country, period of study, follow‐up period, outcomes, methods of randomisation, allocation concealment and blinding.

• Study inclusion and exclusion criteria.

• Participants' characteristics: age, gender, presence of monostotic disease, whether treated previously for Paget's disease of bone, symptomatic participants, numbers who were randomised and excluded (post‐randomisation), reasons for exclusion, participants assessed, withdrawals and reasons for withdrawals.

• Characteristics of intervention: drug analysed, comparator, dosage, duration of treatment, co‐interventions.

• Risk of bias assessment.

• Outcome data (Types of outcome measures).

• Source of funding.

• Conflicts of interest.

We created a specific database to carry out the data extraction process. We resolved any differences of opinion during the data extraction process by discussion and consensus or discussion with a third author (SHR).

When data were neither available from the original manuscript, nor following requests to authors, data were directly extracted from figures in the manuscript using a vector graphics editor.

Assessment of risk of bias in included studies

Two authors (AT, LCG) assessed the risk of bias in included studies independently as recommended by the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).

We assessed the following methodological domains.

• Adequate sequence generation (checking for possible selection bias): were methods used to generate the allocation sequence for each included study reported in sufficient detail to enable assessment of whether groups were comparable.

• Adequate measures to conceal allocation (checking for possible selection bias): were methods used to conceal the allocation sequence for each included study reported in sufficient detail and we could determine whether intervention allocation could have been foreseen in advance or during recruitment, or changed after assignment.

• Blinding of study participants and study researchers (checking for possible performance bias): were methods used to blind participants and study researchers from knowledge of which intervention a participant received for each included study reported.

• Blinding of outcome assessors (checking for possible detection bias): were methods used to blind outcome assessors from knowledge of which intervention a participant received for each included study reported.

• Incomplete outcome data (checking for possible attrition bias through withdrawals, dropouts, protocol deviations): the completeness of data including attrition and exclusions from the analysis was assessed. We stated whether attrition and exclusions were reported; the numbers included in the analysis at each stage (compared with the total randomised participants); reasons for attrition or exclusion where reported; and whether missing data were balanced across groups or were related to outcomes. We did not consider that a high attrition rate was a source of bias if an intention‐to‐treat (ITT) analysis was performed. Where sufficient information was reported or could be supplied by the study authors, we re‐included missing data in the analyses which we undertook.

• Selective outcome reporting (checking for possible reporting bias): differences, if any, between the planned protocol analysis data and the reported data for each included study, to look for unreported findings.

• Other potential threats to validity (considering external validity, e.g. relevant use of co‐intervention).

Each of these criteria were explicitly judged as 'low risk, 'high risk or 'unclear risk' of bias.

We assessed the likely overall magnitude of the bias for each included study and whether it was likely to impact on the findings. We elaborated a summary assessment of the risk of bias for each outcome. In order to assess the risk of bias within a study, we evaluated adequate sequence generation, allocation sequence concealment and incomplete outcome data as key domains for all the outcomes. We evaluated blinding of study participants and study researchers as key domains for change in pain, reporting of adverse events and change in quality of life. We did not consider blinding as a key domain for assessing the risk of bias for fractures, receiving orthopaedic surgery, change in hearing thresholds and total alkaline phosphate activity. Other potential threats to validity such as the use of analgesics as co‐interventions were considered a key domain for changes in pain and quality of life.

For each outcome, we considered low risk of bias to be present across studies when most information was from studies assessed at low risk. We considered high risk of bias across studies when the proportion of information from studies at high risk of bias was sufficient to affect the interpretation of results. Lastly, we considered unclear risk of bias when most information was from studies at low or unclear risk of bias.

We resolved any difference of opinion during the assessment of risk of bias process by discussion and achieving consensus. For disagreements not resolved by consensus, we consulted a third review author (SHR).

Measures of treatment effect

For dichotomous outcomes, we estimated the effect of treatment across trials using the risk ratios (RRs) with the corresponding 95% confidence intervals (CIs). For significant outcomes, we computed the number needed to treat to benefit (NNTB) one participant or the number needed to treat to harm (NNTH) one participant as the inverse of the pooled risk differences (RDs).

Our dichotomous outcomes were numbers of participants who:

• experienced adverse events;

• experienced radiologically‐confirmed clinical fractures;

• received orthopaedic surgery;

• withdrew due to adverse events;

• relapsed due to recurrence of bone pain; and

• relapsed due to recurrence of increased serum alkaline phosphatase level.

Where continuous scales of measurement were used to assess the effects of treatment, we used the mean difference (MD) with the corresponding 95% CI. If different scales were used to measure an outcome, we calculate the standardised mean difference (SMD).

Our continuous outcomes were:

• change in pain;

• change in quality of life;

• change in hearing thresholds; and

• mean percentage change from baseline in serum total alkaline phosphatase activity.

Unit of analysis issues

The unit of analysis was individual people undergoing treatment for Paget's disease of bone. We included only RCTs and CCTs, and we were not expecting studies with non‐standard designs such as cross‐over trials.

For studies with more than one intervention group, such as different doses of the same bisphosphonate, we combined data from the experimental intervention groups to create a single pair‐wise comparison versus the control group.

Dealing with missing data

When outcome data were not available from trials, we contacted the primary investigators of the eligible trials. We contacted study authors using email addresses published in the studies. When there was no response from the study author, we analysed the available data only, ignoring missing data, because we assumed that data were missing at random.

We performed a sensitivity analysis according to the missing data to assess the robustness of our assumption that data were missing data at random (Sensitivity analysis). We addressed the potential impact of missing data on findings in the Discussion.

Assessment of heterogeneity

We computed global estimates for each variable effect by meta‐analysing study single effect measures (RR for dichotomous variables and MD or SMD for continuous variables) using Review Manager 2014. Before calculating estimates of effect, we assessed the presence and degree of heterogeneity using the I² statistic to describe the percentage of variability in effect estimates due to heterogeneity rather than chance. We categorised I² values greater than 75% as considerable heterogeneity (Higgins 2011).

We undertook a narrative review of potential heterogeneity according to variability in populations, interventions, outcomes and settings.

Assessment of reporting biases

Because we anticipated inclusion of several studies with small sample sizes, we assessed the likely impact on our findings of small‐study effects (tendency for estimates of the intervention effect to be more beneficial in smaller studies) by using inverted funnel plot techniques. We planned to construct funnel plots when there were at least 10 included studies in comparisons. However, none of the comparisons included 10 studies, so funnel plots could not be constructed. We provided a narrative summary of potential small‐study effect in the Discussion.

Data synthesis

We used Review Manager 2014 for data synthesis. Quantitative synthesis was planned if more than one eligible study was identified. Where appropriate, we calculated a pooled estimate of treatment effect across similar studies for each pre‐specified outcome. We estimated overall effect by meta‐analysis using a random‐effects model. We did not conduct meta‐analysis if there was considerable heterogeneity (I² > 75%) (Higgins 2011). We calculated RRs and 95% CIs for dichotomous data. We calculated the number needed to treat to provide an indication for each dichotomous outcome, reflecting the number of participants required to obtain a beneficial outcome with the intervention. For continuous data measured on the same scale, we calculated MDs. When different scales were used, we calculated SMDs. When possible, we analysed data using an intention‐to‐treat (ITT) model.

We used the mean and standard deviation when available. If only median and interquartile ranges were reported, we followed guidance from the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011), where the median was used as the mean and the standard deviation was set as 1.35. If no standard deviation was given at the end of the study, we used the baseline standard deviation at the end as well.

We undertook a narrative review of eligible studies where statistical synthesis of data from more than one study was not possible or appropriate.

Although we included different bisphosphonates whose comparisons against placebo which together create a network (Figure 1), we did not perform a network meta‐analysis. The reason for this was that tor the major outcome of change in pain, heterogeneity within studies in the definition of 'change in pain' did not allow us to conduct a meaningful network meta‐analysis. For the other major outcomes, such as fractures or orthopaedic surgery, there were few comparisons between bisphosphonates.

Figure 1

---

---

Geometry of the network of randomised trials of bisphosphonates for Paget's disease of bone. The nodes of the network represent the treatments compared. The links reflect comparisons and the number of links is proportional to the number of comparisons

Subgroup analysis and investigation of heterogeneity

Neither of the planned subgroup analyses: symptomatic bone pain versus those who were asymptomatic; and biochemically active (raised alkaline phosphatase) Paget's disease of bone versus non‐biochemically active Paget's disease of bone (normal alkaline phosphatase) were possible.

Sensitivity analysis

We planned to perform the following sensitivity analyses:

1. To determine the robustness of pooled effect estimate in terms of risk of bias by including or excluding studies with high or unclear risk of bias from the comparative analysis.

2. To determine the robustness of the pooled effect estimate in terms of missing data by including or excluding studies with high levels of missing data, more than 20% of missing data for the overall trial population, or for any of the trial arms from the comparative analysis.

3. To determine the robustness of the pooled effect estimate in terms of withdrawals by performing either worst‐case scenario sensitivity analysis (all withdrawal data treated as negative events) or the same‐as‐control scenario sensitivity analysis, where the rate of negative events in the withdrawal data were the same as in the control group.