Types of studies

Randomised controlled trials (RCTs) and quasi‐randomised studies will be included. However, as we aim to provide evidence of adverse effects that cannot be adequately studied with RCTs alone, and we anticipate a low yield from RCTs, we will also include non‐randomised studies (NRS) such as controlled before and after studies (CBAs), interrupted‐time‐series studies, cohort and case‐control studies as well as consecutive and non‐consecutive case series, provided these are reported from large registry databases, but with no restriction on numbers reported.

Types of participants

Patients at least 18 years of age with a diagnosis of inflammatory arthritis (RA, AS, PsA and SpA) receiving treatment with MTX, at any dose, duration or route, will be included.

Types of interventions

Studies comparing safety of MTX alone to MTX with concurrent NSAIDs and/or paracetamol will be included. There will be no restrictions with regard dose, duration or route of administration. We will exclude studies of valdecoxib, lumiracoxib and rofecoxib as they have been withdrawn from use around the world: valdecoxib has been withdrawn from use in the European Union (EMEA 2005), the United States (FDA 2005), Australia (TGA 2005) and Canada (Health Canada 2005); lumiracoxib has been withdrawn from many countries worldwide (MHRA 2007); and rofecoxib has been withdrawn from use worldwide (Merck 2004).

Types of outcome measures

Electronic searches

We will search the following electronic databases: the Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library); MEDLINE, from 1950 to the present day; and EMBASE, 1980 to the present day, without language restrictions. Specific MeSH headings and additional keywords will be used to identify all relevant studies. The complete search strategies for the database searches are provided in the Appendices (MEDLINE search strategy Appendix 1, EMBASE search strategy Appendix 2, CENTRAL search strategy Appendix 3).

Searching other resources

The Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE) will also be included to ensure that all potential studies are identified for this systematic review. We will manually search the bibliographies of all included papers for information on any other relevant studies. We will contact the first authors of any papers if specific data is missing. We will also handsearch the conference proceedings for the American College of Rheumatology (ACR) and the European League against Rheumatism (EULAR) for 2008‐9, to identify unpublished studies.

We will also check the websites of regulatory agencies for reported adverse events, labels and warnings, including Current Problems in Pharmacovigilance; Drug Safety Update; the Drug Safety Research Unit (UK); Australian Adverse Drug Reactions Bulletin (Australia); MedWatch, the Food and Drug Administration (FDA) Safety Information and Adverse Event Reporting Program (US); the European Public Assessment Reports (EPARs) from the European Medicines Agency; and those agencies recommended by The Canadian Agency for Drugs and Technology in Health (CADTH) manual, "Grey Matters" (Grey Matters 2009); as well as the websites of various international rheumatology societies.

Selection of studies

Two reviewers (ANC, JM) will independently assess each title and abstract for suitability for inclusion in the review. If more information is required to establish if the inclusion criteria are met, we will obtain the full text of the paper. We will record reasons for excluding any studies. Disagreement will be resolved by consensus after review of the full text article, or after the input of the third author (CJE), who will have the final decision. We will not exclude any foreign language studies.

Data extraction and management

Data will be independently extracted from the included studies by two reviewers (ANC, JM) and entered into Review Manager 5. If additional information is required, we will obtain this by contacting the authors of the relevant studies.

We will extract data including:

• general information such as date of extraction;

• review author details;

• study information, including type of study, inclusion and exclusion criteria;

• study population characteristics;

• baseline data, such as age, sex and ethnicity;

• diagnostic criteria;

• medication details, including name of drug, dose, route, frequency and duration of treatment;

• definitions given for the reported adverse events, and methods used to monitor these;

• outcomes, such as what they were and how they were obtained, confounding factors, analysis, including statistical techniques, sample size based on power calculation, adjustment for confounding, and losses to follow‐up; and

• results including direction of relationship, size of effect and measure of precision of effect estimate, such as 95% confidence interval or standard error.

As we will include NRS for this review, we will assess the following study features when deciding which NRS should be included:

• presence of a comparison group;

• method used to allocate the participants to groups;

• which parts of the study were prospective; and

• variables used to assess comparability between the groups.

Assessment of risk of bias in included studies

Two reviewers (ANC, JM) will assess the quality of the included RCTs using risk of bias (Higgins 2009).

This includes assessment of the following domains:

• sequence generation;

• allocation concealment;

• blinding;

• incomplete outcome data;

• selective outcome reporting; and

• other sources of bias.

Each criteria will be judged according to low risk of bias, high risk of bias or lack of information, or uncertainty over the degree of bias. We will discuss and resolve any disagreement between the reviewers by consensus meeting. If agreement still can not be reached, a third reviewer (CJE) will make the final decision.

As we are also including NRS, we will assess quality of these study types using the following tools. For CBAs and ITSs, we will use the criteria described by the Cochrane Effective Practice and Organisation of Care (EPOC) Group, which describes seven standard criteria for each of these study types, scored as done, not done or not clear (EPOC 2008); for cohort studies, we will use the Newcastle‐Ottawa Scale (NOS), which assesses quality according to selection, comparability and outcome; for case‐control studies, the NOS, which assesses quality according to selection, comparability and exposure (Wells 2008); and for case series, the guidance recommended by the Centre for Reviews and Dissemination (CRD, CRD 2001), which assesses the following criteria.

• Is the study based on a representative sample selected from a relevant population?

• Are the criteria for inclusion explicit?

• Did all individuals enter the survey at a similar point in their disease progression?

• Was follow‐up long enough for important events to occur?

• Were outcomes assessed using objective criteria or was blinding used?

• If comparisons of sub‐series are being made, was there sufficient description of the series and the distribution of prognostic factors?

Measures of treatment effect

The treatment effect for this SLR is concerned only with issues of safety. The results of the studies will be analysed and collated using Review Manager 5. The results of each study will be expressed as relative risks (RR) with corresponding 95% confidence intervals (95% CI) for dichotomous data, and difference in means with corresponding 95% CI for continuous data.

Adverse events in each study will be reported descriptively in the table of included studies, with studies grouped by design (RCTs, CBAs, ITSs, cohort, case‐control and case series). For adverse events reported in controlled before‐after studies and cohort studies, frequency and RR of the event and time to event will be recorded when data is available. For case‐control studies, frequency of the event in cases and controls, and the odds ratio (OR) will be recorded. For data extracted from case series, the frequency of each event will be recorded.

Unit of analysis issues

We will consider and address any issues arising from unit of analysis as recommended in the Cochrane Handbook (Higgins 2009). This will include consideration of the method used for randomisation in any RCTs we identify, such as cluster‐randomisation, which can be accounted for by using appropriate statistical methods such as a "multilevel model", a "variance components analysis" or a "generalised estimating equation" which we will consider with statistical advice; the effect of individuals receiving more than one intervention, for example in cross‐over trials, for which we will consider analysing the results as if they were from two separate studies, or by including data from only one period (most likely the first period) of the study; and repeated observations on individual participants at different time points, for which we will report the outcomes at the different periods of follow‐up in order to identify outcomes that occur within different time frames, for example to reflect short, medium and long‐term follow‐up.

Dealing with missing data

We will attempt to obtain any relevant missing data from the study authors. We will perform intention‐to‐treat (ITT) analyses where trials have not included all randomised participants in their analyses. ITT analysis is a strategy for analysing data in which all participants are included in the group to which they were assigned, regardless of whether they completed the intervention given to the group. Performing an ITT analysis is important as it prevents bias caused by loss of participants.

Assessment of heterogeneity

As we will be including NRS in this review, heterogeneity will be greater than if RCTs alone were included. Therefore, we will be less likely to be able to pool our findings quantitatively. However, we will analyse studies grouped by design (RCTs, CBAs, ITSs, cohort, case‐control and case series), and perform meta‐analysis where possible for clinically homogenous studies with the same design. Heterogeneity will be addressed using forest plots, and statistical heterogeneity will be assessed using the I² statistic, using the following as a rough guide for interpretation: 0‐40% might not be important, 30‐60% may represent moderate heterogeneity, 50‐90% may represent substantial heterogeneity, and 75‐100% considerable heterogeneity (Higgins 2009).

Assessment of reporting biases

We will consider the possibility of reporting bias for the studies included using funnel plots, if 10 or more studies are available. We will compare the fixed‐effect estimate against the random effects model to assess the possible presence of small sample bias in the published literature (i.e., intervention effect is more beneficial in the smaller studies). In the presence of small sample bias, the random‐effects estimate of the intervention effect is more beneficial than the fixed‐effect estimate (Sterne 2008).

Data synthesis

We will pool data from studies that are sufficiently homogenous, and with the same study design in order to perform a meta‐analysis. Data across different study designs will therefore not be pooled. We use a random‐effects model as the default. Analysis will be performed using Review Manager 5 and forest plots will be produced for all analyses. A summary of findings table will be produced using GRADEpro software.

Subgroup analysis and investigation of heterogeneity

Ideally, we will undertake the following subgroup analyses:

• effects of age and gender, to minimise possible confounding by indication;

• difference in effect between the diagnoses (RA, AS, PsA, SpA); and

• dose, route and duration of treatment with MTX, NSAIDs and/or paracetamol.

Sensitivity analysis

We will consider whether the conclusions drawn from this review are robust, including inclusion/exclusion of particular studies in any meta‐analyses we perform, dealing with missing data, the choices regarding analysis methods used, and the effect of reporting bias.