Types of studies

We included double‐blind studies comparing a single dose of oral analgesic plus caffeine with the same dose of the analgesic alone for the treatment of acute pain in adults; the caffeine had to be administered at the same time as the analgesic. Studies had to have a minimum of 10 participants randomly allocated to each treatment group and report some measure of patient‐reported pain relief.

We included studies using multiple doses to treat a single episode only if appropriate data from the first dose were available. We included cross‐over studies and studies reporting treatment of consecutive episodes (for example, consecutive migraine attacks) in pain conditions that result in comparable, recurrent, acute pain episodes, such as migraine. We used first dose only data where possible, but we accepted data from both phases of a cross‐over, or consecutive phases for recurrent conditions, if there was adequate washout (at least 48 hours pain‐ and medication‐free between phases).

Types of participants

Studies included adult participants (at least 16 years of age) with any acute painful condition. We did not include studies of experimental pain in healthy volunteers because these do not accurately correlate with clinical pain. Ideally participants had moderate to severe pain, to ensure sensitivity to detect a change in pain intensity, but mild to moderate pain was accepted.

Types of interventions

Included studies had to use a single dose of oral analgesic plus caffeine to treat an acute painful episode. The analgesics we were particularly interested in were paracetamol, ibuprofen, aspirin, diclofenac, naproxen, oxycodone, ergotamine, and the triptans, although studies using other analgesics were not excluded. There was no restriction on dose of analgesic or caffeine.

To investigate the effect of caffeine on the efficacy of the analgesic with which it is combined, it was essential that the comparator was the same drug and dose as the combination, minus caffeine.

Types of outcome measures

We collected data on the type of painful condition and baseline pain intensity.

Electronic searches

We searched the following databases:

• the Cochrane Central Register of Controlled Trials (CENTRAL) via CRSO (to 28 August 2014);

• MEDLINE via Ovid (1946 to 28 August 2014);

• EMBASE via Ovid (1974 to 28 August 2014);

• Oxford Pain Relief Database for the original review (Jadad 1996a). This database is no longer being updated.

See Appendix 1 for the search strategy for MEDLINE, Appendix 2 for EMBASE, and Appendix 3 for CENTRAL. We did not impose any language restrictions.

Searching other resources

We searched reference lists of retrieved studies and review articles for additional studies. We know of a number of unpublished trials using a caffeine analgesic combination, and we contacted relevant manufacturers to try to determine the extent of, and obtain, any unpublished data. We carried out Internet searches to identify any studies or study results that may have been reported to agencies such as the Food and Drug Administration (FDA).

For the update we searched two clinical trials databases (ClinicalTrials.gov (ClinicalTrials.gov)) and World Health Organization International Clinical Trials Registry Platform (ICTRP) (http://apps.who.int/trialsearch/) to identify additional published or unpublished data.

We did not search grey literature and short abstracts (meeting reports).

Selection of studies

Two review authors independently carried out the searches and selected studies for inclusion. We viewed the titles and abstracts of all studies identified by electronic searches on screen, and excluded any that clearly did not satisfy the inclusion criteria. We obtained full copies of the remaining studies to identify those suitable for inclusion, and settled any disagreements or uncertainty by discussion with the third review author.

Data extraction and management

Two review authors independently extracted data from included studies using a standard data extraction form. Disagreements and uncertainty were settled by discussion with the third review author. One review author entered data into RevMan 5.3 (RevMan 2014).

Assessment of risk of bias in included studies

We used the Oxford Quality Score as the basis for inclusion, limiting inclusion to studies that were randomised and double‐blind as a minimum (Jadad 1996b).

Two authors independently assessed the risk of bias for each study, using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011), and adapted from those used by the Cochrane Pregnancy and Childbirth Group, with any disagreements resolved by discussion. We assessed the following for each study:

1. Random sequence generation (checking for possible selection bias). We assessed the method used to generate the allocation sequence as: low risk of bias (any truly random process, for example random number table; computer random number generator); or unclear risk of bias (when the method used to generate the sequence is not clearly stated). We excluded studies at a high risk of bias using a non‐random process (for example, odd or even date of birth; hospital or clinic record number).

2. Allocation concealment (checking for possible selection bias). The method used to conceal allocation to interventions before assignment determines whether intervention allocation could have been foreseen in advance of, or during, recruitment, or changed after assignment. We assessed the methods as: low risk of bias (for example, telephone or central randomisation; consecutively numbered, sealed, opaque envelopes); or unclear risk of bias (when the method is not clearly stated). We excluded studies that did not conceal allocation and are therefore at a high risk of bias (for example, open list).

3. Blinding of outcome assessment (checking for possible detection bias). We assessed the methods used to blind study participants and outcome assessors from knowledge of which intervention a participant received. We assessed the methods as: low risk of bias (for example, study stated that it was blinded and described the method used to achieve blinding, for example, identical tablets, matched in appearance and smell); or unclear risk of bias (study stated that it was blinded but did not provide an adequate description of how it was achieved). We excluded studies at a high risk of bias that were not double‐blind.

4. Size of study (checking for possible biases confounded by small size). We assessed studies as being at low risk of bias (200 participants or more per treatment arm); unclear risk of bias (50 to 199 participants per treatment arm); or high risk of bias (fewer than 50 participants per treatment arm).

Measures of treatment effect

We used risk ratio (RR), calculated using a fixed‐effect model, to determine statistical difference between treatment groups, and number needed to treat to benefit (NNT) to provide an absolute measure of treatment effect. See 'Data synthesis' section for details.

Unit of analysis issues

We accepted randomisation of individual participants only. We did not, for example, accept studies where randomisation was by centre.

Dealing with missing data

The most likely source of missing data in single dose studies is cross‐over studies, in which multiple successive attacks are treated. In these studies, participants are excluded from the efficacy analyses after taking an initial dose simply because they do not have a sufficient number of qualifying pain episodes (for example, separate migraine attacks) to complete the cross‐over study. This is unlikely to introduce bias where it occurs equally in both treatment arms.

Assessment of heterogeneity

We assessed heterogeneity of studies visually using L'Abbé plots of the percentage of participants with the outcome with caffeine compared with those without caffeine (L'Abbé 1987), and with the use of the I² statistic.

Assessment of reporting biases

Unpublished studies in this field are known to exist, as demonstrated in the review by Laska et al (Laska 1984), which analysed data from 30 clinical trials ‐ of which only four have been published in full. Obtaining unpublished studies, many of which were conducted 25 or more years ago, from the pharmaceutical companies sponsoring them was not possible. It is therefore difficult to make any meaningful assessment of reporting bias, and results must be interpreted with caution, with the knowledge that some degree of reporting bias is likely.

Approaches to pharmaceutical companies that may have had data were unsuccessful. In addition, although we know of data relating to three unpublished studies, we have been unable to obtain permission to use it. We identified further unpublished studies, with no available results, during this update.

We assessed the number of trials of average size amongst the included studies, with a RR of one (no effect), needed to reduce any statistically significant result to one that fails to meet statistical significance (following Moore 2008).

Data synthesis

Where possible, we used dichotomous data to calculate the RR with 95% confidence intervals (CIs) using a fixed‐effect model (Morris 1995). We calculated NNT with 95% CIs using the pooled number of events by the method of Cook and Sackett (Cook 1995). We assumed a statistically significant difference from control when the 95% CI of the RR did not include the number one.

Many studies provided data on pain measures using:

• five‐point categorical pain relief (PR) scales with comparable wording to "none, slight, moderate, good or complete";

• four‐point categorical pain intensity (PI) scales with comparable wording to "none, mild, moderate, severe";

• visual analogue scale (VAS) for pain relief;

• VAS for pain intensity.

We summed these pain measures over a four to six‐hour period to generate a measure of total pain relief (TOTPAR) or summed pain intensity difference (SPID) over this time period.

Where only non‐dichotomous, mean data are reported, we transformed them into dichotomous data using standardised methods. We converted any mean TOTPAR, SPID, VAS TOTPAR or VAS SPID values to %maxTOTPAR or %maxSPID by division into the calculated maximum value (Cooper 1991). We calculated the proportion of participants in each treatment group achieving at least 50%maxTOTPAR using verified equations (Moore 1996; Moore 1997a; Moore 1997b), and converted these proportions into the number of participants achieving at least 50%maxTOTPAR by multiplying by the total number of participants in the treatment group. We then used information on the number of participants with at least 50%maxTOTPAR for the analgesic plus caffeine and the analgesic alone to calculate estimates of RR and NNT as before.

We tested for significant differences between groups in sensitivity analyses using the z test (Tramèr 1997).

Subgroup analysis and investigation of heterogeneity

We analysed data for different pain conditions and different analgesics separately where there were sufficient data (minimum of two studies and 200 participants).

Sensitivity analysis

We planned to carry out sensitivity analyses for dose of caffeine (< 70 mg, 70 mg to 150 mg, and > 150 mg), methodological quality (2 versus ≥ 3 on the Oxford Quality Scale), baseline pain intensity (mild versus ≥ moderate), and size (< 50 versus ≥ 50 participants in each treatment arm), where there were sufficient data (minimum of two studies and 200 participants). We also tried to ascertain whether any adjuvant effect of caffeine depended on the particular analgesic drug with which it was combined, and to investigate whether analgesic effect size with analgesic drug alone affects any adjuvant effects of caffeine through limiting the upside sensitivity of the analgesic assay (Cooper 1991).