Types of studies

We included randomised controlled trials (RCTs).

Types of participants

People with carotid stenosis and recent transient ischaemic attacks (TIA) or minor ischaemic strokes in the territory of that artery.

Types of interventions

Unconfounded comparison of carotid surgery with no carotid surgery (i.e. best medical therapy plus surgery versus best medical therapy alone).

Types of outcome measures

Electronic searches

We searched the Cochrane Stroke Group's Trials Register, which was last searched by the Managing Editor in October 2019. In addition, we searched the Cochrane Central Register of Controlled Trials (CENTRAL; 2019, Issue 10 of 12) in the Cochrane Library (searched 23 October 2019; Appendix 1), MEDLINE Ovid (1966 to 23 October 2019; Appendix 2), and Embase Ovid (1980 to 23 October 2019; Appendix 3). We also searched the Web of Science Core Collection (last searched 23 October 2019; Appendix 4).

The subject strategies for databases were modelled on the search strategy designed for MEDLINE in Appendix 1, by the Cochrane Stroke Group’s Information Specialist. All search strategies deployed were combined with subject strategy adaptations of the highly sensitive search strategy designed by Cochrane for identifying RCTs and controlled clinical trials, as described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2019)

Having identified the two major studies in this area, we used a post‐hoc search strategy of ECST (text word) or European Carotid Surgery Trial (text word) or NASCET (text word) or North American Symptomatic Carotid Endarterectomy Trial (text word) in the CENTRAL and in the MEDLINE Ovid database from 1990 to 23 October 2019, in the hope of identifying previously unretrieved publications from these trials.

Searching other resources

In an effort to identify further published, unpublished, and ongoing trials, we:

• searched ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform (apps.who.int/trialsearch; last searched 23 October 2019: Appendix 4); and

• reviewed the reference lists of all relevant studies and abstract books in research proceedings.

Selection of studies

Two review authors (AR, SO) independently read the titles and abstracts of the records obtained from the electronic searches and excluded obviously irrelevant studies. We obtained the full texts of the remaining papers, and the same authors independently selected studies for inclusion, based on the predefined criteria. We resolved any disagreements through discussion. One review author (AR) selected those trials that met the inclusion criteria and another review author (SO) independently reviewed these decisions. We resolved all disagreements through discussion with other review authors (DPH, KR).

Data extraction and management

We extracted details of the method of randomisation, the blinding of outcome assessments, losses to follow‐up, and crossovers and exclusions after randomisation from the publications. We also extracted participant characteristics (age, sex, vascular risk factors, indication for surgery) and details of the operation (type of cerebral monitoring, use of carotid patching, anaesthetic technique, use of perioperative antiplatelet therapy). One review author (AR) extracted those trials that were selected and another review author (SO) independently reviewed these decisions. We resolved all disagreements through discussion with other review authors (DPH, KR).

Assessment of risk of bias in included studies

One review author (SO) assessed the risk of bias and another review author (KR) independently reviewed these decisions. We resolved all disagreements through discussion. We assessed risk of bias (high risk, low risk, unclear risk) using the Cochrane 'Risk of bias' tool as described in the Cochrane Handbook for Systematic Reviews of Interventions and reported the details in the 'Risk of bias' tables (Higgins 2019). These risks of bias included random sequence generation (selection bias), allocation concealment (selection bias), blinding of participants and personnel (performance bias), blinding outcome assessment (detection bias), and incomplete outcome data (attrition bias).

Measures of treatment effect

We measured the treatment effect in the following outcomes within five years of randomisation: stroke, death, ipsilateral ischaemic stroke, disabling or fatal ipsilateral ischaemic stroke, stroke and death within 30 days after carotid endarterectomy.

Unit of analysis issues

We extracted details of all outcome events. Some studies included participants who had bilateral operations, but only reported the number of participants in each group, and not the number of arteries. Unit of analysis issues originate when bilateral carotid arteries for the same participants are studied in a trial and such highly correlated data are regarded as independent, when multiple assessments of the same outcome are presented, or both. We recorded whether trials presented outcomes in relation to an artery, a participant, or as multiple (bilateral) carotid arteries for the same participant.

When a cluster‐randomised trial has been conducted and correctly analysed, effect estimates and their standard errors may be meta‐analysed using the generic inverse variance method in RevMan 5 (Review Manager 2014). If the randomisation was performed on the clusters rather than the individuals, we approximated the correct analyses using data suggested in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2019):

• the number of clusters (or groups) randomised to each intervention group, or the average (mean) size of each cluster;

• the outcome data, ignoring the cluster design, for the total number of individuals (for example, number or proportion of individuals with events, or means and standard deviations); and

• an estimate of the intracluster (or intraclass) correlation coefficient (ICC).

Dealing with missing data

When data were missing, we contacted the corresponding author or co‐author through the address given in the publication. If this information was not available, we searched for the study group via the Internet, and contacted them for missing information.

Assessment of heterogeneity

We assessed heterogeneity between study results using the I² statistic (Higgins 2003). We examined the percentage of total variations across the studies due to heterogeneity, rather than to chance.

We used I² results for quantifying inconsistency across studies. Values of I² over 0% indicated a considerable level of heterogeneity and were investigated.

Assessment of reporting biases

We performed an extensive search, and are confident that we identified all major relevant trials. We also contacted experts in this field. We searched for trials published in all languages, and we arranged translation of all possibly relevant publications when required. We had planned to use funnel plots to assess publication bias when more than 10 studies were included (Sterne 2011). However, if there were insufficient studies to conduct this analysis, we had planned to compare the study protocols with the final study reports to evaluate selective reporting of outcomes.

Data synthesis

We included all participants included in the final analysis of the results of the original trials in the combined analysis, using the Mantel‐Haenszel method (fixed‐effect). We stratified the main analyses according to the stenosis groups that were used in the NASCET 1991 trial (less than 30%, 30% to 49%, 50% to 69%, 70% to 99%), and analysed near‐occlusions separately (Rothwell 2003).

We performed all analyses of the effect of surgery on an intention‐to‐treat basis, according to the randomised treatment allocation. We assessed significance of the differences between treatment groups by the log rank test, stratified by study. We determined estimates of the absolute treatment effect (and 95% confidence intervals) at five‐year follow‐up from the Kaplan‐Meier event‐free survival curves. We tested significance of the differences in baseline data between trials and treatment groups using the Chi² test or Student's t‐test, as appropriate. We used RevMan 5 software (Review Manager 2014), and SPSS for Windows version 10.0 for all analyses (SPSS 1999). We used the fixed‐effect model for meta‐analysis in the absence of clinical, methodological, and statistical heterogeneity. If the I² statistic was greater than 0%, we also applied a random‐effects model to see whether the conclusions differed, and we noted any difference. If pooling was not possible or appropriate, we had planned to present a narrative summary (Deeks 2011).

GRADE and 'Summary of findings' table

We created summary of findings Table 1 with GRADEpro GDT (GRADEpro GDT), which imports data from RevMan 5. This table presents the results and the quality of the evidence of the main outcomes, using the GRADE system, which classifies the quality of evidence as high, moderate, low, and very low (Schünemann 2011). We included three main outcomes in this table: 1) any stroke or operative death, 2) Ipsilateral ischaemic stroke, and any operative stroke or death, 3) disabling or fatal ipsilateral ischaemic or operative stroke and death.

Pooling of individual patient data

We obtained the original individual patient data for the three included trials. We merged data on presenting events; baseline clinical, brain imaging, and angiographic characteristics; surgical and anaesthetic technique; and follow‐up into a single composite database. We gave detailed consideration to the definitions of each variable used in the original trials. Where definitions were identical, we merged comparable data. Where possible, we resolved differences in definitions of variables between studies by reconstructing definitions to achieve comparability.

Reassessment of carotid angiograms and identification of near‐occlusions

We collected all ECST 1998 and NASCET 1991 pre‐randomisation angiograms and reviewed them centrally for a previous version of this review (Rerkasem 2011). Therefore, the analyses could be consistently stratified by the degree of symptomatic carotid stenosis. As the first version of this review was performed without reclassifying angiograms, it was difficult to compare the results between studies (ECST 1998; NASCET 1991; VACSP 1991). This was due to different grading of the carotid stenosis used by the trials. One observer, who was blind to outcome events, re‐measured the 3018 ECST 1998 angiograms, and re‐calculated the degree of stenosis using the method used in NASCET 1991 and VACSP 1991. The NASCET 1991 method was based on measurement of the minimum residual lumen at the point of maximum stenosis and the diameter of the normal internal carotid artery well beyond the carotid bulb, where the walls of the artery were parallel. We assessed observer agreement between one observer and the NASCET 1991 principal neuroradiologist on 120 randomly selected angiograms (60 from ECST 1998 and 60 from NASCET 1991).

The degree of stenosis could not be calculated by the method used in the NASCET 1991 and VACSP 1991 trials on angiograms, in which the post‐stenotic internal carotid artery (ICA) was narrowed to the point of near occlusion. In the original NASCET 1991 reports, these 'near‐occlusions' were identified and assigned as 95% stenosis for the purpose of analysis. Therefore, we identified near‐occlusions during the reassessment of the ECST 1998 angiograms for this review. We used the previously reported NASCET 1991 angiographic criteria for near‐occlusion: severe stenosis with evidence of reduced flow in the distal ICA (delayed arrival of contrast into the distal ICA, or evidence of collateral flow of contrast towards the symptomatic cerebral hemisphere from other arterial territories, or both) and evidence of narrowing of the post‐stenotic ICA (lumen diameter similar to, or less than, the ipsilateral external carotid artery, and less than the contralateral ICA). To ensure comparability with NASCET 1991, the NASCET 1991 principal neuroradiologist assessed all potential near‐occlusions identified in ECST 1998. The VACSP 1991 trial angiograms were not available for further review, and were not included in the analysis of near‐occlusions. For the purpose of analysis, all VACSP 1991 trial angiograms with stenosis of 70% or more were considered to have stenosis of 70% or more without near‐occlusion.

Redefinition of outcome events

In the NASCET 1991 and VACSP 1991 trials, a stroke outcome was defined as a cerebrovascular event with symptoms lasting longer than 24 hours. ECST 1998 recorded all such events, but confined analysis to events with symptoms that lasted for at least seven days. In the NASCET 1991 and VACSP 1991 trials, retinal infarcts were included as stroke outcomes. In ECST 1998, they were not, although they were recorded. For the purpose of the combined analyses, we defined stroke as any cerebral or retinal event with symptoms lasting longer than 24 hours. ECST 1998 and NASCET 1991 used the modified Rankin Scale to define disabling stroke. VACSP 1991 used an equivalent 'in‐house' scale. Disability was defined at three months after the stroke in NASCET 1991, at six months in ECST 1998, and at the next routine follow‐up assessment in the VACSP 1991 trial.

The modified Rankin Scale is a scale for indicating the degree of disability or dependence in the daily activities of patients who have suffered a stroke or other causes of neurological disability (Farrell 1991). It is a 6‐point disability scale with possible scores ranging from 0 to 5. A score of 0 is no disability, 5 is disability requiring constant care for all needs; 6 is death. For the purposes of the combined analysis in this study, we defined disabling stroke as a stroke that resulted in a Rankin score of 3 or more, or equivalent, at these points of follow‐up.

Subgroup analysis and investigation of heterogeneity

To reduce the risk of chance findings, the collaborators met in 1999, prior to the pooling of data, to predefine a limited number of subgroups on the basis of potential clinical importance and availability in both trials (Rothwell 2004). These subgroups were derived from the risk factors that were predefined at the beginning of NASCET 1991. The following subgroup analyses were specified:

• men versus women;

• age (less than 65 years old versus 65 to 74 years old versus 75+ years);

• time from most recent symptomatic ischaemic event to randomisation (less than two weeks, two to four weeks, four to 12 weeks, more than 12 weeks);

• primary ischaemic event in the territory of the stenosed artery during the six months prior to randomisation; this was defined in a hierarchical manner as hemispheric stroke versus hemispheric TIA but no stroke versus retinal event only;

• diabetes versus no diabetes;

• irregular or ulcerated symptomatic carotid plaque versus smooth plaque on the pre‐randomisation angiogram; and

• contralateral carotid occlusion versus no occlusion.

To identify any important but unexpected treatment effect modifiers, we identified seven post‐hoc subgroup variables on the basis that comparable baseline data were available from the two trials:

• duration of cerebral TIA (one hour or less versus more than one hour);

• previous TIA or stroke (i.e. events prior to the six‐month eligibility period, as well as recent events);

• previous myocardial infarction;

• previous angina;

• treated hypertension (defined as that requiring a blood pressure‐lowering drug);

• treated hyperlipidaemia (defined as that requiring a dietary change or cholesterol‐lowering drug); and

• regular smoking during the previous year.

We first assessed the relationship between each subgroup variable and: 1) the risk of ipsilateral carotid territory ischaemic stroke in participants randomised to medical treatment (medical risk); and 2) any stroke or death that occurred within 30 days after trial surgery (perioperative risk). We determined the five‐year cumulative risks of ipsilateral ischaemic stroke in relation to each subgroup variable. We also determined the associations in a Cox proportional hazards model with adjustment for source trial and degree of carotid stenosis.

Determination of the significance of treatment effect modification by subgroup was complicated by the differential changes in event rate with time in the two treatment groups. Nevertheless, we initially performed a Cox proportional hazards model with treatment allocation, a source study term, degree of stenosis, a subgroup by treatment allocation interaction term, and a stenosis by treatment allocation interaction term. We also performed an additional test for trend for the analysis of the effects of age and time from last event to randomisation. To maximise statistical power to detect treatment effect modification by subgroup, we performed these analyses of subgroup by treatment interaction across all degrees of stenosis. We considered it unlikely that the direction of any treatment effect modification by subgroup would differ qualitatively with degree of stenosis. However, we performed a further Cox model to test the significance of the three‐way interaction between stenosis, subgroup, and treatment allocation.

Sensitivity analysis

When the decisions for the process undertaken in this systematic review were somewhat arbitrary or unclear, we undertook sensitivity analyses. For example, we performed both fixed‐effect and random‐effects meta‐analyses to evaluate the consistency of the results, or we compared pooled estimates of all studies' results with the results of the studies excluded because of higher risk of bias.