Was a consecutive or random sample of patients enrolled?

Where sampling is used, the methods least likely to cause bias are consecutive sampling or random sampling, which should be stated and/or described. Non‐random sampling or sampling based on volunteers is more likely to be at high risk of bias.

Weighting: high risk of bias

Was a case‐control design avoided?

Case‐control study designs have a high risk of bias, but sometimes they are the only studies available especially if the index test is expensive and/or invasive. Nested case‐control designs (systematically selected from a defined population cohort) are less prone to bias but they will still narrow the spectrum of patients that receive the index test. Study designs (both cohort and case‐control) that may also increase bias are those designs where the study team deliberately increase or decrease the proportion of participants with the target condition, for example a population study may be enriched with extra dementia participants from a secondary care setting.

Weighting: high risk of bias

Did the study avoid inappropriate exclusions?

We will automatically grade the study as unclear if exclusions are not detailed (pending contact with study authors). Where exclusions are detailed, we will grade the study as 'low risk' if exclusions are felt to be appropriate by the review authors. Certain exclusions common to many studies of dementia are: medical instability; terminal disease; alcohol/substance misuse; concomitant psychiatric diagnosis; other neurodegenerative condition. However if 'difficult to diagnose' groups are excluded this may introduce bias, so exclusion criteria must be justified. For a community sample we would expect relatively few exclusions. We will label post hoc exclusions 'high risk' of bias.

Weighting: high risk of bias

Were the index test results interpreted without knowledge of the reference standard?

Terms such as 'blinded' or 'independently and without knowledge of' are sufficient and full details of the blinding procedure are not required. This item may be scored as 'low risk' if explicitly described or if there is a clear temporal pattern to the order of testing that precludes the need for formal blinding, i.e. all (neuropsychological test) assessments were performed before the dementia assessment. As most neuropsychological tests are administered by a third party, knowledge of dementia diagnosis may influence their ratings; tests that are self administered, for example using a computerized version, may have less risk of bias.

Weighting: high risk of bias

Were the index test thresholds pre‐specified?

For neuropsychological scales there is usually a threshold above which participants are classified as 'test positive'; this may be referred to as threshold, clinical cut‐off or dichotomiation point. Different thresholds are used in different populations. A study is classified as at higher risk of bias if the authors define the optimal cut‐off post hoc based on their own study data. Certain papers may use an alternative methodology for analysis that does not use thresholds and these papers should be classified as not applicable.

Weighting: low risk of bias

Were sufficient data on (neuropsychological test) application given for the test to be repeated in an independent study?

Particular points of interest include method of administration (for example self completed questionnaire versus direct questioning interview); nature of informant; language of assessment. If a novel form of the index test is used, for example a translated questionnaire, details of the scale should be included and a reference given to an appropriate descriptive text, and there should be evidence of validation.

Weighting: low risk of bias

Is the reference standard likely to correctly classify the target condition?

Commonly used international criteria to assist with clinical diagnosis of dementia include those detailed in DSM‐IV and ICD‐10. Criteria specific to dementia subtypes include but are not limited to NINCDS‐ADRDA criteria for Alzheimer's dementia; McKeith criteria for Lewy Body dementia; Lund criteria for frontotemporal dementias; and the NINDS‐AIREN criteria for vascular dementia. Where the criteria used for assessment are not familiar to the review authors and Cochrane Dementia and Cognitive Improvement, this item should be classified as 'high risk of bias'.

Weighting: high risk of bias

Were the reference standard results interpreted without knowledge of the results of the index test?

Terms such as 'blinded' or 'independent' are sufficient and full details of the blinding procedure are not required. This may be scored as 'low risk' if explicitly described or if there is a clear temporal pattern to order of testing, i.e. all dementia assessments performed before (neuropsychological test) testing.

Informant rating scales and direct cognitive tests present certain problems. It is accepted that informant interview and cognitive testing is a usual component of clinical assessment for dementia, however specific use of the scale under review in the clinical dementia assessment should be scored as high risk of bias.

Weighting: high risk of bias

Was sufficient information on the method of dementia assessment given for the assessment to be repeated in an independent study?

Particular points of interest for dementia assessment include the training/expertise of the assessor, whether additional information was available to inform the diagnosis (for example neuroimaging, other neuropsychological test results), and whether this was available for all participants.

Weighting: variable risk, but high risk if method of dementia assessment not described

Was there an appropriate interval between the index test and reference standard?

For a cross‐sectional study design, there is potential for the subject to change between assessments, however dementia is a slowly progressive disease, which is not reversible. The ideal scenario would be a same‐day assessment, but longer periods of time (for example, several weeks or months) are unlikely to lead to a high risk of bias. For delayed‐verification studies the index and reference tests are necessarily separated in time given the nature of the condition.

Weighting: low risk of bias

Did all participants receive the same reference standard?

There may be scenarios where participants who score 'test positive' on the index test have a more detailed assessment for the target condition. Where dementia assessment (or reference standard) differs between participants this should be classified as high risk of bias.

Weighting: high risk of bias

Were all participants included in the final analysis?

Attrition will vary with study design. Delayed verification studies will have higher attrition than cross‐sectional studies due to mortality, and it is likely to be greater in participants with the target condition. Dropouts (and missing data) should be accounted for. Attrition that is higher than expected (compared to other similar studies) should be treated as a high risk of bias. We have defined a cut‐off of greater than 20% attrition as being high risk but this will be highly dependent on the length of follow‐up in individual studies.

Weighting: high risk of bias