Types of studies

We included cross‐sectional studies where the index test(s) and reference standard were administered within one month of each other.

We excluded studies with a small number of cases (fewer than 10), as these studies are prone to various selection biases.

Case‐control studies in which participants are selected to undergo a telehealth assessment on the basis of their cognitive diagnostic status were not eligible for inclusion.

Participants

Our population of interest is any adult (aged over 18 years) with a suspected cognitive impairment who has presented to a specialist service to be assessed for dementia or mild cognitive impairment (MCI).
Studies in populations with and without a routine non‐specialist assessment before presentation at the specialist assessment service were eligible. This was considered to be a potential source of heterogeneity (since pre‐assessment probability of dementia will differ).

Studies of selected populations (e.g. studies in traumatic brain injury or in stroke) were also eligible, again noting this as a potential source of heterogeneity.

Index tests

Our index test of interest is a criterion‐based diagnosis of dementia or MCI made by means of an assessment in which all contact between the person with suspected cognitive impairment and the clinician responsible for making or excluding the diagnosis of dementia or MCI is by telephone or videoconference technologies. Although there is no face‐to‐face contact, the diagnostic assessment must involve a human‐to‐human interface; diagnostic assessments made using computer‐based algorithms without human contact were not eligible for inclusion.

Some defined elements of the assessment (assistance with technology, standardised cognitive testing, physical assessment) may be conducted face‐to‐face by a trained person (e.g. a nurse, primary healthcare worker), as long as this person is not responsible for synthesising the clinical information and making a diagnostic judgement. The degree of face‐to‐face contact in the index test assessment process was considered as a source of heterogeneity. If the same information obtained face‐to‐face is also used in the reference standard assessment, then we considered the effect of possible incorporation bias as part of our quality assessment.

Target conditions

Our target conditions are as follows.

1. All‐cause dementia, diagnosed using validated criteria (e.g. ICD or DSM).

2. All subtypes of dementia, diagnosed using validated criteria.

3. Mild cognitive impairment (MCI) diagnosed using any system which incorporates evidence of cognitive decline and preserved functional independence, including DSM‐5 criteria for Minor Neurocognitive Disorder and National Institute on Aging/Alzheimer’s Association (NIA‐AA) criteria for MCI due to Alzheimer’s disease.

Reference standards

Our reference standard is a diagnosis of dementia or MCI made by means of an assessment which is conducted primarily face‐to‐face and, in particular, which involves face‐to‐face contact with the clinician responsible for making or excluding a diagnosis of dementia or MCI. We anticipated that this assessment would be the usual standard of care in the study setting, and we accepted any assessment protocol which is sufficient to make a diagnosis according to validated diagnostic criteria for dementia. Criteria for making a diagnosis of MCI are less well‐established, but the same assessment protocol is required. All reference standard assessments have to include as a minimum: a history from the patient and, wherever possible, from a knowledgeable informant, including an account and/or structured assessment of daily functioning; an objective assessment of cognition; and a mental state examination sufficient to exclude alternative psychiatric causes of cognitive symptoms, such as depression. Blood tests and special investigations, such as neuroimaging, if done, are often conducted prior to memory clinic assessment. So that patients do not have unnecessary repetition of invasive investigations, any such investigations are expected to contribute to diagnostic decisions made at both face‐to‐face and telehealth assessments, and are therefore not considered to be part of either the index test or the reference standard. We recorded how many patients in each study underwent neuroimaging and considered this as a potential source of between‐study heterogeneity.

The reference standard assessment had to take place in secondary care. Settings described as primary care were only eligible if the assessment was provided as an enhanced service (e.g. by general practitioner (GP) with a special interest who has undertaken additional training and who receives referrals from other GPs); we considered this model to be equivalent to a secondary care model.

The assessment may involve a multidisciplinary team, that is the person making the diagnosis need not conduct all aspects of the assessment personally. Use of information derived from pre‐assessment questionnaires and remote contact with an informant who is not able to attend in person are acceptable components of a reference standard assessment. If these materials also featured in the index test, then we considered the effect of incorporation bias as part of our quality assessment.

Masking of the reference standard assessment team to the index test outcome was an important part of the quality assessment.

Studies that make a postmortem diagnosis or base diagnosis on imaging or other biomarkers without a comprehensive clinical assessment were not eligible.

Electronic searches

We searched MEDLINE (OvidSP) (1946 to present), Embase (OvidSP) (1974 to present), Web of Science Core Collection (Clarivate Analytics) (1900 to present), PsycINFO (Ovid) (1806 to present), and LILACS (BIREME) (Latin American and Caribbean Health Science Information database) (1982 to present) on 4 November 2020. These databases cover grey literature that is appropriate to the topic of this review. We searched the following trial registers: US National Institutes of Health Ongoing Trials Register ClinicalTrials.gov (www.clinicaltrials.gov/) and World Health Organization International Clinical Trials Registry Platform (apps.who.int/trialsearch/). (See Appendix 1 for the search strategy we ran in MEDLINE). We used controlled vocabulary such as MeSH terms and EMTREE where appropriate. In the searches developed, we made no attempt to restrict studies on the basis of sampling frame or setting. This approach is intended to maximise sensitivity and allow inclusion on the basis of population‐based sampling to be assessed at screening (see Selection of studies). We did not use search filters (collections of terms aimed at reducing the number need to screen) as an overall limiter because those that are published have not proved sensitive enough (Whiting 2008). We did not apply any language restriction to the electronic searches, using translation services as necessary. We searched ALOIS, the CDCIG Specialized Register, which includes both intervention and diagnostic test accuracy (DTA) studies in dementia. We performed forward and backward citation searching from potentially relevant or included papers.

Searching other resources

We included only papers published in peer‐reviewed scientific journals, so our searching of the grey literature was limited to the electronic databases described.

Selection of studies

The search was run by the Cochrane Dementia & Cognitive Improvement Group's Information Specialist, who removed duplicates and conducted a 'first pass' assessment to exclude obviously irrelevant records. She imported the remaining citations into Covidence. Two review authors (JMcC and JL), working independently, first assessed the titles and abstracts of the citations in Covidence, and then assessed the full text of any article thought by either review author to be potentially eligible for inclusion in the review.

For consistency with our other DTA titles, we adopted a hierarchical approach to exclusion, first excluding on the basis of index test and reference standard, then on the basis of study design, and then for any other reason.

Where the available material (trial registry entry, conference abstract, or full paper) did not include the data required to judge the eligibility of a study or the data needed for analyses, we contacted the lead author to request the necessary information.

We detailed the study selection process in a PRISMA flow diagram (Figure 1).

Figure 1

---

---

Study flow diagram.

Data extraction and management

Two review authors (JMcC and JL) independently extracted the data from eligible papers to a bespoke data extraction form. The form was tested on two papers and modified as necessary.

The data extraction form covered: population (including country, demographics, setting, eligibility criteria, pre‐referral screening processes, special characteristics (e.g. specific health conditions)); details of index test (including person conducting and setting of all components of assessment); details of reference standard (including person conducting and setting of all components of assessment); order of implementation of index test and reference standard. We sought numbers of potentially eligible participants who were excluded or who declined participation to get an idea of generalisability and acceptability.

We extracted data on categorical diagnoses of all‐cause dementia, subtypes of dementia (divided into probable and possible if reported), and MCI.

We sought and extracted data on any quantitative measures of patient or clinician satisfaction, resource use (e.g. time taken), feasibility (e.g. the number of incomplete assessments) and cost.

We did not seek to collect qualitative data on users' experience of remote and face‐to‐face assessments for this review.

The two review authors compared their extracted data and resolved initial disagreements through discussion.

Assessment of methodological quality

Two review authors (JMcC and JL), blinded to each other’s scores, assessed risk of bias (internal validity) and generalisability (external validity) using a modified version of the Quality Assessment tool for Diagnostic Accuracy Studies (QUADAS‐2) tool (www.bristol.ac.uk/population-health-sciences/projects/quadas/quadas-2/).

QUADAS‐2 assessment covers issues relating to patient selection, index test, reference standard, and participant flow. Each domain is assessed for issues which may introduce bias, and the first three domains are also assessed for generalisability concerns. We based our risk of bias assessment on a previously described, modified version of the generic QUADAS‐2 tool that was developed for studies of neuropsychological tests in dementia, and includes operationalised scoring rules at item and domain level (Davis 2013). This tool was further tailored to this review by adding a signalling question to the reference standard domain to address risk of incorporation bias, and by removing from the index test domain a signalling question on prespecifying test thresholds which is not relevant to this review (Appendix 2).

A particular concern for this review is incorporation bias: are the telehealth and face‐to‐face assessments based on the same data? The optimal study design for our purposes is where all the data informing the two assessments are acquired by different means for the index test and the reference standard. However, in some circumstances, there may be a degree of overlap (e.g. cognitive testing conducted face‐to‐face in an index test assessment, or informant history obtained remotely in some reference standard assessments) (see Index tests and Reference standards). In such circumstances, it is possible that the same data will inform both assessments. We considered incorporation bias as part of our QUADAS‐2 assessment by means of a specific signalling question in the index test domain. We also considered any element of face‐to‐face assessment in the index test as a potential source of heterogeneity to be examined using subgroup analysis.

We present QUADAS‐2 results as graphical displays and as a narrative within the text. There were not enough included studies to perform a sensitivity analysis limited to those papers with low concern for risk of bias in the index test and reference standard domains (pre‐specified because of the critical importance of index test and reference standard assessments being interpreted ‘blind’).

Statistical analysis and data synthesis

In practice, the diagnosis of dementia and MCI is made at various levels of precision, for example differentiating dementia and non‐dementia, assigning a pathological subtype, and describing the certainty of the formulation (possible or probable subtype diagnoses). We proposed a hierarchy of analyses so as to make the best use of all available information. In practice, the data allowed only analyses at the first level of the hierarchy.

• First, we described accuracy at the level of dementia versus not‐dementia and MCI versus not‐MCI, where dementia covers all‐cause (undifferentiated) dementia and all dementia subtypes, and MCI is undifferentiated by cause. These were our primary analyses. We also conducted an analysis looking at the accuracy of identification of any cognitive impairment syndrome (dementia and MCI combined) versus no cognitive impairment syndrome.

• Second, we planned to perform an analysis limited to those papers that describe dementia subtypes, assessing accuracy of the subtyping.

• Third, if data permitted, we planned to perform analyses that looked at both dementia diagnosis and certainty (e.g. possible and probable Alzheimer’s disease dementia).

In all analyses, a 'positive' result is a clinician's criterion‐based diagnosis of dementia or a subtype of dementia or of MCI (as defined above in Target conditions), and a 'negative' result is the absence of such a diagnosis. We constructed 2x2 tables for each study and used the data to calculate sensitivities and specificities, with 95% confidence intervals, employing Review Manager 2014 software. We present the results graphically in forest plots of sensitivity and corresponding specificity.

We did not calculate summary estimates of sensitivity and specificity because of the small number of included studies and the clinical differences between them. (See Differences between protocol and review).

The data needed to populate a 2x2 table were not available for one study. For this study, we report the Kappa statistic for agreement between face‐to‐face and telehealth diagnoses, as presented in the original study publication.

Investigations of heterogeneity

There were insufficient data to conduct any of the planned subgroup analyses to explore the following potential areas of heterogeneity:

1. population: special populations, referral pathway (self‐referral or referral after a non‐specialist assessment), severity of dementia, proportion undergoing neuroimaging;

2. index test: technology used, degree of face‐to‐face assessment;

3. reference standard: subtype of dementia.

Sensitivity analyses

We did not conduct any sensitivity analyses.

Assessment of reporting bias

We did not plan to perform a quantitative assessment of reporting bias. We recognise that there is a lack of consensus about the most robust approach to assessment of reporting bias in DTA (Wilson 2015), and uncertainty about how to apply standard approaches such as funnel plots (van Enst 2014).