Exfoliative cytology for diagnosing basal cell carcinoma and other skin cancers in adults

Summary of findings Summary of findings table

Question:	What is the diagnostic accuracy of exfoliative cytology for detecting BCC, cSCC or cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults?
Population	Adults with lesions suspicious for BCC, cSCC or for melanoma
Index test	Exfoliative cytology
Comparator test	Dermoscopy
Target condition	BCC
Reference standard	Histology, any method
Action	If accurate, positive diagnosis by exfoliative cytology would reduce the need for biopsies in suspected BCC and help to appropriately select lesions for excision
Quantity of evidence
Number of studies	9	Total lesions with test results	1655	Total with BCC	1120^a
				Total with cSCC	41^b
				Total with melanoma	10^c
Limitations
Risk of bias	High risk for patient selection due to case‐control study design (2/9) or inappropriate exclusion of lesions (1/9), and unclear due to poor reporting of recruitment and exclusion criteria (3/9). Unclear risk for the index test due to lack of reporting diagnostic thresholds and blinding from the reference standard diagnosis (7/9). Unclear risk of bias due to inadequate reporting of blinding the reference standard (7/9) or the index test (7/9). High risk of bias in flow and timing domain from differential verification (2/9) and exclusion of slides from analysis (1/9); timing of tests was not mentioned in 7/9.
Applicability of evidence to question	High concern due to narrowly defined populations and multiple lesions per patient (6/9), and unclear concern due to poor reporting of patient groups (2/9), so may not be representative of populations eligible for exfoliative cytology. High concern for clinical applicability of exfoliative cytology from lack of reporting cytodiagnostic criteria in adequate detail (5/9). Little information was given concerning the expertise of the cytopathologist or histopathologist.
Detection of BCC: pooled analysis^d
Datasets	Lesions	BCCs	Sensitivity (95% CI)	Specificity (95% CI)
7	1264	1045	97.5% (94.5 to 98.9)	90.1% (81.1 to 95.1)
Numbers observed in a cohort of 1000 people being tested^e
	True positive	False negative	False positive	True negative
	(Appropriately do not receive excision)	(Inappropriately receive excision or undertreated)	(Inappropriately do not receive excision, or overtreated)	(Receive appropriate management – excision or other)
At prevalence 63%	614	16	37	333
At prevalence 86%	839	21	14	126
At prevalence 88%	858	22	12	108
Detection of BCC: pooled analysis^f
Datasets	Lesions	BCCs	Sensitivity (95% CI)	Specificity (95% CI)
7	1264	1045	97.3% (93.5 to 98.9)	94.2% (88.7 to 97.1)
Detection of cSCC, melanoma, any skin cancer
Findings	Studies also evaluated cSCC (2 studies), melanoma (1 study) or any skin cancer (6 studies). cSCC – studies could not be pooled due to different diagnostic approaches; sensitivity ranged from 89% to 100% and specificity from 75% to 99% melanoma – only study (10 melanomas) conducted in 185 pigmented skin lesions, also providing a comparison with dermoscopy: sensitivity and specificity 100% any skin cancer – 4 studies pooled 573 suspicious lesions, with 495 malignant lesions (476 BCCs, 13 cSCCs, 1 melanoma, 4 carcinomas of unspecified histological type, 1 apocrine carcinoma). Pooled sensitivity 97.3% (95% CI 93.5% to 98.9%) and specificity 86.0% (95% CI 73.5% to 93.1%) (uncertain diagnoses classified as test positives). When uncertain diagnoses classified as test negatives, pooled sensitivity became 96.6% (95% CI 90.3% to 98.9%) and specificity 94.7% (95% CI 80.2% to 98.7%).
BCC: basal cell carcinoma; cSCC: cutaneous squamous cell carcinoma; CI: confidence interval. ^aTotal of 1122 BCC cases, of which 2 excluded due to absence of exfoliative cytology result ('test fails'). ^bTotal of 55 cSCC cases, of which 14 excluded: 3 due to absence of exfoliative cytology result ('test fails') and 11 due to insufficient cSCC lesion numbers in individual studies (< 5 cSCCs per study). ^cTotal of 11 cases, of which 1 excluded due to insufficient melanoma lesion numbers in individual studies (< 5 melanomas per study). ^d'Possible BCC' cases classified as index test positive. ^eNumbers for a hypothetical cohort of 1000 lesions are presented for three examples representing different prevalences of BCC, estimated at 25th, 50th (median) and 75th percentiles of BCC prevalence observed across the 9 included studies. ^f'Possible BCC' cases classified as index test negative.

Background

This review is one of a series of Cochrane Diagnostic Test Accuracy (DTA) reviews on the diagnosis and staging of melanoma and keratinocyte skin cancers conducted for the National Institute for Health Research (NIHR) Cochrane Systematic Reviews Programme. Appendix 1 shows the content and structure of the programme. Appendix 2 provides a glossary of terms used and a table of acronyms used is provided in Appendix 3.

Target condition being diagnosed

The commonest skin cancers in white populations are keratinocyte skin cancers, namely basal cell carcinoma (BCC) and cutaneous squamous cell carcinoma (cSCC) (Gordon 2013; Madan 2010). BCC is the more common of the two keratinocyte carcinomas, and approximately one third of people with a BCC will develop at least one other BCC over time (Flohill 2013). In 2003, the World Health Organization (WHO) estimated that between 2 and 3 million 'non‐melanoma' skin cancers occur globally each year (of which BCC and cSCC are estimated to account for around 80% and 16% of cases, respectively) and 132,000 melanoma skin cancers occur globally each year (WHO 2003). Rather than defining BCC and cSCC by what they are not (i.e. non‐melanoma skin cancer), we collectively refer to these conditions using the preferred and more accurate term of 'keratinocyte carcinoma' in this DTA review (Karimkhani 2015).

Exfoliative cytology is a simple procedure designed to detect the presence of malignancy through analysis of cell structure. Since its main benefit would be to replace histology, basal cell carcinoma has been chosen as the primary target condition for this review since this is the condition for which exfoliative cytology could potentially have the clearest role (see Role of index test(s) and Rationale below). Secondary target conditions include: cSCC,invasive melanoma and atypical intraepidermal melanocytic variants, and any other skin cancer, including keratinocyte skin cancer, invasive melanoma and atypical intraepidermal melanocytic variants.

Basal cell carcinoma

BCC can arise from multiple stem cell populations, including from the bulge and interfollicular epidermis (Grachtchouk 2011). Growth is usually localised, but it can infiltrate and damage surrounding tissue, and if left untreated it can cause considerable destruction and disfigurement, particularly when located on the face (Figure 1). The four main subtypes of BCC are superficial, nodular, morphoeic or infiltrative, and pigmented. They typically present as slow‐growing asymptomatic papules, plaques, or nodules that may bleed or form ulcers that do not heal (Firnhaber 2012). People with a BCC often present to healthcare professionals with a non‐healing lesion rather than specific symptoms such as pain. Clinicians frequently make the diagnosis incidentally rather than as a result of people presenting with symptoms (Gordon 2013).

Figure 1

Sample photographs of BCC (left) and cSCC (right). Copyright © 2012 Dr Rubeta Matin: reproduced with permission.

BCCs most frequently occur on sun‐exposed areas of the head and neck (McCormack 1997), and they are more common in men and in people over 40 years of age. A rising incidence of BCC in younger people has been attributed to increased recreational sun exposure (Bath‐Hextall 2007a; Gordon 2013; Musah 2013). Other risk factors include Fitzpatrick skin types I and II (Fitzpatrick 1975; Lear 1997; Maia 1995); previous skin cancer history; immunosuppression; arsenic exposure; and genetic predisposition, such as in basal cell naevus (Gorlin) syndrome (Gorlin 2004; Zak‐Prelich 2004). Annual incidence is increasing worldwide; Europe has experienced an average increase of 5.5% per year over the last four decades, the USA 2% per year, while estimates for the UK show incidence appears to be increasing more steeply at a rate of an additional 6/100,000 person‐years (Lomas 2012). Some authors have explained the rising incidence by an ageing population, changes in the distribution of known risk factors, particularly ultraviolet radiation, and improved detection due to the increased awareness amongst both practitioners and the general population (Verkouteren 2017). Hoorens 2016 points to evidence for a gradual increase in the size of BCCs over time, with delays in diagnosis ranging from 19 to 25 months.

According to the National Institute for Health and Care Excellence (NICE) guidance (NICE 2010), low‐risk BCCs that may be considered for excision include nodular lesions occurring in patients older than 24 years old who are not immunosuppressed and do not have Gorlin syndrome. Furthermore, lesions should be located below the clavicle; should be small (diameter of less than 1 cm), with well‐defined margins; not recurrent following incomplete excision; and not in awkward or highly visible locations (NICE 2010). Superficial BCCs are also typically low risk and may be amenable to medical treatments such as photodynamic therapy or topical chemotherapy (Kelleners‐Smeets 2017). Assigning BCCs as low or high risk influences the management options (Batra 2002; Randle 1996).

It is recognised that basosquamous carcinoma (more like a high risk SCC in behaviour and not considered a true BCC) is likely to have accounted for many cases of apparent metastases of BCC, hence the spuriously high reported incidence in some studies of up to 0.55%, which is not seen in clinical practice (Garcia 2009).

Advanced locally destructive BCC can arise from long‐standing untreated lesions or from a recurrence of a basal cell carcinoma after primary treatment (Lear 2012). Very rarely, BCC metastasises to regional and distant sites resulting in death, especially cases of large neglected lesions in those who are immunosuppressed or those with Gorlin syndrome (McCusker 2014). Rates of metastasis are reported at 0.0028% to 0.55% (Lo 1991), with very poor survival rates.

Squamous cell carcinoma of the skin (cSCC)

Primary cSCC arises from the keratinising cells of the outermost layer of the skin. People with cSCC often present with an ulcer or firm (indurated) papule, plaque or nodule (Firnhaber 2012; Griffin 2016), sometimes with an adherent crust and poorly defined margins (Madan 2010). cSCC can arise in the absence of a precursor lesion or it can develop from pre‐existing actinic keratosis, with an estimated annual risk of progression of anywhere from under 1% to 20% (Alam 2001), or Bowen's disease (squamous cell carcinoma in situ), with about a 5% risk of progression (Kao 1986). It remains locally invasive for a variable length of time but has the potential to spread to the regional lymph nodes or via the bloodstream to distant sites, especially in immunosuppressed individuals (Lansbury 2010). High risk lesions are those arising on the lip or ear, recurrent cSCC, lesions arising on non‐exposed sites, scars or chronic ulcers, tumours more than 20 mm in diameter and depth of invasion more than 4 mm and poor differentiation on pathological examination (Motley 2009).

Chronic ultraviolet light exposure through recreation or occupation is strongly linked to cSCC occurrence (Alam 2001). It is particularly common in people with fair skin and in rare genetic disorders of pigmentation, such as albinism, xeroderma pigmentosum and recessive dystrophic epidermolysis bullosa (RDEB) (Alam 2001). Other recognised risk factors include immunosuppression; chronic wounds; arsenic or radiation exposure; certain drug treatments, such as voriconazole and BRAF inhibitors; and previous skin cancer history (Baldursson 1993; Chowdri 1996; Dabski 1986; Fasching 1989; Lister 1997; Maloney 1996; O'Gorman 2014). In transplant recipients, cSCC is the most common form of skin cancer, with estimates of the risk of developing cSCC 65 to 253 times that of the general population (Hartevelt 1990; Jensen 1999; Lansbury 2010). Overall, local and metastatic recurrence of cSCC at five years is estimated at 8% and 5%, respectively. Five‐year survival rate following metastatic recurrence is only 25% to 40% (Rowe 1992).

Melanoma

Melanoma arises from uncontrolled proliferation of melanocytes – the epidermal cells that produce pigment or melanin. Cutaneous melanoma refers to skin lesions with malignant melanocytes present in the dermis, and includes superficial spreading, nodular, acral lentiginous, and lentigo maligna melanoma variants. Melanoma in situ describes malignant melanocytes that lay within the epidermis without invasion of the dermis, but they are at risk of progressing to melanoma if left untreated. Lentigo maligna, a subtype of melanoma‐in‐situ in chronically sun‐damaged skin, can progress to invasive melanoma if its growth breaches the dermo‐epidermal junction during a vertical growth phase (when it becomes known as 'lentigo maligna melanoma'), however its malignant transformation is both lower and slower than for melanoma in situ (Kasprzak 2015). Melanoma in situ and lentigo maligna are both atypical intraepidermal melanocytic variants. Melanoma is one of the most dangerous forms of skin cancer, with the potential to metastasise to other parts of the body via the lymphatic system and blood stream. It accounts for only a small percentage of skin cancer cases but is responsible for up to 75% of skin cancer deaths (Boring 1994; Cancer Research UK 2017).

The incidence of melanoma rose to over 200,000 newly diagnosed cases worldwide in 2012 (Erdmann 2013; Ferlay 2015), with an estimated 55,000 deaths (Ferlay 2015). The highest incidence is observed in Australia with 13,134 new cases of melanoma of the skin in 2014 (ACIM 2017) and in New Zealand with 2341 registered cases in 2010 (HPA and MelNet NZ 2014). For 2014 in the USA, the predicted incidence was 73,870 per annum and the predicted number of deaths was 9940 (Siegel 2015). The highest rates in Europe are seen in north‐western Europe and the Scandinavian countries, with a highest incidence reported in Switzerland: 25.8 per 100,000 in 2012. Rates in England have tripled from 4.6 and 6.0 per 100,000 in men and women, respectively, in 1990, to 18.6 and 19.6 per 100,000 in 2012 (EUCAN 2012). Indeed, in the UK, melanoma has one of the fastest rising incidence rates of any cancer, and has had the biggest projected increase in incidence between 2007 and 2030 (Mistry 2011). In the decade leading up to 2013, age‐standardised incidence increased by 46%, with 14,500 new cases in 2013 and 2459 deaths in 2014 (Cancer Research UK 2017). Rates are higher in women than in men; however, the rate of incidence in men is increasing faster than in women (Arnold 2014). This rising incidence is thought to be primarily related to an increase in recreational sun exposure, tanning bed use and an increasingly ageing population with higher lifetime recreational ultraviolet (UV) exposure, in conjunction with possible earlier detection (Belbasis 2016; Linos 2009). Putative risk factors are reviewed in detail elsewhere (Belbasis 2016).

A database of over 40,000 US patients from 1998 onwards, which assisted the development of the 8th American Joint Committee on Cancer (AJCC) staging system, indicated a five‐year survival of 97% to 99% for stage I melanoma, dropping to between 32% and 93% in stage III disease depending on tumour thickness, the presence of ulceration and number of involved nodes (Gershenwald 2017). While these are substantial increases relative to survival in 1975 (Cho 2014), mortality rates have remained static during the same period. This observation, coupled with increasing incidence of localised disease, suggests that improvements in survival may be due to earlier detection and heightened vigilance (Cho 2014). New targeted therapies for advanced (stage IV), melanoma (e.g. BRAF inhibitors), have improved survival, and immunotherapies are evolving such that long‐term survival is being documented (Pasquali 2018; Rozeman 2017). No new data regarding the survival prospects for patients with stage IV disease were analysed for the AJCC 8 staging guidelines due to lack of contemporary data (Gershenwald 2017).

Treatment

Treatment for BCC and cSCC include surgery, other destructive techniques such as cryotherapy or electrodesiccation and topical chemotherapy. A Cochrane Review of 27 randomised controlled trials (RCTs) of interventions for BCC found very little good quality evidence for any of the interventions used (Bath‐Hextall 2007b). Complete surgical excision of primary BCC has a reported five‐year recurrence rate of less than 2% (Griffiths 2005; Walker 2006), leading to significantly fewer recurrences than treatment with radiotherapy (Bath‐Hextall 2007b). After apparent clear histopathological margins (serial vertical sections) following standard excision biopsy with 4 mm surgical peripheral margins taken, reported five‐year recurrence rate is around 4% (Drucker 2017). Mohs micrographic surgery, whereby horizontal sections of the tumour undergo histological analysis, and re‐excisions are made until the margins are tumour‐free, can be considered for high‐risk lesions such as on the centre of the face, where standard wider excision margins might lead to considerable functional impairment (Bath‐Hextall 2007b; Lansbury 2010; Motley 2009; Stratigos 2015). Bath‐Hextall and colleagues (Bath‐Hextall 2007b) found a single trial comparing Mohs micrographic surgery with a 3mm surgical margin excision in BCC (Smeets 2004); the update of this study showed non‐significantly lower recurrence at 10 years with Mohs micrographic surgery (4.4% compared to 12.2% after surgical excision, P = 0.10) (van Loo 2014).

Destructive techniques other than excisional surgery include electrodesiccation and curettage (ED&C) as well as cryotherapy (Alam 2001; Bath‐Hextall 2007b). Alternatively, non‐surgical (or non‐destructive) treatments may be options (Bath‐Hextall 2007b; Kim 2014; Drew 2017), including topical chemotherapy such as imiquimod (Williams 2017), 5‐fluorouracil (Arits 2013), ingenol mebutate (Nart 2015), and photodynamic therapy (Bath‐Hextall 2007b;Roozeboom 2016). These non‐surgical approaches are increasingly used for the superficial subtypes of BCC, for multiple lesions on low‐risk sites, where there are relevant comorbidities, or where surgery would be associated with risk of poor wound healing or significant scarring (Marsden 2010). However, non‐surgical techniques do not allow histological confirmation of tumour clearance, and their use is dependent on accurate characterisation of the histological subtype and depth of tumour. The 2007 systematic review of BCC interventions found limited evidence from very small RCTs for these approaches (Bath‐Hextall 2007b), which have only partially been filled by subsequent studies (Bath‐Hextall 2014; Kim 2014; Roozeboom 2012). Most BCC trials have compared interventions within the same treatment class, and few have compared medical versus surgical treatments (Kim 2014).

A systematic review of interventions for primary cSCC found only one RCT eligible for inclusion (Lansbury 2010). Current practice therefore relies on evidence from observational studies, as reviewed in Lansbury 2013, for example. Surgical excision with predetermined margins is usually the first‐line treatment (Motley 2009; Stratigos 2015). Observational studies suggest low recurrence rates for small, low‐risk lesions treated with cryotherapy or ED&C (recurrence rates of less than 2%). Estimates of recurrence after Mohs micrographic surgery, surgical excision, or radiotherapy, which researchers are likely to have evaluated in higher risk populations, have shown pooled recurrence rates of 3%, 5.4% and 6.4%, respectively, with overlapping confidence intervals; the review authors advise caution when comparing results across treatments (Lansbury 2013).

For primary melanoma, the mainstay of definitive treatment is wide local excision of the lesion, to remove both the tumour and any malignant cells that might have spread into the surrounding skin (Garbe 2016; Marsden 2010; NICE 2015a; SIGN 2017; Sladden 2009). Recommended surgical margins vary according to tumour thickness, as described in Garbe 2016, and by stage of disease at presentation, as in NICE 2015a. Following histological confirmation of diagnosis, the lesion is pathologically staged from 0 (referring to melanoma in situ) to IV (indicating the presence of distant metastasis) according to the AJCC staging system to guide treatment (Balch 2009). The main prognostic indicators can be divided into histological and clinical factors. Histologically, Breslow thickness is the single most important predictor of survival, as it is a quantitative measure of tumour invasion which correlates with the propensity for metastatic spread (Balch 2001). Independent of tumour thickness, prognosis is worse in older people, males, those with recurrent lesions, and in those with distant lymph node involvement (micro or macroscopic) and/or metastatic disease at the time of primary presentation.

Index test(s)

Exfoliative cytology is a non‐invasive test that uses the Tzanck smear technique to identify disease through the examination of the structure of cells (Tzank 1949). It is also known as 'skin scrape cytology', which is perhaps a better description of the technique than 'exfoliative' which traditionally refers to the removal of superficial dead cells from the skin surface. Clinicians clean skin lesions, remove any surface crust, and then scrape the lesions with a scalpel or curette to collect cell material and subsequently smear them onto one or more glass slides (Chandra 2009). They can then fix the material using alcohol or air‐drying, and then they stain it using one of several methods recommended by the British Society of Cytopathology, namely the Papanicolaou (Pap) and May‐Grünwald Giemsa (MGG; also called Romanowsky) methods (Chandra 2009). A cytopathologist or a dermatologist with experience of the technique can immediately examine the slides under a microscope to determine the presence of malignant cells (Bakis 2004). Superficial shave biopsy differs from a cytological scrape in that it slices off a superficial (largely epidermal) section from a BCC that protrudes above the skin surface. The specimen retains the architecture of the part of lesion that is shaved off. Shave biopsy typically contains only tumour tissue rather than the interface between BCC and normal tissue, which provides important information on the depth and pattern of tumour invasion. Shave biopsy specimens are processed using normal paraffin block histopathology; this technique is only suitable for elevated/protruding BCCs and does not provide the immediate results that cytology can provide (Russell 1999).

Exfoliative cytology may be used for confirming the presence of clinically diagnosed BCC with a view to definitive treatment such as radiotherapy. The cellular appearance of BCC is characteristic (Figure 2), with 'palisade' arrangements of typically basal cells positioned around the margins of densely packed masses of larger and intensely stained cells (Figure 3 and Figure 4) (Ruocco 2011). Cytological features differ for the detection of cSCC, tending to show larger cells with less coherence that are more atypical in appearance with a more varied shape and size (pleomorphic) and abnormal nuclei (Bocking 1987; Fortuno‐Mar 2013; Ruocco 2011). The cytological appearance of melanoma is much more varied, but it can include larger cells than those observed, which are typical of BCC, with prominent and often multiple large nuclei, large nuclear inclusions of cytoplasm, and often a presence of melanin pigment in tumour cells (Bocking 1987; Fortuno‐Mar 2013).

Figure 2

Cytological image of BCC using Papanicoloau stain showing a tissue fragment of BCC on the left and anucleate squamous cells from the epidermis on the right. Copyright © 2017 Derek Roskell: reproduced with permission.

Figure 3

Cytological image of a BCC using Giemsa stain. Focally the nuclei are aligned perpendicular to the basement edge of the cluster (peripheral palisading), a feature characteristic of BCC. Copyright © 2017 Derek Roskell: reproduced with permission.

Figure 4

Cytological image of a BCC using Giemsa stain. The BCC cells are tightly cohesive in a cluster with a distinct edge to the group. Copyright © 2017 Derek Roskell: reproduced with permission.

Clinical pathway

The diagnosis of melanoma can take place in primary, secondary, and tertiary care settings by both generalist and specialist healthcare providers. In the UK, people with concerns about a new or changing lesion will usually present first to their general practitioner (GP) or, less commonly, directly to a specialist in secondary care, which could include a dermatologist, plastic surgeon, other specialist surgeon (such as an ear, nose, and throat (ENT) specialist or maxillofacial surgeon), or ophthalmologist (Figure 5). Current UK guidelines recommend that GPs should assess all suspicious pigmented lesions presenting in primary care by taking a clinical history and visually inspecting them using the revised seven‐point checklist (MacKie 1990). Clinicians should refer those with suspected melanoma or cSCC for appropriate specialist assessment within two weeks (Chao 2013; London Cancer Alliance 2013; Marsden 2010; NICE 2015a). In the UK, low‐risk BCCs are usually recommended for routine referral, with urgent referral for those in whom a delay could have a significant impact on clinical outcomes, for example due to large lesion size or critical site (NICE 2015b). Appropriately qualified generalist care providers increasingly undertake management of low‐risk BCCs in the UK, for example by excising low‐risk lesions (NICE 2010). Similar guidance is in place in Australia (CCAAC Network 2008).

Figure 5

Current clinical pathway for people with skin lesions.

For referred lesions, the specialist clinician will use history‐taking, visual inspection of the lesion (in conjunction with other skin lesions), and often dermoscopy to inform a clinical decision. If melanoma or cSCC is suspected, then urgent excision is advisable. Equivocal lesions for which a definitive diagnosis cannot be reached may undergo surveillance to identify any lesion changes that would indicate biopsy or reassurance and discharge for those that remain stable over a period of time. Low‐risk BCC and pre‐malignant skin lesions potentially eligible for non‐surgical treatment may undergo a diagnostic biopsy before initiating therapy.

Prior test(s)

The diagnosis of skin cancer is based on history‐taking and clinical examination. In the UK, this is typically undertaken at two decision points – first in the GP surgery, where a decision is made to refer or not to refer, and then a second time where a dermatologist or other secondary care clinician makes a decision whether or not to biopsy or excise. A range of technologies have emerged to aid diagnosis to reduce the number of diagnostic biopsies or inappropriate surgical procedures. Dermoscopy using a hand‐held microscope has become the most widely used tool for clinicians to improve diagnostic accuracy of pigmented lesions, in particular melanoma, following visual inspection (Argenziano 1998; Argenziano 2012; Haenssle 2010; Kittler 2002), although it is less well established for the diagnosis of BCC or cSCC (Dinnes 2018a). A further three reviews in this series have evaluated the diagnostic accuracy, and comparative accuracy, of visual inspection and dermoscopy (Dinnes 2018a, Dinnes 2018b, Dinnes 2018c).

Visual inspection of the skin is iterative, using both implicit pattern recognition (non‐analytical reasoning) and more explicit 'rules' based on conscious analytical reasoning (Norman 2009), the balance of which will vary according to experience and familiarity with the diagnostic question. Authors have made various attempts to formalise the mental rules involved in analytical pattern recognition, ranging from a setting out of lesion characteristics that should be considered to formal scoring systems or algorithms with explicit numerical thresholds of skin cancer (Friedman 1985; Sober 1979).

Role of index test(s)

For the diagnosis of BCC, the potential role of exfoliative cytology could be to confirm a strong clinical suspicion of malignancy. If shown to be sufficiently accurate, this simple procedure could avoid the need for an invasive diagnostic skin biopsy in patients whose lesions might be more amenable to non‐surgical treatment. In ulcerated lesions (such as BCC), removing the overlying dead cells or dried exudate is straightforward, and the procedure is therefore potentially less invasive than shave or punch biopsy (though more invasive than dermatoscopic examination). Thus, exfoliative cytology could replace histology or allow treatment to be initiated prior to biopsy results in some patients. The test might also be of value to confirm a clinical suspicion of cSCC in recurrent lesions, or those that are critically located around the eyes, nose, lips, ears and neck, since these are suitable sites for Mohs micrographic surgery. The potential role for exfoliative cytology to detect melanoma is less clear, given the optimal treatment in these patients is excision (Murali 2009). Melanomas are frequently solid skin lesions for which scraping is likely to be more invasive, as removal of the dead layer alone is difficult to achieve. In these cases, histological biopsy is likely to be equally traumatic and but may provide more thorough and reliable diagnostic information.

Although skin is the largest and most accessible organ in the body, cutaneous cytology is not standard practice when diagnosing skin cancer lesions (NICE 2015a; SIGN 2014; Stratigos 2015; Telfer 2008). Although clinicians occasionally use cytology in practice to confirm a clinical diagnosis of BCC when planning radiotherapy or surgery, the nature of the sample obtained lacks the additional histological information, such as pathological subtype and interaction with surrounding skin and structures, that clinicians need to decide on best treatment and which is readily available following biopsy of suspicious lesions (Barr 1984; Ruocco 2011). Nonetheless, the simplicity, immediacy and non‐invasive nature of exfoliative cytology are clearly desirable attributes, which could benefit both health services and patients, albeit in a limited number of circumstances. This is true for confirming a clinical diagnosis of BCC which can present as multiple lesions, and commonly occur on the face, head and neck, which are cosmetically critical sites (Powell 2000). Once diagnosed, superficial BCC can be treated using non‐invasive treatments (listed in Target condition being diagnosed). Excisional surgery and Mohs micrographic surgery are the most successful treatments for nodular BCC, although smaller nodular BCCs in low risk areas can also be treated with topical treatments (Williams 2017); therefore, the ability to confirm a diagnosis in these patients using a fast and non‐invasive approach is attractive (Ruocco 2011). The test can take place during a consultation, with negative results in the presence of clinical concerns for malignancy indicating the need to proceed to a definitive biopsy (Ozden 2013). However, such potential benefits may be outweighed by mistaking more aggressive forms of BCC for a low‐risk BCC, and cytology will never be able to match the additional pathological information regarding cellular behaviour and interaction with surrounding tissues provided by routine histopathology.

In order for exfoliative cytology to realise its potential in low‐risk BCC, it would need to have a high positive predictive value (from a high specificity) to be sure that patients receiving positive results could safely proceed to treatment without biopsy. Any patients with negative cytology findings would still require biopsy to be sure that cytology did not miss another malignancy. A delay in the diagnosis of a BCC as a result of a false‐negative test is usually not as serious as for melanoma because BCC is typically slow‐growing and very unlikely to metastasise. However, delayed diagnosis can result in a larger and more complex excision. Very sensitive tests for BCC, however, are likely to compromise on specificity, leading to a higher false‐positive rate and an enormous burden of skin surgery. Thus, a balance between sensitivity and specificity is needed. The situation for cSCC is more similar to melanoma in that the consequences of falsely reassuring a person that they do not have skin cancer can be serious and potentially fatal. Thus, a good diagnostic test for cSCC should demonstrate high sensitivity and a corresponding high negative predictive value. A test that can reduce false positive diagnoses without missing true cases of disease has patient and resource benefits. False‐positive diagnoses not only cause unnecessary morbidity from the biopsy but could lead to initiation of inappropriate therapies and also increase patient anxiety. Notwithstanding these advantages, cytology does not allow the diagnostician to observe the tumour's histologic growth pattern, a characteristic that can influence management decisions since more aggressive growth patterns require more aggressive treatment (Oram 1997). For melanoma, high test sensitivity is a key requirement, as the cost of missing an early, thin curable lesion can make the difference between life and death.

Alternative test(s)

Standard practice for suspected skin cancer diagnosis in specialist settings involves visual and dermatoscopic examination by a dermatologist, and this review therefore considers these tests to be the comparators. In suspicious lesions these tests are followed by histopathologic analysis of biopsy or excision specimens. This review uses histopathology as the reference standard for definitive diagnosis, and does not review it as an index test. We have also omitted alternative methods of exfoliative cytology, in particular imprint or 'touch imprint' methods, which involve pressing cytology slides directly onto the surface of suspicious lesions (Christensen 2008).

Our series of Cochrane DTA reviews on the diagnosis of skin cancer also reviews a number of other tests, including visual inspection and dermoscopy, teledermatology, mobile phone applications, reflectance confocal microscopy (RCM), optical coherence tomography (OCT) and computer‐assisted diagnosis techniques applied to dermoscopic and other types of image (Chuchu 2018a; Chuchu 2018b; Dinnes 2018a; Dinnes 2018b; Dinnes 2018c; Dinnes 2018d; Dinnes 2018e; Dinnes 2018f; Ferrante di Ruffano 2018a; Ferrante di Ruffano 2018b). RCM and OCT both provide depth‐resolved optical reflectance imaging and are emerging as non‐invasive adjuncts to dermoscopy in a specialist setting, and RCM potentially as an alternative to dermoscopy for skin cancer diagnosis (Edwards 2016). Relative to exfoliative cytology, both methods are resource intensive, and they require specialist training. High‐frequency ultrasound may prove to be an additional tool to assist in the diagnosis of melanoma; however, evidence to date is scarce and generally of poor quality (Dinnes 2018a).

Computer‐assisted diagnosis or artificial intelligence‐based techniques use predefined algorithms to process and manipulate acquired data to identify the features that discriminate malignant from benign lesions, and they may be applied to any types of image or spectra (e.g. Wallace 2000; Wallace 2000a). They have most commonly been applied to digital dermoscopy images (Esteva 2017; Rajpara 2009), with further developments in diffuse reflectance spectroscopy such as SIAscopy (Moncrieff 2002; Walter 2012), MelaFind (Hauschild 2014; Monheit 2011; Wells 2012), and electrical impedance spectroscopy, e.g. the Nevisense system (Malvehy 2014).

Evidence permitting, the accuracy of available tests will be compared in an overview of reviews, exploiting within‐study comparisons of tests and allowing the analysis and comparison of commonly used diagnostic strategies where tests may be used alone or in combination.

Rationale

This review is part of a series of reviews of diagnostic tests used to assist clinical diagnosis that aims to identify the most accurate approaches to diagnosis and provide clinical and policy decision‐makers with the highest possible standard of evidence on which to base diagnostic and treatment decisions. With increasing rates of skin cancer and the push towards the use of dermoscopy and other high‐resolution image analysis in primary care, the anxiety around missing early cases needs to be balanced against sending too many people with benign lesions for a specialist opinion. Although its role for the diagnosis of melanoma is unconvincing because of the loss of vital additional histological information needed for optimal treatment, exfoliative cytology has the potential to improve the health of BCC patients through less invasive and more accessible diagnosis that avoids an additional visit for a skin biopsy result. These benefits must be weighed carefully against the potential limitations of exfoliative cytology to detect the additional pathological features seen on histological examination that help to identify lesions requiring immediate attention. For the subgroup of patients who will go on to receive non‐invasive treatments, the technique could also enable quicker treatment with the potential for better cosmetic results – key objectives from patient groups (NICE 2010) – whilst saving health services the costs of unnecessary biopsies. Treatment of BCCs currently requires diagnostic confirmation using histopathology (NICE 2010), so it is important to assess whether these potential benefits could be attained by comparing the accuracy of exfoliative cytology against that of the reference standard, histological diagnosis.

Since assessing the appearance of a cytological smear is essentially a subjective one that depends on adequate material, the diagnostic performance of exfoliative cytology is likely to be influenced by the experience and training of the individual collecting the sample, as well as the diagnostician. Reproducibility is a known issue in other areas of cytopathology, for example cervical cytology, where the ability to make a diagnosis is influenced by the technician's proficiency in retrieving a sufficient cell sample from scraping (Baena 2017). Evidence arising from diagnosticians' experience with other tests involving the analysis of visual images, such as histopathology, often show variation in diagnosis (Farmer 1996; Shoo 2010), as well as in the availability of clinical data used at the time of diagnosis (Ferrara 2009). This review will therefore also aim to evaluate the impact of clinician experience and training on the adequate retrieval of cell material for cytopathological analysis, as well as on the accuracy of diagnosis.

We identified a single meta‐analysis published in 2004 which considered the accuracy of exfoliative cytology for differentiating between BCC and other conditions (Bakis 2004). Synthesising eight studies, it incorporated three studies not eligible for our review including those conducted on eyelid lesions and evaluating imprint techniques. It also found no studies evaluating the effect of clinician experience. Given that it only included studies published up to 2000, there is a need for an up‐to‐date analysis of the accuracy of exfoliative cytology for the diagnosis of BCCs as well as cSCCs and melanoma skin cancer.

This review follows generic protocols which cover the full series of Cochrane DTA Reviews for the diagnosis of melanoma (Dinnes 2015a), and for diagnosis of keratinocyte skin cancers (Dinnes 2015b). The Background and Methods sections of this review therefore use some text that was originally published in those protocols, along with text that overlaps some of our other reviews (Dinnes 2018a; Dinnes 2018b; Dinnes 2018d; Ferrante di Ruffano 2018a).

Objectives

To determine the diagnostic accuracy of exfoliative cytology for the detection of basal cell carcinoma in adults, and to compare its accuracy with that of current standard diagnostic practice (visual inspection with or without dermoscopy).

Secondary objectives

To determine the diagnostic accuracy of exfoliative cytology for the detection of cutaneous squamous cell carcinoma, and to compare its accuracy with that of standard diagnostic practice (visual inspection with or without dermoscopy).

To determine the diagnostic accuracy of exfoliative cytology for the detection of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants, and to compare its accuracy with that of standard diagnostic practice (visual inspection with or without dermoscopy).

For each of the target conditions, we aimed:

to compare the accuracy of exfoliative cytology versus dermoscopy in direct test comparisons (where the same studies evaluated both tests);
to determine the effect of observer experience.

Investigation of sources of heterogeneity

We set out to address a range of potential sources of heterogeneity for investigation across our series of reviews, as outlined in Dinnes 2015a and Dinnes 2015b and described in Appendix 4. Our ability to investigate these and other sources of heterogeneity was necessarily limited by the available data on each individual test reviewed.

Methods

Criteria for considering studies for this review

Types of studies

We included test accuracy studies that allow comparison of the result of the index test with that of a reference standard, including the following:

studies where all participants receive a single index test and a reference standard;
studies where all participants receive more than one index test and reference standard;
studies where participants are allocated (by any method) to receive different index tests or combinations of index tests, and all receive a reference standard (between‐person comparative studies (BPC));
studies that recruit series of participants unselected by true disease status (referred to as case series for the purposes of this review);
diagnostic case‐control studies that separately recruit diseased and non‐diseased groups (see Rutjes 2005); however, we did not include studies that compared results for malignant lesions to those for healthy skin (i.e. with no lesion present); and
both prospective and retrospective studies.

We excluded studies from which we could not extract 2 × 2 contingency data or if they included fewer than five disease‐positive (for each of BCC, cSCC or melanoma) or disease‐negative (i.e. benign) cases. The size threshold of five is arbitrary. However such small studies are unlikely to add precision to estimate of accuracy.

Studies available only as conference abstracts were excluded; however, attempts were made to identify full papers for potentially relevant conference abstracts (Searching other resources).

Participants

We included studies in adults with lesions suspicious for BCC, cSCC or melanoma. We excluded studies that recruited only participants with malignant diagnoses. We excluded studies conducted in children or where authors clearly reported that more than 50% of participants were aged 16 years old and under.

Index tests

We included studies evaluating exfoliative cytology alone, or exfoliative cytology versus visual inspection and/or dermoscopy. All techniques involving scraping of skin lesions in vivo and subsequent cytological analysis of material were eligible. We excluded swabbed lesions, tape stripping, use of ex vivo specimens, imprint cytodiagnosis and fine needle aspiration.

We also excluded studies evaluating the accuracy of subjective assessment of the presence or absence of individual cytomorphological features (with no overall diagnosis of malignancy) as well as those using the test in intraoperative settings, such as for margin control during excision.

We made no exclusions according to the test observer.

Target conditions

The target condition was basal cell carcinoma (all types).

This decision reflected our assessment that the clearest role of exfoliative cytology would be to replace histological confirmation of disease (see Role of index test(s) and Rationale sections above).

In secondary analyses, we considered three additional definitions of the target condition.

Cutaneous squamous cell carcinoma.
Any form of invasive cutaneous melanoma or atypical melanocytic intraepidermal variants (i.e. including melanoma in situ, or lentigo maligna, which have a risk of progression to invasive melanoma).
Any skin cancer.

Reference standards

The ideal reference standard was histopathological diagnosis of the excised lesion or biopsy sample in all eligible lesions. All biopsy methods were eligible. A qualified pathologist or dermatopathologist should perform histopathology. Ideally, reporting should be standardised, detailing a minimum dataset to include the histopathological features of BCC, cSCC or melanoma to determine the AJCC staging system (e.g. Slater 2014a; Slater 2014b; Slater 2014c). We did not apply the reporting standard as a necessary inclusion criterion but extracted any pertinent information.

We also accepted clinical follow‐up of benign‐appearing lesions as an eligible reference standard, whilst recognising the risk of differential verification bias (as misclassification rates of histopathology and follow‐up will differ) in our quality assessment of studies. 'Expert diagnosis' of benign lesions with no histology or clinical follow‐up was also acceptable as long as at least 50% of all participants with benign lesions had a histological diagnosis. We required all study participants with a final diagnosis of malignancy to have a histological diagnosis, either subsequent to the application of the index test or after a period of clinical follow‐up.

Search methods for identification of studies

Electronic searches

The Information Specialist (SB) carried out a comprehensive search for published and unpublished studies. A single large literature search was conducted to cover all topics in the programme grant (see Appendix 1 for a summary of reviews included in the programme grant). This allowed for the screening of search results for potentially relevant papers for all reviews at the same time. A search combining disease related terms with terms related to the test names, using both text words and subject headings was formulated. The search strategy was designed to capture studies evaluating tests for the diagnosis or staging of skin cancer. As the majority of records were related to the searches for tests for staging of disease, a filter using terms related to cancer staging and to accuracy indices was applied to the staging test search, to try to eliminate irrelevant studies, for example, those using imaging tests to assess treatment effectiveness. A sample of 300 records that would be missed by applying this filter was screened and the filter adjusted to include potentially relevant studies. When piloted on MEDLINE, inclusion of the filter for the staging tests reduced the overall numbers by around 6000. The final search strategy, incorporating the filter, was subsequently applied to all bibliographic databases as listed below (Appendix 5). The final search result was cross‐checked against the list of studies included in five systematic reviews; our search identified all but one of the studies, and this study was not indexed on MEDLINE. The Information Specialist devised the search strategy, with input from the Information Specialist from Cochrane Skin. No additional limits were used.

We searched the following bibliographic databases to 29 August 2016 for relevant published studies:

MEDLINE via OVID (from 1946);
MEDLINE In‐Process & Other Non‐Indexed Citations via OVID; and
Embase via OVID (from 1980).

We searched the following bibliographic databases to 30 August 2016 for relevant published studies:

the Cochrane Central Register of Controlled Trials (CENTRAL; 2016, Issue 7) in the Cochrane Library;
the Cochrane Database of Systematic Reviews (CDSR; 2016, Issue 8) in the Cochrane Library;
Cochrane Database of Abstracts of Reviews of Effects (DARE; 2015, Issue 2);
CRD HTA (Health Technology Assessment) database, 2016, Issue 3;
CINAHL (Cumulative Index to Nursing and Allied Health Literature via EBSCO from 1960).

We searched the following databases for relevant unpublished studies using a strategy based on the MEDLINE search:

CPCI (Conference Proceedings Citation Index), via Web of Science™ (from 1990; searched 28 August 2016); and
SCI Science Citation Index Expanded™ via Web of Science™ (from 1900, using the 'Proceedings and Meetings Abstracts' Limit function; searched 29 August 2016).

We searched the following trials registers using the search terms 'melanoma', 'squamous cell', 'basal cell' and 'skin cancer' combined with 'diagnosis':

Zetoc (from 1993; searched 28 August 2016).
The US National Institutes of Health Ongoing Trials Register (www.clinicaltrials.gov); searched 29 August 2016.
NIHR Clinical Research Network Portfolio Database (www.nihr.ac.uk/research‐and‐impact/nihr‐clinical‐research‐network‐portfolio/); searched 29 August 2016.
The World Health Organization International Clinical Trials Registry Platform (apps.who.int/trialsearch/); searched 29 August 2016.

We aimed to identify all relevant studies regardless of language or publication status (published, unpublished, in press, or in progress). We applied no date limits.

Searching other resources

We had not identified any potentially ongoing studies at the time of publication. We screened relevant systematic reviews identified by our searches for their included primary studies and included any missed by our searches. We checked the reference lists of all included papers, and subject experts within the author team have reviewed the final list of included studies. We did not conduct any citation searching.

Data collection and analysis

Selection of studies

At least one author (JDi or NC or both) screened titles and abstracts, discussing and resolving any queries by consensus. A pilot screen of 539 MEDLINE references showed good agreement (89% with a kappa of 0.77) between screeners. We included primary test accuracy studies and test accuracy reviews (for scanning of reference lists) of any test used to investigate suspected melanoma, BCC, or cSCC at initial screening. Both a clinical reviewer (from one of a team of 12 clinician reviewers) and a methodologist reviewer (JDi or NC) independently applied inclusion criteria (Appendix 6) to all full‐text articles, resolving disagreements by consensus or in consultation with a third party (JDe, CD, HW or RM). We contacted authors of eligible studies when studies did not present enough data to allow for the construction of 2 × 2 contingency tables.

Data extraction and management

One clinical (as detailed above) and one methodologist reviewer (JDi, NC or LFR) independently extracted data concerning details of the study design, participants, index test(s) or test combinations and criteria for index test positivity, reference standards, and data required to populate a 2 × 2 diagnostic contingency table for each index test using a piloted data extraction form. Diagnostic thresholds were all qualitative, with cytopathology criteria used to indicate the presence or absence of the target condition. Some studies used a third diagnostic category for 'possible disease', extracting two datasets for these studies: one grouping 'possible' cases with index test positives (used for the primary analysis), and another grouping 'possible' cases with index test negatives. Disagreements were resolved by consensus or by a third party (JDe, CD, HW or RM).

We contacted authors of conference abstracts published from 2013 to 2015 to ask whether full data were available. If we could not locate a full paper, we marked conference abstracts as 'pending' and will revisit them in a future review update. It was not necessary to contact authors of included studies due to missing information regarding the target condition or diagnostic threshold.

Dealing with multiple publications and companion papers

We did not identify multiple publications for any of our included studies.

Assessment of methodological quality

We assessed risk of bias and applicability of included studies using the QUADAS‐2 checklist (Whiting 2011), tailored to the review topic (see Appendix 7). We piloted the modified QUADAS‐2 tool on a small number of included full‐text articles. One clinical and one methodologist reviewer (JDi, NC or LFR) independently assessed quality for the remaining studies, resolving any disagreement by consensus or in consultation with a third party where necessary (JDe, CD, HW or RM).

Statistical analysis and data synthesis

Due to paucity of data and differences in patient populations and thresholds used to define test positivity, we did not undertake meta‐analysis for the diagnosis of melanoma or cSCC. However, we did perform statistical pooling for the diagnosis of BCC.

In these analyses, we considered any other skin cancers (for example melanomas or cSCCs) in the 'disease negative' group that exfoliative cytology incorrectly identified as BCCs to be false positive results. We took this decision because the clinical management of a lesion considered to be a BCC (for example, initiation of Mohs micrographic surgery, destructive techniques or non‐surgical treatments) could be quite different to that for a melanoma or cSCC and could potentially lead to a negative outcome for those concerned. For the diagnosis of melanoma, however, we considered any other skin cancers (BCC, cSCC etc) that were incorrectly identified as melanomas (i.e. positive on exfoliative cytology) to be true negative test results rather than as false positives, on the basis that excision of such lesions may still have been appropriate for the participants concerned.

Our unit of analysis was the lesion rather than the person. This is because in skin cancer initial treatment is directed to the lesion rather than systemically (thus it is important to be able to correctly identify cancerous lesions for each person), and it is also the most common way in which the primary studies reported data. Although there is a theoretical possibility of correlations of test errors when the same people contribute data for multiple lesions, most studies include very few people with multiple lesions, and any potential impact on findings is likely to be very small, particularly in comparison with other concerns regarding risk of bias and applicability. For each analysis, we included only one dataset per study to avoid multiple counting of lesions. We conducted separate analyses according to the definition of the target condition, i.e. detection of BCC, melanoma or cSCC, and detection of any skin lesion requiring excision, as defined under Target condition being diagnosed. We used Review Manager 5 (RevMan 5) for preliminary analyses of the data by plotting estimates of sensitivity and specificity on coupled forest plots and in receiver operating characteristic (ROC) space (RevMan 2014). We used the bivariate model to obtain summary estimates of sensitivity and specificity (Macaskill 2013). We fitted the bivariate models using the meqrlogit command in STATA 15.

We made comparisons with standard diagnostic practice by comparing the accuracy of exfoliative cytology with visual inspection or dermoscopy. We included direct comparisons using data on the accuracy of visual inspection and/or dermoscopy only if reported in the included studies of exfoliative cytology due to the known substantial unexplained heterogeneity in all studies of the accuracy of dermoscopy (Dinnes 2018b). We did not perform comparative meta‐analysis because of the limited number of studies.

We obtained 95% confidence intervals for sensitivity and specificity using the delta method and Wald tests, respectively. When the number of studies was insufficient for meta‐analysis, we examined individual study results and calculated 95% CIs using the Newcombe‐Wilson method without continuity correction (Newcombe 1998).

Investigations of heterogeneity

We examined heterogeneity between studies by visually inspecting forest plots and summary ROC plots. Due to the limited number of studies in each analysis, we were unable to formally assess heterogeneity using meta‐regression.

Sensitivity analyses

We were unable to perform sensitivity analyses due to limited data.

Assessment of reporting bias

Because of uncertainty about the determinants of publication bias for diagnostic accuracy studies and the inadequacy of tests for detecting funnel plot asymmetry (Deeks 2005), we did not test for publication bias.

Results

Results of the search

We identified and screened a total of 34,517 unique references for inclusion. Of these, we reviewed 1051 full‐text papers for eligibility for any one of the suite of DTA reviews of tests for diagnosing melanoma or keratinocyte skin cancer. Figure 6 documents a PRISMA flow diagram of search and eligibility results. We tagged 40 full‐text publications as potentially eligible for this review, ultimately including 9. We excluded 9 studies that were not primary studies, 8 including fewer than five benign lesions, 1 that insufficiently reported test accuracy data, 10 that used an ineligible index test (including swabbing (Bocking 1987), tape‐stripping (Berardi 1992), imprint cytology (Hering 1970; Melek 1970; Urbach 1957), and fine needle aspiration (Jakasa 1976; Korabiec 1977; Rojo 1998; von Gizycki‐Nienhaus 1992; Yu 2005)), 2 using exfoliative cytology in an ineligible context (intraoperative care or margin control), 4 in an ineligible patient population, and 4 using an ineligible reference standard. We excluded three studies for multiple reasons. A list of the 31 studies excluded from this review with reasons for exclusion is provided in Characteristics of excluded studies, with a list of all studies excluded from the full series of reviews available as a separate pdf (please contact skin.cochrane.org for a copy of the pdf).

Figure 6

PRISMA flow diagram.

Across all skin cancer DTA reviews, we contacted the corresponding authors of 86 studies, asking 37 to supply further information to allow study inclusion, 18 to clarify diagnostic thresholds, and 30 to define the target condition. It was not necessary to contact any authors for the current review.

Included studies

We included nine studies evaluating the use of exfoliative cytology in participants with lesions suspected of skin cancer, providing 25 datasets (14 for BCC, 2 for cSCC, 1 for melanoma, and 8 for any malignant condition). One of these also performed a direct comparison between exfoliative cytology and dermoscopy (3 datasets: one each for melanoma, BCC and any malignant condition). A total of 1697 lesions were examined by the nine studies, of which 42 were excluded from analysis due to the absence of exfoliative cytology test results (see 'Test failures' below and Table 1) leaving 1655 lesions for analysis, including 1120 BCCs, 41 cSCCs, and 10 melanomas.

Table 1. Test failures due to insufficient cellular material

Study	Stain technique	Slides with inadequate material n (%)	Histological diagnosis
Gordon 1984	Papanicolaou	9 (6)	BCC: 1 cSCC: 1 Actinic keratosis: 7
Christensen 2008^a	Papanicolaou	1 (1)	Actinic keratosis: 1
Christensen 2008^a	MGG	3 (4)	BCC: 1 Actinic keratosis: 2
Durdu 2011	MGG	15 (8)	Melanocytic benign: 6 Non‐melanocytic benign: 9
Nauth 1988	Papanicolaou	18 (8)	BCC: 1 cSCC: 2 Severe precancerous disease: 2 Mild precancerous disease: 6 Benign tumour: 5 Inflammation: 2

BCC: basal cell carcinoma; cSCC: cutaneous squamous cell carcinoma; MGG: May‐Grünwald Giemsa stain technique.
^aWhen diagnosis was made using both Papanicolaou and MGG stained slides, all lesions could be diagnosed cytologically

Appendix 8 describes the thresholds used for diagnosis across the studies, along with summary study details.

Six studies recruited series of lesions with clinically suspected BCCs that also underwent histological evaluation by excision or biopsy. Two were prospective (Berner 1999; Gordon 1984), two retrospective (Powell 2000; Ruocco 1992), and two unclear (Brown 1979; Derrick 1994). Two case‐control studies, Christensen 2008 and Nauth 1988, selectively included a mix of histologically confirmed lesions, while a single prospective case series, Durdu 2011, was conducted in participants with pigmented skin lesions considered to be difficult to diagnose on clinical grounds. No studies provided further details regarding the degree of investigation prior to receiving exfoliative cytology. Four took place in the UK (Berner 1999; Brown 1979; Derrick 1994; Powell 2000), one in Italy (Ruocco 1992), one in Norway (Christensen 2008), one in Germany (Nauth 1988), one in Australia (Gordon 1984) and one in Turkey (Durdu 2011). None reported being funded by manufacturers of diagnostic technology.

The number of participants ranged from 30 to 240 with a median of 101 (interquartile range (IQR) 73 to 188), but one study did not report this detail (Ruocco 1992). Studies included a median of 150 lesions (range 37 to 578, IQR 83 to 224). In the BCC studies, disease prevalence ranged from 52% in Gordon 1984 to 95% in Derrick 1994 in the 6 case series, and it was pre‐set in the two case‐control studies, at 19% in Nauth 1988 and 64% in Christensen 2008. In the series evaluating pigmented skin lesions, the prevalence of melanoma was 5% and of BCC, 17% (Durdu 2011). This was the only study to include significant numbers of melanocytic benign lesions (Durdu 2011), whilst the remaining eight studies included mainly non‐melanocytic benign lesions including actinic keratoses, seborrhoeic keratoses, Bowen's disease, and keratoacanthoma. Four studies that did not contribute datasets for the analysis of cSCC included small numbers of cSCCs (Berner 1999; Brown 1979; Derrick 1994; Ruocco 1992). Two studies did not report specific benign diagnoses (Berner 1999; Nauth 1988). Appendix 8 lists a full breakdown of differential diagnoses for each study.

Studies used a variety of staining methods. Three employed Papanicolau (Christensen 2008; Gordon 1984; Nauth 1988), and three May‐Grünwald Giemsa (MGG; Christensen 2008; Derrick 1994; Durdu 2011). One study used Diff‐Quick (Berner 1999). Three studies used more than one technique, but two (Brown 1979; Ruocco 1992) failed to report which they had used in particular participants, and a fourth failed to report the stain method (Powell 2000). One study that performed a direct comparison of diagnoses made using Pap, MGG, and both Pap and MGG investigated the impact of varying stain methods (Christensen 2008).

All studies based their index diagnoses on cytomorphological findings, though three failed to outline the diagnostic criteria used (Brown 1979; Christensen 2008; Powell 2000). Features diagnostic for BCC were similar across the remaining studies, except for Nauth 1988, who clearly implemented a different approach by using a classification developed from vaginal cytology (the 'Munchener scheme', a modification of the original Papanicolaou classification) to decide whether a lesion was malignant. For the diagnosis of melanoma, Durdu 2011 provided a basic definition of disease, defining melanoma as the presence of 'epithelioid or spindle‐type atypical nevoid cells'. Durdu 2011 also reported dermoscopic diagnoses for all patients, which followed a two‐step method, differentiating melanocytic from non‐melanocytic lesions before applying the ABCD algorithm. Appendix 8 lists specific diagnostic criteria for each study.

The dermatologist performed skin scrapes in one study (Durdu 2011), but the remaining studies did not describe the operating clinician. Studies described the experience of the clinician performing cytodiagnosis as the cytologist (Gordon 1984), cytopathologist (Berner 1999; Christensen 2008), pathologist (Derrick 1994), or dermatologist (Durdu 2011), but four studies did not report this (Brown 1979; Nauth 1988; Powell 2000; Ruocco 1992). No study evaluated interobserver variability.

In eight studies the reference standard diagnosis was by histology alone, while Brown 1979 used expert opinion to overrule the histological diagnosis in two lesions whose clinical and cytological appearance was 'characteristic' of BCC.

Test failures

Four studies reported instances of insufficient cellular material to make a cytological diagnosis (Christensen 2008; Durdu 2011; Gordon 1984; Nauth 1988), listed in Table 1. Comprising between 1% and 8% of slides evaluated in each study, these were considered as test failures and excluded from analysis of accuracy. One study excluded inadequate slides at study entry (Berner 1999), and the remaining four studies did not report the adequacy of cellular material, suggesting this may have been an implicit eligibility criterion (Brown 1979; Derrick 1994; Powell 2000; Ruocco 1992).

Methodological quality of included studies

Overall study quality was low or unclear, particularly in terms of the clinical applicability of results (Figure 7 and Figure 8).

Figure 7

Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study. One study, Durdu 2011, was assessed in the comparative domain as 'unclear' for both risk of bias and applicability concerns.

Figure 8

Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies. One study, Durdu 2011, was assessed in the comparative domain as 'unclear' for both risk of bias and applicability concerns.

Three of the nine studies were at low risk of bias for participant selection (Gordon 1984; Powell 2000; Ruocco 1992); three were at high risk of bias: two because they recruited non‐consecutively and selected participants according to histological diagnosis (Christensen 2008; Nauth 1988), and one because it excluded lesions inappropriately (Derrick 1994). Three did not clearly describe consecutive patient recruitment or exclusions. Concern was high for the applicability of setting and included participants in six studies: due to poor reporting regarding the composition of study populations in five (Christensen 2008; Gordon 1984; Nauth 1988; Powell 2000; Ruocco 1992), inclusion of narrowly defined study groups in two (Berner 1999; Derrick 1994), and inclusion of multiple lesions per patient in five (Berner 1999; Christensen 2008; Durdu 2011; Gordon 1984; Powell 2000). We could not determine the clinical applicability of participant populations in two studies due to insufficient reporting of study populations (Nauth 1988; Ruocco 1992).

Risk of bias for the index test was low in two studies (Berner 1999; Gordon 1984), but we could not determine this in the remaining seven studies due to poor reporting of diagnostic thresholds and whether cytology slides were interpreted without knowledge of the lesion's histology results. More than half of the studies (5/9) caused high concern regarding the applicability of the index test, since examiners did not did have access to the clinical diagnosis during review of cytology slides (Christensen 2008; Gordon 1984), and they did not report cytodiagnostic criteria in sufficient detail to allow replication (Brown 1979; Christensen 2008; Gordon 1984; Powell 2000; Ruocco 1992); we could not assess two studies due to poor reporting of the diagnostician's cytological expertise (Durdu 2011; Nauth 1988). The remaining two studies were of low concern (Berner 1999; Derrick 1994).

All studies reported the use of an acceptable reference standard with one exception: Nauth 1988 failed to state the reference standard used to confirm the absence of disease in 14 of 224 included diseased participants (Nauth 1988). Only two studies clearly blinded the reference standard diagnosis to the cytology results (Berner 1999; Brown 1979), while in the remaining seven studies a failure to clearly report this aspect meant that the risk of bias due to conduct of the reference standard was unclear. We were also unclear as to whether most studies (7/9) used the reference standard in a clinically applicable way, largely due to inadequate description of the conduct and interpretation of histology; only one study reported histopathological interpretation by an experienced dermatopathologist (Derrick 1994). We judged Brown 1979 to be of high concern due to use of expert opinion (discipline and qualifications not reported) to overrule the reference standard diagnosis in 2 of 85 cases.

We judged only one study, Durdu 2011, to be at low risk of bias for the flow and timing domain, while the rest were at high and/or unclear risk. Brown 1979 and Nauth 1988 used different reference standard tests, and Christensen 2008 excluded slides 'unavailable for examination'; these aspects conferred a high risk of bias. Seven studies were unclear in that they failed to report the time interval between exfoliative cytology and histology examinations (Berner 1999; Christensen 2008; Derrick 1994; Gordon 1984; Nauth 1988; Powell 2000; Ruocco 1992).

The single study comparing exfoliative cytology with dermoscopy, Durdu 2011, reported blinding the diagnoses of the two index tests; however, authors did not describe the time interval between tests or give sufficient details on their conduct, thus its risk of bias and applicability in the comparative domain remain unclear.

Findings

Detection of BCC

Seven of the nine studies provided data eligible for pooling. We did not pool the remaining two studies in the meta‐analysis because Nauth 1988 used a different diagnostic classification system (the Munchener scheme), and Durdu 2011 evaluated exfoliative cytology in a distinct patient group (pigmented skin lesions).

The seven pooled studies were in participants with clinically suspect BCC lesions, and they used standard cytomorphology to investigate 1264 lesions, 1045 of which were BCCs, using MGG stain (Derrick 1994), Pap stain (Gordon 1984), Diff‐Quick (Berner 1999), or a mixture of stain techniques (Brown 1979; Ruocco 1992); Powell 2000 did not report the stain method. Christensen 2008 used two slides per lesion: one MGG and the other Pap stain, selecting the slide showing the greatest degree of cytological atypia for the final diagnosis. These were pooled regardless of stain method used, giving a summary sensitivity of 97.5% (95% CI 94.5% to 98.9%) with a summary specificity of 90.1% (95% CI 81.1% to 95.1%). Table 2 provides a summary of all results.

Table 2. Summary of main results

Analysis	Target condition Test	No. studies	Lesions with cytology results (n)	Diseased lesions (n)	Sensitivity (95% CI)	Specificity (95% CI)
Detection of basal cell carcinoma (BCC)
All studies	Studies with cases of BCC	9	1655	1120	—	—
Pooled studies	Standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC ('possible BCC' cases classified as BCC test positive)	7	1264	1045^a	97.5 (94.5 to 98.9)	90.1 (81.1 to 95.1)
—	Standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC ('possible BCC' cases classified as BCC test negative)	7	1264	1045^a	97.3 (93.5 to 98.9)	94.2 (88.7 to 97.1)
Studies not pooled		2	391	75	—	—
—	Nauth 1988: different diagnostic criteria ‐ Munchener scheme (class V = malignant)	1	206	41^b	80.5 (66.0 to 89.8)	74.6 (67.4 to 80.6)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (exfoliative cytology)	1	185^c	34	100 (89.9 to 100)	100 (97.5 to 100)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (dermoscopy)	1	200	34	94.1 (80.9 to 98.4)	98.2 (94.8 to 99.4)
Detection of cutaneous squamous cell carcinoma (cSCC)
All studies	Studies with cases of cSCC	6	1357	52	—	—
Studies not pooled		2	401	41^d	—	—
—	Gordon 1984: standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC	1	141	5^e	100 (56.6 to 100)	98.5 (94.8 to 99.6)
—	Nauth 1988: different diagnostic criteria ‐ Munchener scheme (class V = malignant)	1	206	36^f	88.9 (74.7 to 95.6)	74.7 (67.7 to 80.6)
Studies not included in dataset	< 5 cSCC cases	4	1010	11	—	—
Detection of invasive melanoma and atypical intraepidermal melanocytic variants (MM)
All studies	Studies with cases of MM	2	270	11	—	—
Studies not pooled		1	185^c	10	—	—
—	Durdu 2011: different patient group ‐ pigmented skin lesions (exfoliative cytology)	1	185^c	10	100 (72.3 to 100)	100 (97.6 to 100)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (dermoscopy)	1	200	10	80.0 (49.0 to 94.3)	97.4 (94.0 to 98.9)
Studies not included in dataset	< 5 MM cases	1	85	1	—	—
Detection of any potential skin cancer (BCC or other skin cancer)
All studies	Studies with any skin cancer lesions	9	1655	1200	—	—
Pooled studies	Standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC ('possible BCC' cases classified as BCC test positive)	4	573	495	97.3 (93.5 to 98.9)	86.0 (73.5 to 93.1)
—	Standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC ('possible BCC' cases classified as BCC test negative)	4	573	495	96.6 (90.3 to 98.9)	94.7 (80.2 to 98.7)
Studies not pooled		2	391	123	—	—
—	Nauth 1988: different diagnostic criteria ‐ Munchener scheme (class V = malignant)	1	206	77^g	84.4 (74.7 to 90.9)	92.3 (86.3 to 95.7)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (exfoliative cytology)	1	185^c	46	100 (92.3 to 100)	100 (97.3 to 100)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (dermoscopy)	1	200	46	97.8 (88.7 to 99.6)	98.1 (94.4 to 99.3)
Studies not included in dataset	No skin cancer other than BCC (Christensen 2008; Powell 2000)	2	113	72	—	—
—	Data not reported (Ruocco 1992)	1	578	507	—	—

BCC: basal cell carcinoma; CI: confidence interval; cSCC: cutaneous squamous cell carcinoma; MM: invasive melanoma and atypical intraepidermal melanocytic variants.
^aTwo additional BCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material.
^b1/42 BCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material.
^cFrom a total population of 200 lesions (15 excluded from exfoliative cytology analysis due to insufficient cell material, all 200 examined by dermoscopy).
^d3/44 cSCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material.
^e1/6 cSCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material.
^f2/38 cSCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material.
^g3/80 lesions could not be analysed by exfoliative cytology, due to insufficient cell material.

Common diagnoses mistaken for BCC were actinic keratosis in Christensen 2008 and Gordon 1984 and trichoepithelioma in Derrick 1994 and Ruocco 1992. Only 3 of the 22 false positive cases (listed in Table 3) were malignant lesions, and all 3 were confirmed carcinomas, but it was not possible to classify histological type due to insufficient biopsy material (Berner 1999). No false positive cases were melanomas. Six of the seven false positive cases in Gordon 1984 were 'possible but not diagnostic for BCC' lesions, histologically diagnosed as marked atypia (n = 4) and seborrhoeic keratosis (n = 2). Consideration of these uncertain diagnoses as test negatives did not impact on pooled sensitivity (97.3%, 95% CI 93.5% to 98.9%) but raised the specificity estimate to 94.2% (95% CI 88.7% to 97.1%; Figure 9 Figure 10). All 16 cSCCs were correctly identified as true negative cases (Berner 1999; Brown 1979; Derrick 1994; Gordon 1984; Ruocco 1992); however, two studies misdiagnosed three BCCs as cSCCs (Brown 1979; Gordon 1984).

Figure 9

Forest plot of exfoliative cytology to detect BCC in patients with suspected BCCs, showing classification of 'possible BCCs' as test positives or as test negatives.

Figure 10

Summary ROC plot of exfoliative cytology to detect BCC in patients with suspected BCCs, showing classification of 'possible BCCs' as test positives or as test negatives.

Table 3. Exfoliative cytology for the detection of BCC: false positive diagnoses

Study	False positive n (%)	Histological diagnosis
Berner 1999	5 (4.6)	Carcinoma, type not specified: 3 atypia: 2
Brown 1979	0 (0)	—
Christensen 2008^a	1 (1.3)	Actinic keratosis: 1
Derrick 1994	1 (0.4)	Trichoepithelioma: 2
Durdu 2011	0 (0)	—
Gordon 1984	7 (5.0)	Actinic keratosis: 1 Marked atypia: 4 Sebhorroeic keratosis: 2
Nauth 1988	18 (8)	BCC: 1 cSCC: 2 Severe precancerous disease: 2 Mild precancerous disease: 6 Benign tumour: 5 Inflammation: 2
Powell 2000	2 (5.4)	Bowenoid actinic keratosis: 1 Bowen's disease: 1
Ruocco 1992	6 (1)	Trichoepithelioma: 3 Syringocystadenoma papilliferum: 2 Pilomatricoma: 1

BCC: basal cell carcinoma; CI: confidence interval; cSCC: cutaneous squamous cell carcinoma.
^aDiagnosis made using both Papanicolaou and May‐Grünwald Giemsa stained slides

The study using the Munchener scheme, Nauth 1988, identified fewer BCCs and incorrectly diagnosed 42 lesions as being BCC, giving a sensitivity of 80.5% (95% CI 66.0 to 89.8%) and specificity of 74.6% (95% CI 67.4% to 80.6%). Authors did not report misdiagnosis by lesion type.

The study in pigmented skin lesions (MGG stain) reported no false diagnoses amongst slides for 185 lesions, giving a sensitivity of 100% (95% CI 89.9% to 100%) and specificity of 100% (95% CI 97.5% to 100%) for the diagnosis of BCC (Durdu 2011); however, 15 lesions were excluded from analysis due to the retrieval of insufficient cell material. Results for dermoscopy, conducted on the full sample of 200 lesions, demonstrated a lower sensitivity of 94.1% (95% CI 80.9% to 98.4%) and higher specificity 98.2% (95% CI 94.8% to 99.4%), but the differences could be explained by chance.

Christensen 2008 found no difference in sensitivity or specificity between the three stain techniques (Pap versus MGG versus Pap + MGG), with each method identifying the same number of false positive (n = 1) and false negative (n = 2) cases to give a sensitivity of 96% and specificity of 96%.

Detection of cSCC

Six studies examined cSCC lesions, although those from four studies (totalling 11 lesions) were excluded from analysis due to each study having an inadequate number of cSCC cases (fewer than 5 per study). The two remaining studies contributed 41 analysed cSCC lesions amongst their 347 lesions, however their results were not pooled due to their use of different diagnostic criteria. Using standard cytomorphological criteria to diagnose 5 cSCC slides from 141 lesions, Gordon 1984 report a sensitivity of 100% (95% CI 56.6% to 100%) and specificity of 98.5% (95% CI 94.8% to 99.6%) with two false positive results, both showing squamous differentiation with cellular pleomorphism and a histological diagnosis of pleomorphic BCC. Nauth 1988's use of the Munchener scheme to diagnose 36 cSCC slides from 206 lesions resulted in a lower sensitivity of 88.9% (95% CI 74.7% to 95.6%) and lower specificity of 74.7% (95% CI 67.7% to 80.6%), reporting the only false negative cSCCs of any included study. Two were diagnosed as 'questionable dyskeratoses and/or questionable anaplastic tumour cells', one as mild dysplasia and the fourth as severe dysplasia.

No data were available to compare exfoliative cytology for detection of cSCC with routine diagnostic practice.

Detection of invasive melanoma and atypical intraepidermal melanocytic variants

The single study evaluating exfoliative cytology for the detection of 10 melanomas in 185 lesions, Durdu 2011, reported sensitivity of 100% (95% CI 72.3% to 100%) and specificity of 100% (95% CI 97.6% to 100%); dermoscopic diagnosis in the full sample (200 lesions, 10 melanomas) produced a sensitivity of 80.0% (95% CI 49.0% to 94.3%) and specificity of 97.4% (95% CI 94.0% to 98.9%). One other study included a single case of melanoma as a BCC‐negative case (Brown 1979).

Detection of any skin cancer

Four studies in 573 clinically suspect BCC lesions provided data for detection of any skin cancer (Berner 1999; Brown 1979; Derrick 1994; Gordon 1984); 495 histologically confirmed malignant lesions were included (476 BCCs, 13 cSCCs, 1 melanoma, 4 carcinomas of unspecified histological type (Berner 1999), plus 1 apocrine carcinoma (Derrick 1994)). Pooled sensitivity was estimated to be 97.3% (95% CI 93.5% to 98.9%) with a pooled specificity of 86.0% (95% CI 73.5% to 93.1%). Consideration of uncertain diagnoses as test negatives did not impact on pooled estimates of sensitivity (96.6%, 95% CI 90.3% to 98.9%) or specificity (94.7%, 95% CI 80.2% to 98.7%; Figure 11; Figure 12; Berner 1999; Gordon 1984).

Figure 11

Forest plot of studies pooled for accuracy of exfoliative cytology to detect any skin cancer in patients with suspected BCCs, comparing: classification of 'possible BCCs' as test positives versus classification of 'possible BCCs' as test negatives.

Figure 12

Summary ROC plot of pooled studies for accuracy of exfoliative cytology to detect any skin cancer in patients with suspected BCCs, comparing: classification of 'possible BCCs' as test positives versus classification of 'possible BCCs' as test negatives.

The Munchener scheme, used in Nauth 1988, was less sensitive in its detection of 36 cSCCs and 41 BCCs amongst 206 included cases, with a sensitivity of 84.4% (95% CI 74.7% to 90.9%) and specificity of 92.3% (95% CI 86.3% to 95.7%).

The study in pigmented skin lesions included 10 melanomas, 34 BCCs, 1 case of pigmented mammary Paget's disease, and 1 pigmented metastatic mammary carcinoma (Durdu 2011). In 185 lesions, exfoliative cytology was able to differentiate between these and benign conditions with a sensitivity of 100% (95% CI 92.3% to 100%) and a specificity of 100% (95% CI 97.3% to 100%). Whilst this was marginally more accurate when compared to dermoscopy alone (sensitivity 97.8%, 95% CI 88.7% to 99.6%; specificity 98.1%, 95% CI 94.4% to 99.3%), the difference could be due to chance. Also, dermoscopy was conducted on the full sample of 200 lesions (Durdu 2011).

Effect of observer experience

No included studies evaluated the effect of observer experience on the accuracy of exfoliative cytology in any skin cancer.

Investigations of heterogeneity

We were unable to undertake formal investigations of heterogeneity due to insufficient study numbers.

Discussion

Summary of main results

This review aimed to assess the accuracy of exfoliative cytology for diagnosing BCC, cSCC or melanoma in adults, yet most studies focus on its use for confirming the clinical diagnosis in lesions with a high clinical suspicion of BCC. Studies were poorly reported and of uncertain to poor methodological quality, particularly in terms of the applicability of their results to the current clinical setting in the UK, thus limiting the strength of conclusions that can be drawn. The summary of findings Table presents key results for the primary target condition of BCC.

Pooled results from seven studies with 1264 clinically suspected BCC lesions that included 1045 BCCs provided a sensitivity of 97.5% (95% CI 94.5% to 98.9%) and specificity of 90.1% (95% CI 81.1% to 95.1%). The summary of findings Table translates these estimates to a hypothetical cohort of 1000 lesions clinically suspected of being BCC. At the median BCC prevalence of 86%, exfoliative cytology would miss 21 BCCs and would result in 14 false positive diagnoses. As BCCs are usually relatively slow growing, delayed treatment of 21 out of 860 BCCs may not have serious consequences. However, if the test was used as a basis for initiation of non‐surgical treatment, and any of the false positive results were lesions requiring excision, such as melanomas or cSCCs, the consequences could be potentially fatal. At the lower and upper quartile prevalence of BCC of 63% and 88%, 16 and 22 BCCs would be missed, respectively, with 37 and 12 false positive diagnoses. While evidence for the ability of exfoliative cytology to detect cSCC is scarce (with only one study using a clinically relevant application of exfoliative cytology), it is worth noting that all 17 cSCCs included in the primary analysis were correctly identified using standard cytomorphology, albeit with some difficulty in discriminating BCCs from cSCC correctly in the presence of pleomorphic features. This suggests that in populations with very high clinical suspicion of BCC, and therefore high prevalence of disease, exfoliative cytology could have a potential role in guiding the use of non‐surgical therapy and avoiding biopsy. Decisions to start some non‐surgical treatments in patients with superficial BCC, for example topical imiquimod, are in practice unlikely to require additional confirmation when clinical suspicion is already high, and thus cytodiagnosis is likely to have minimal utility in these cases. Conversely, exfoliative cytology may be most valuable to the management of patients with BCC lesions considered for radiotherapy, since a tissue diagnosis is typically required for confirmation before the therapy can proceed.

The 'perfect' results for the detection of both BCC and melanoma from the single study recruiting only pigmented lesions is likely to be explained by the unique case‐mix of patients (Durdu 2011), with high proportions of benign melanocytic naevi and of benign non‐melanocytic lesions such as seborrhoeic keratosis, warts and dermatofibroma (Durdu 2011). The high rate of uninterpretable benign slides in this study (8%) may also have influenced its specificity. In the absence of additional studies and greater numbers of lesions, these data did not provide sufficient evidence on the performance of exfoliative cytology to detect melanomas, its use in populations with pigmented skin lesions, or its performance using other cytological classification approaches.

Observed limitations of primary studies

Studies were limited by universally poor reporting and poor methodological quality. In addition to scarce reporting of participant selection, studies failed to outline the prior referral pathway of eligible patients, including a description of which clinical methods they had used to arrive at a clinical suspicion of BCC that was strong enough to make the patients eligible for study inclusion. For the purposes of our BCC analysis, we have assumed that all approaches resulted in similar population groups; however, in reality the spectrum of disease in included groups remains unclear and could differ considerably.

Similarly, very limited reporting of the diagnostic criteria used to define the cytomorphologic presence of disease could obscure actual differences in diagnostic thresholds. Along with missing descriptions of how histopathological diagnoses were made and the experience of clinicians performing or interpreting scrapes, these limitations of the primary studies limit the generalisability of our findings to current clinical practice, as well as our understanding of the efficacy of exfoliative cytology to distinguish between skin cancers.

There was a similar lack of clarity in description of most items necessary to determine the risk of bias, including: recruitment methods, study design, threshold selection, blinding of the reference standard to the index test result and the time interval between exfoliative cytology and definitive histology. When these details were reported, studies were often at high risk of bias, so their accuracy estimates may not adequately reflect the true sensitivity and specificity of exfoliative cytology.

Strengths and weaknesses of the review

The strengths of this review include an in‐depth and comprehensive electronic literature search, systematic review methods including double extraction of papers by both clinicians and methodologists, and contact with authors to allow study inclusion or clarify data. We planned a clear analysis structure to allow estimation of test accuracy in discrete study populations using only scrape techniques to gather a cell sample for cytopathology. We undertook a detailed and replicable assessment of methodological quality. This is the only review we are aware of to have examined the accuracy of exfoliative cytology for detecting cSCC, melanoma or any skin cancer.

Published in 2004, Bakis 2004 was an earlier meta‐analysis of exfoliative cytology based on eight studies including 1261 BCCs. Reviewers arrived at very similar pooled estimates (97% sensitivity and 86% specificity), despite including three studies that did not meet our inclusion criteria due to: differing target condition (Barton 1996), ineligible method of exfoliation (Bocking 1987), and insufficient numbers of individuals with benign disease (Vega‐Memije 2000). By comparison, the present review provides an updated estimate of the accuracy of exfoliative cytology to detect BCC using a larger number of studies (three published after 2004), evaluating the same target conditions, and all of which have used scrape techniques to gather a cell sample for cytopathology. Ours has included a non‐English language study, Nauth 1988, which was excluded from the Bakis 2004 review (because the article could not be located), so the present review constitutes a more current and comprehensive summary of the accuracy of exfoliative cytology to detect BCC.

The main concerns for this review are the small number of studies and their poor reporting of patients' prior referral pathways, criteria used to arrive at cytopathological or histopathological diagnoses, observer experience, and other aspects relating to participant selection or methods of performing exfoliative cytology. Some authors have questioned the ability of cytopathology to provide sufficient discrimination of skin cancer subtypes (Barr 1984); however, we did not address this topic in the current review. Thus, echoing the findings of Bakis 2004, the main weakness of this review is the poor reporting of primary studies, which has limited our appraisal of study quality and, critically, impedes our understanding of whether summary estimates are applicable to current clinical settings.

Applicability of findings to the review question

Not all data included in this review are likely to be generally applicable to the current clinical setting. In particular, Durdu 2011 used exfoliative cytology in a clearly different population to that in which the test is likely to be used in clinical practice, whilst Nauth 1988 used a diagnostic classification used for vaginal cytology (the Munchener scheme) to grade the degree of cell dysplasia from normal to anaplastic, an approach which is clearly different from the other seven studies that sought to determine whether a lesion was a BCC, cSCC or melanoma. We pooled the remaining seven studies, and summary accuracy estimates do appear to show that exfoliative cytology confirms clinically suspected BCC with a high sensitivity and specificity; however, poor reporting limits any more detailed statements regarding which patient populations these results would be replicated in. Furthermore, the lack of description in all studies regarding the diagnostic criteria used for both index test and reference standard may restrict applicability and transferability of results in practice.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Current clinical pathway for people with skin lesions.

Figure 6

PRISMA flow diagram.

Figure 7

Figure 8

Figure 9

Forest plot of exfoliative cytology to detect BCC in patients with suspected BCCs, showing classification of 'possible BCCs' as test positives or as test negatives.

Figure 10

Summary ROC plot of exfoliative cytology to detect BCC in patients with suspected BCCs, showing classification of 'possible BCCs' as test positives or as test negatives.

Figure 11

Figure 12

Test 1

Exfoliative cytology ‐ BCC (possible BCCs classified as test positives).

Test 2

Exfoliative cytology ‐ BCC (possible BCCs classified as test negatives).

Test 3

Exfoliative cytology ‐ BCC (pigmented lesion population).

Test 4

Dermoscopy ‐ BCC (pigmented lesion population).

Test 5

Exfoliative cytology ‐ BCC (mixed population, Munchener diagnostic criteria).

Test 6

Exfoliative cytology ‐ cSCC (suspected BCC population).

Test 7

Exfoliative cytology ‐ cSCC (mixed population, Munchener diagnostic criteria).

Test 8

Exfoliative cytology ‐ melanoma (pigmented lesion population).

Test 9

Dermoscopy ‐ melanoma (pigmented lesion population).

Test 10

Exfoliative cytology ‐ any skin cancer (suspected BCC population, possible BCCs classified as test positives).

Test 11

Exfoliative cytology ‐ any skin cancer (suspected BCC population, possible BCCs classified as test negatives).

Test 12

Exfoliative cytology ‐ any skin cancer (pigmented lesion population).

Test 13

Dermoscopy ‐ any skin cancer (pigmented lesion population).

Test 14

Exfoliative cytology ‐ any skin cancer (mixed population, Munchener diagnostic criteria).

Test 15

Exfoliative cytology (Papanicolaou + MGG stain) ‐ BCC (stain comparison).

Test 16

Exfoliative cytology (MGG stain) ‐ BCC (stain comparison).

Test 17

Exfoliative cytology (Papanicolaou stain) ‐ BCC (stain comparison).

Summary of findings Summary of findings table

Question:	What is the diagnostic accuracy of exfoliative cytology for detecting BCC, cSCC or cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults?
Population	Adults with lesions suspicious for BCC, cSCC or for melanoma
Index test	Exfoliative cytology
Comparator test	Dermoscopy
Target condition	BCC
Reference standard	Histology, any method
Action	If accurate, positive diagnosis by exfoliative cytology would reduce the need for biopsies in suspected BCC and help to appropriately select lesions for excision
Quantity of evidence
Number of studies	9	Total lesions with test results	1655	Total with BCC	1120^a
				Total with cSCC	41^b
				Total with melanoma	10^c
Limitations
Risk of bias	High risk for patient selection due to case‐control study design (2/9) or inappropriate exclusion of lesions (1/9), and unclear due to poor reporting of recruitment and exclusion criteria (3/9). Unclear risk for the index test due to lack of reporting diagnostic thresholds and blinding from the reference standard diagnosis (7/9). Unclear risk of bias due to inadequate reporting of blinding the reference standard (7/9) or the index test (7/9). High risk of bias in flow and timing domain from differential verification (2/9) and exclusion of slides from analysis (1/9); timing of tests was not mentioned in 7/9.
Applicability of evidence to question	High concern due to narrowly defined populations and multiple lesions per patient (6/9), and unclear concern due to poor reporting of patient groups (2/9), so may not be representative of populations eligible for exfoliative cytology. High concern for clinical applicability of exfoliative cytology from lack of reporting cytodiagnostic criteria in adequate detail (5/9). Little information was given concerning the expertise of the cytopathologist or histopathologist.
Detection of BCC: pooled analysis^d
Datasets	Lesions	BCCs	Sensitivity (95% CI)	Specificity (95% CI)
7	1264	1045	97.5% (94.5 to 98.9)	90.1% (81.1 to 95.1)
Numbers observed in a cohort of 1000 people being tested^e
	True positive	False negative	False positive	True negative
	(Appropriately do not receive excision)	(Inappropriately receive excision or undertreated)	(Inappropriately do not receive excision, or overtreated)	(Receive appropriate management – excision or other)
At prevalence 63%	614	16	37	333
At prevalence 86%	839	21	14	126
At prevalence 88%	858	22	12	108
Detection of BCC: pooled analysis^f
Datasets	Lesions	BCCs	Sensitivity (95% CI)	Specificity (95% CI)
7	1264	1045	97.3% (93.5 to 98.9)	94.2% (88.7 to 97.1)
Detection of cSCC, melanoma, any skin cancer
Findings	Studies also evaluated cSCC (2 studies), melanoma (1 study) or any skin cancer (6 studies). cSCC – studies could not be pooled due to different diagnostic approaches; sensitivity ranged from 89% to 100% and specificity from 75% to 99% melanoma – only study (10 melanomas) conducted in 185 pigmented skin lesions, also providing a comparison with dermoscopy: sensitivity and specificity 100% any skin cancer – 4 studies pooled 573 suspicious lesions, with 495 malignant lesions (476 BCCs, 13 cSCCs, 1 melanoma, 4 carcinomas of unspecified histological type, 1 apocrine carcinoma). Pooled sensitivity 97.3% (95% CI 93.5% to 98.9%) and specificity 86.0% (95% CI 73.5% to 93.1%) (uncertain diagnoses classified as test positives). When uncertain diagnoses classified as test negatives, pooled sensitivity became 96.6% (95% CI 90.3% to 98.9%) and specificity 94.7% (95% CI 80.2% to 98.7%).
BCC: basal cell carcinoma; cSCC: cutaneous squamous cell carcinoma; CI: confidence interval. ^aTotal of 1122 BCC cases, of which 2 excluded due to absence of exfoliative cytology result ('test fails'). ^bTotal of 55 cSCC cases, of which 14 excluded: 3 due to absence of exfoliative cytology result ('test fails') and 11 due to insufficient cSCC lesion numbers in individual studies (< 5 cSCCs per study). ^cTotal of 11 cases, of which 1 excluded due to insufficient melanoma lesion numbers in individual studies (< 5 melanomas per study). ^d'Possible BCC' cases classified as index test positive. ^eNumbers for a hypothetical cohort of 1000 lesions are presented for three examples representing different prevalences of BCC, estimated at 25th, 50th (median) and 75th percentiles of BCC prevalence observed across the 9 included studies. ^f'Possible BCC' cases classified as index test negative.

Summary of findings Summary of findings table

Table 1. Test failures due to insufficient cellular material

Study	Stain technique	Slides with inadequate material n (%)	Histological diagnosis
Gordon 1984	Papanicolaou	9 (6)	BCC: 1 cSCC: 1 Actinic keratosis: 7
Christensen 2008^a	Papanicolaou	1 (1)	Actinic keratosis: 1
Christensen 2008^a	MGG	3 (4)	BCC: 1 Actinic keratosis: 2
Durdu 2011	MGG	15 (8)	Melanocytic benign: 6 Non‐melanocytic benign: 9
Nauth 1988	Papanicolaou	18 (8)	BCC: 1 cSCC: 2 Severe precancerous disease: 2 Mild precancerous disease: 6 Benign tumour: 5 Inflammation: 2
BCC: basal cell carcinoma; cSCC: cutaneous squamous cell carcinoma; MGG: May‐Grünwald Giemsa stain technique. ^aWhen diagnosis was made using both Papanicolaou and MGG stained slides, all lesions could be diagnosed cytologically

Table 1. Test failures due to insufficient cellular material

Table 2. Summary of main results

Analysis	Target condition Test	No. studies	Lesions with cytology results (n)	Diseased lesions (n)	Sensitivity (95% CI)	Specificity (95% CI)
Detection of basal cell carcinoma (BCC)
All studies	Studies with cases of BCC	9	1655	1120	—	—
Pooled studies	Standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC ('possible BCC' cases classified as BCC test positive)	7	1264	1045^a	97.5 (94.5 to 98.9)	90.1 (81.1 to 95.1)
—	Standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC ('possible BCC' cases classified as BCC test negative)	7	1264	1045^a	97.3 (93.5 to 98.9)	94.2 (88.7 to 97.1)
Studies not pooled		2	391	75	—	—
—	Nauth 1988: different diagnostic criteria ‐ Munchener scheme (class V = malignant)	1	206	41^b	80.5 (66.0 to 89.8)	74.6 (67.4 to 80.6)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (exfoliative cytology)	1	185^c	34	100 (89.9 to 100)	100 (97.5 to 100)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (dermoscopy)	1	200	34	94.1 (80.9 to 98.4)	98.2 (94.8 to 99.4)
Detection of cutaneous squamous cell carcinoma (cSCC)
All studies	Studies with cases of cSCC	6	1357	52	—	—
Studies not pooled		2	401	41^d	—	—
—	Gordon 1984: standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC	1	141	5^e	100 (56.6 to 100)	98.5 (94.8 to 99.6)
—	Nauth 1988: different diagnostic criteria ‐ Munchener scheme (class V = malignant)	1	206	36^f	88.9 (74.7 to 95.6)	74.7 (67.7 to 80.6)
Studies not included in dataset	< 5 cSCC cases	4	1010	11	—	—
Detection of invasive melanoma and atypical intraepidermal melanocytic variants (MM)
All studies	Studies with cases of MM	2	270	11	—	—
Studies not pooled		1	185^c	10	—	—
—	Durdu 2011: different patient group ‐ pigmented skin lesions (exfoliative cytology)	1	185^c	10	100 (72.3 to 100)	100 (97.6 to 100)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (dermoscopy)	1	200	10	80.0 (49.0 to 94.3)	97.4 (94.0 to 98.9)
Studies not included in dataset	< 5 MM cases	1	85	1	—	—
Detection of any potential skin cancer (BCC or other skin cancer)
All studies	Studies with any skin cancer lesions	9	1655	1200	—	—
Pooled studies	Standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC ('possible BCC' cases classified as BCC test positive)	4	573	495	97.3 (93.5 to 98.9)	86.0 (73.5 to 93.1)
—	Standard cytological criteria used to confirm disease in participants with clinical suspicion of BCC ('possible BCC' cases classified as BCC test negative)	4	573	495	96.6 (90.3 to 98.9)	94.7 (80.2 to 98.7)
Studies not pooled		2	391	123	—	—
—	Nauth 1988: different diagnostic criteria ‐ Munchener scheme (class V = malignant)	1	206	77^g	84.4 (74.7 to 90.9)	92.3 (86.3 to 95.7)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (exfoliative cytology)	1	185^c	46	100 (92.3 to 100)	100 (97.3 to 100)
—	Durdu 2011: different patient group ‐ pigmented skin lesions (dermoscopy)	1	200	46	97.8 (88.7 to 99.6)	98.1 (94.4 to 99.3)
Studies not included in dataset	No skin cancer other than BCC (Christensen 2008; Powell 2000)	2	113	72	—	—
—	Data not reported (Ruocco 1992)	1	578	507	—	—
BCC: basal cell carcinoma; CI: confidence interval; cSCC: cutaneous squamous cell carcinoma; MM: invasive melanoma and atypical intraepidermal melanocytic variants. ^aTwo additional BCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material. ^b1/42 BCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material. ^cFrom a total population of 200 lesions (15 excluded from exfoliative cytology analysis due to insufficient cell material, all 200 examined by dermoscopy). ^d3/44 cSCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material. ^e1/6 cSCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material. ^f2/38 cSCC lesions could not be analysed by exfoliative cytology, due to insufficient cell material. ^g3/80 lesions could not be analysed by exfoliative cytology, due to insufficient cell material.

Table 2. Summary of main results

Table 3. Exfoliative cytology for the detection of BCC: false positive diagnoses

Study	False positive n (%)	Histological diagnosis
Berner 1999	5 (4.6)	Carcinoma, type not specified: 3 atypia: 2
Brown 1979	0 (0)	—
Christensen 2008^a	1 (1.3)	Actinic keratosis: 1
Derrick 1994	1 (0.4)	Trichoepithelioma: 2
Durdu 2011	0 (0)	—
Gordon 1984	7 (5.0)	Actinic keratosis: 1 Marked atypia: 4 Sebhorroeic keratosis: 2
Nauth 1988	18 (8)	BCC: 1 cSCC: 2 Severe precancerous disease: 2 Mild precancerous disease: 6 Benign tumour: 5 Inflammation: 2
Powell 2000	2 (5.4)	Bowenoid actinic keratosis: 1 Bowen's disease: 1
Ruocco 1992	6 (1)	Trichoepithelioma: 3 Syringocystadenoma papilliferum: 2 Pilomatricoma: 1
BCC: basal cell carcinoma; CI: confidence interval; cSCC: cutaneous squamous cell carcinoma. ^aDiagnosis made using both Papanicolaou and May‐Grünwald Giemsa stained slides

Table 3. Exfoliative cytology for the detection of BCC: false positive diagnoses

Table Tests. Data tables by test

Test	No. of studies	No. of participants
1 Exfoliative cytology ‐ BCC (possible BCCs classified as test positives) Show forest plot	7	1264

2 Exfoliative cytology ‐ BCC (possible BCCs classified as test negatives) Show forest plot	7	1264

3 Exfoliative cytology ‐ BCC (pigmented lesion population) Show forest plot	1	185

4 Dermoscopy ‐ BCC (pigmented lesion population) Show forest plot	1	200

5 Exfoliative cytology ‐ BCC (mixed population, Munchener diagnostic criteria) Show forest plot	1	206

6 Exfoliative cytology ‐ cSCC (suspected BCC population) Show forest plot	1	141

7 Exfoliative cytology ‐ cSCC (mixed population, Munchener diagnostic criteria) Show forest plot	1	206

8 Exfoliative cytology ‐ melanoma (pigmented lesion population) Show forest plot	1	185

9 Dermoscopy ‐ melanoma (pigmented lesion population) Show forest plot	1	200

10 Exfoliative cytology ‐ any skin cancer (suspected BCC population, possible BCCs classified as test positives) Show forest plot	4	573

11 Exfoliative cytology ‐ any skin cancer (suspected BCC population, possible BCCs classified as test negatives) Show forest plot	4	573

12 Exfoliative cytology ‐ any skin cancer (pigmented lesion population) Show forest plot	1	185

13 Dermoscopy ‐ any skin cancer (pigmented lesion population) Show forest plot	1	200

14 Exfoliative cytology ‐ any skin cancer (mixed population, Munchener diagnostic criteria) Show forest plot	1	206

15 Exfoliative cytology (Papanicolaou + MGG stain) ‐ BCC (stain comparison) Show forest plot	1	76

16 Exfoliative cytology (MGG stain) ‐ BCC (stain comparison) Show forest plot	1	73

17 Exfoliative cytology (Papanicolaou stain) ‐ BCC (stain comparison) Show forest plot	1	77

Table Tests. Data tables by test