Item and Guide to classification

1. Was the spectrum of patients representative of the patients who will receive the test in practice? Is it a selective sample of patients?  

Differences in demographic or clinical features between the study population and the source population may lead to selection bias or spectrum variation. In this item we will focus on selection bias: is a selective sample of patients included?

• Classify as ‘yes’ if a consecutive series of patients or a random sample has been selected. Information should be given about setting, in‐ and exclusion criteria, and preferably number of patients eligible and excluded. If a mixed population of primary and secondary care patients is used: the number of participants from each setting is presented.

• Classify as ‘no’ if healthy controls are used. Score also ‘no’ if non‐response is high and selective, or there is clear evidence of selective sampling. Score also ‘no’ if a population is selected that is otherwise unsuitable, for example, patients are known to have other specific causes of LBP (severe OA, malignancies, etc).

• Classify as ‘unclear’ if insufficient information is given on the setting, selection criteria, or selection procedure to make a judgment. 

2. Is the reference standard likely to classify the target condition correctly?

Estimates of test performance are based on the assumption that the reference standard will identify nerve root compression due to disc herniation with 100% sensitivity and 100% specificity. Such reference standards are rare. Errors due to an imperfect reference standard may bias the estimation of diagnostic performance. For this review acceptable reference standards are: 1) findings at surgery demonstrating nerve root compression or irritation due to disc herniation; and 2) myelography indicating nerve root compression; and 3) although probably of lower quality, CT/MRI findings indicating nerve root compression;

• Classify as ‘yes’ if one of these procedures is used as reference standards.

• Classify as ‘no’ if you seriously question the methods used, if consensus among observers, or a combination of aspects of physical examination and history (‘clinical judgement’) is used as reference standard. (Use of imaging/surgery is actually a selection criterion, so the latter may not occur )

• Classify as ‘unclear’ if insufficient information is given on the reference standard.

• Classify as ‘not able’ if you consider yourself not capable to assess this item. If you have doubts, for example, regarding the quality of MRI‐procedures but feel not competent to make an adequate assessment, we can consult a radiologist.

3. Is the time period between the reference standard and the index test short enough to be reasonably sure that the target condition did not change between the two tests?

The index tests and reference standard should ideally be carried out at the same time. If there is a considerable delay, misclassification (due to spontaneous recovery or worsening of the condition) may occur.

• Classify as ‘yes’ if the time period between physical examination and the reference standard is one week or less.

• Classify as ‘no’ if the time period between physical examination and the reference standard is longer than one week.

• Classify as ‘unclear’ if there is insufficient information on the time period between index tests and reference standard.  

4. Did the whole sample or a random selection of the sample receive verification using a reference standard of diagnosis?

When not all of the study patients receive confirmation of their diagnosis by a reference standard, partial verification bias may occur. Bias is very likely if the results of the index test influence the decision to perform the reference standard. Random allocation of patients to the reference standard should in theory not affect diagnostic performance. [Verification bias is also known as work‐up bias or sequential ordering bias].

• Classify as ‘yes’ if it is clear that all patients who received the index test went on to receive a reference standard, even if the reference standard is not the same for all patients.

• Classify as ‘no’ if not all patients who received the index test received verification by a reference standard.

• Classify as ‘unclear’ if insufficient information is provided to assess this item.

5. Did patients receive the same reference standard regardless of the index test result?

Differential verification bias occurs when the results of the index tests are verified by different reference standards. This is not unlikely in this review: some patients may be referred for surgery following physical examination, whereas others only go on to receive diagnostic imaging. Bias is likely to occur when this decision depends on the results of the index test.

• Classify as ‘yes’ if it is clear that all patients receiving the index test are subjected to the same reference standard.

• Classify as ‘no’ if different reference standards are used.

• Classify as ‘unclear’ if insufficient information is provided to assess this item.

6. Was the reference standard independent of the index test (i.e. the index test did not form part of the reference standard)?

It is not unlikely that the results of the physical examination are used when establishing the final diagnosis. In this case incorporation bias may occur (overestimating diagnostic accuracy). Knowledge of the results of the index test does not necessarily mean that these results are incorporated in the reference standard. For example, if the reference standard consists of MRI‐results only (regardless of knowledge of the results of the straight leg raising test), the index test is not part of the reference standard. However, if the final diagnosis is based on the results of both MRI‐findings and a positive straight leg raising test, incorporation bias will occur.

• Score ‘yes’ if the index is no part of the reference standard.

• Score ‘no’ if the index test is clearly part of the reference standard.

• Score ‘unclear’ if insufficient information is provided to assess this item.

7. Were the index test results interpreted without knowledge of the results of the reference standard?

8. Were the reference standard results interpreted without knowledge of the results of the index test?

Interpretation of the results of physical examination may be influenced by knowledge of the results of the reference standard, and vice versa. This is known as reviewer bias, and may lead to over‐estimation of diagnostic accuracy. In our review the risk of bias may be substantial as both index test and reference standard often involve a subjective assessment of results. If the index test always precedes the reference standard, interpretation of the results of the index test will usually be without knowledge of the results of the reference standard. The reverse may also be true, although surgery is unlikely to precede physical examination.

• Classify as ‘yes’ if the test results (index or reference standard) are interpreted blind to the results of the other test. Score also ‘yes’ if the sequence of testing is always the same and, consequently, one of the test is interpreted blind for the other.

• Classify as ‘no’ if the assessor is aware of the results of the other test.

• Classify as ‘unclear’ if insufficient information is given on independent or blind assessment of the index test or reference standard.

9. Were the same clinical data available when the index test results were interpreted as would be available when the test is used in practice?

The knowledge of demographic and clinical data, such as age, gender, symptoms, history of low‐backpain, previous treatments, or other aspects of physical examination may influence the interpretation of test results. The way this item is scored depends on the objective of the index test. If an aspect of physical examination is intended to replace other tests, these clinical data should not be available. However, if in practice clinical data are usually available when interpreting the results of the index test, this information should be available to the assessors of the index test.

• Classify as ‘yes’ if clinical data (i.e. patient history, other physical tests) would normally be available when the test results are interpreted and similar data are available in the study.

• Classify as ‘yes’ if clinical data would normally not be available when the test results are interpreted and these data are also not available in the study.

• Classify as ‘no’ if this is not the case, e.g. if other test results are available that can not be regarded as part of routine care.

• Classify as ‘unclear’ if the paper does not explain which clinical information was available at the time of assessment.

10. Were uninterpretable / intermediate test results reported?

Uninterpretable or intermediate test results are often not reported in diagnostic studies. Authors may simply remove these results from the analysis, which may lead to biased results of diagnostic performance. If uninterpretable or intermediate test results occur randomly and are not related to disease status, bias is unlikely. Whatever the cause of uninterpretable results they should be reported in order to estimate their potential influence on diagnostic performance.

• Classify as ‘yes’ if all test results are reported for all patients, including uninterpretable, indeterminate or intermediate results.  

• Classify as ‘yes’ if the authors do not report any uninterpretable, indeterminate or intermediate results AND the results are reported for all patients who were described as having been entered into the study.

• Classify as ‘no’ if you think that such results occurred, but have not been reported.

• Classify as ‘unclear’ if it is unclear whether all results have been reported.

11. Were withdrawals from the study explained?

Patients may withdraw from the study before the results of both index test and reference standard are known. If withdrawals systematically differ from patients remaining in the study, then estimates of diagnostic test performance may be biased. A flow chart is sometimes provided (in more recently published papers) which may help to score this item.

• Classify as ‘yes’ if it is clear what happens to all patients who entered the study (all patients are accounted for, preferably in a flow chart).  

• Classify as ‘yes’ if the authors do not report any withdrawals AND if the results are available for all patients who were reported to have been entered in the study.

• Classify as ‘no’ if it is clear that not all patients who were entered completed the study (received both index test and reference standard), and not all patients are accounted for.

• Classify as ‘unclear’ when the paper does not clearly describe whether or not all patients completed all tests, and are included in the analysis.

Note: In many diagnostic studies one may doubt whether or not all eligible patients have been entered in the study and are described in the paper. This issue is more strongly related to selection bias and will be scored under item 1.

Additional QUADAS items

12. Did the study provide a clear definition of what was considered to be a “positive” result of the index test?

Aspects of physical examination, for example the straight leg raising test, require a subjective judgement. Furthermore, several methods of performing the test have been described, and several cut‐offs have been proposed. Consequently, it is essential that an adequate description is given of the methods used to carry out (aspects of) physical examination, and how a positive result is defined.

• Classify as ‘yes’ if the paper provides a clear description of the way the index test is performed, including a definition of a positive test result.

• Classify as ‘no’ if no description is given of the way the index test is performed, and no definition is given of a positive test result.

• Classify as ‘unclear’ if the methods of the index test are described, but no clear definition of a positive result has been provided, or vice versa.

13. Was treatment withheld until both index test and reference standard were performed?

If index tests and reference standard are not performed on the same day, some type of intervention may be initiated in between index test and reference standard. This might lead to misclassification (if some recovery of symptoms occurs).

• Classify as ‘yes’ if no treatment is given in the time period between physical examination and the reference standard.

• Classify as ‘no’ if an intervention is given that in your opinion could possibly influence the prognosis of low‐backpain due to nerve root compression / irritation.

• Classify as ‘unclear’ if there is insufficient information regarding treatment between index test and reference standard.

14. Were data on observer variation reported and within acceptable range?

Studies on the reproducibility of physical examination in patients with musculoskeletal pain show that there may be considerable inter‐observer variation. This may strongly influence the diagnostic performance of the index test. It is difficult to give minimal cut‐off scores for inter‐observer agreement. A kappa or ICC of 0.70 is often considered to be acceptable, but this is certainly an arbitrary definition.

• Classify as ‘yes’ if the paper provides information on inter‐observer variation, and the results are acceptable. 

• Classify as ‘no’ if information is given on inter‐observer variation, and the results demonstrate poor agreement.

• Classify as ‘unclear’ if there is insufficient information is provided regarding inter‐observer variation.