Using different statistical formats for presenting health information
Abstract
This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:
To evaluate the effects of using different statistical presentations of the same health information on persuasiveness, understanding, attitude and behavior of healthcare providers and consumers.
Background
Please refer to the Cochrane Handbook Glossary for an explanation of technical terms (http://www.cochrane.org/resources/handbook/glossary.pdf)
Recent efforts to better integrate the results of research into clinical practice have coincided with a growing consensus among researchers, healthcare providers, and healthcare consumers that each of these groups should more directly participate in decisions about health care at all levels. The success of both goals ‐ evidence based practice and participation in healthcare decisions ‐ depends inherently on the clear and effective communication of research evidence.
It has become widely known that clinicians may be more inclined to prescribe a drug that reduces the risk of death by 50% than one that reduces the risk of death from 2% to 1%, although both presentations of the evidence may relate the same benefit (Nexoe 2002). The former refers to the relative risk reduction (RRR) and the latter to the absolute risk reduction (ARR). Investigators have shown similar differences when presenting RRR versus number needed to treat (NNT) to physicians (e.g. the drug must be used by 100 people in order to prevent one death) (McGettigan 1999) Similarly, patients are more willing to start a lipid lowering drug when benefit is presented as an RRR rather than ARR (Hux 1995). Unpublished data from an international internet‐based series of randomized trials testing different approaches to risk communication to consumers (Health Information Project: Presentation Online or HIPPO (http://www.icru.no/hippo/cholesterol/)) are consistent with the above findings (Montori 2003).
Hoffrage and colleagues showed that physicians' inferences about statistical outcomes are more appropriate when they deal with natural frequencies (e.g. 10 out of 1000 women have breast cancer) rather then probabilities (1% of women have breast cancer) (Hoffrage 1998). Similarly, Mellers showed that probabilities are more difficult to understand than frequencies, particularly when events are rare (Mellers 1999). Consequently, health providers as well as consumers are prone to effects exerted by different statistical presentations of the same evidence. While standardization may be important in improving the presentation of research evidence (and participation in healthcare decisions), we are not sure which of the many presentations leads to decisions that are most consistent with the values and preferences of consumers.
The aim of this systematic review is to compare the effects of different statistical presentations of the same evidence about health on healthcare providers and consumers. We will include studies that compared two or more statistical presentations of an effect measure (i.e. RRR, risk difference NNT, natural frequencies and probabilities). These are the types of presentations that are commonly generated by scientific studies, easily replicated, and generally applied. Other authors have reviewed the effects of different graphical, numerical and presentations of risks (Lipkus 1999; Edwards 2002). We review the evidence for negative versus positive framing of health information separately (Akl 2007a).
Objectives
To evaluate the effects of using different statistical presentations of the same health information on persuasiveness, understanding, attitude and behavior of healthcare providers and consumers.
Methods
Criteria for considering studies for this review
Types of studies
Randomized controlled trials (RCTs), quasi‐RCTs, controlled before and after studies (CBAs).
Types of participants
Healthcare providers, policy makers, patients, and the general public.
Types of interventions
Interventions that compare two or more statistical presentations of an effect measure (i.e. RRR, risk difference, NNT, natural frequencies and probabilities) of the same evidence about health.
We will exclude interventions that compare different framing of the message on the same evidence (as this is covered by a separate review: Akl 2007a), different graphical or verbal presentations of the same evidence, different orders of comparing risks or comparisons, different amount of information or different media to present the same information.
Types of outcome measures
Any outcome measures (including self‐reported) of persuasiveness, understanding, attitude and behavior (including choice made). Persuasiveness refers to the power to induce taking a course of action or the embracing of a point of view by means of argument or entreaty. We will consider these outcomes in the setting of both real and hypothetical decisions.
Search methods for identification of studies
The search for this review will be part of a larger search for studies assessing different presentations of the same information about health. We will search of MEDLINE (Ovid), EMBASE (Ovid), PsycINFO (Ovid) and the Cochrane Central Register of Controlled Trials (CENTRAL), using no language or date restriction.
Search strategies for MEDLINE, EMBASE and PsycINFO are presented in Additional Table 1.
| Database | Search strategy |
| MEDLINE (starting January 1966) | 1‐randomized controlled trial.pt. |
| EMBASE (starting January 1980) | 1‐randomized controlled trial.pt. |
| PsycINFO (starting January 1887) | 1‐randomi#ed controlled trial$.tw. |
We will search CENTRAL using FRAM* and PRESENT* as text words. In addition, we will search MEDLINE, EMBASE and PscyINFO using "framing" as title word (framing.ti).
We will use the "Related Articles" feature of PubMed MEDLINE to find additional articles. We will search MEDLINE and PsycINFO databases for articles published by the first authors of included articles and of excluded but closely related articles. We will review the reference lists of related systematic reviews, included articles and excluded but closely related articles. Finally, we will contact experts in the field.
Data collection and analysis
Selection of trials
Two review authors will independently screen the title and abstract of identified articles for relevance. We will retrieve the full text of articles judged potentially relevant by at least one review author. Two authors will then independently screen the full text article for inclusion or exclusion. The authors will resolve any disagreements by discussion or by consulting a third review author.
Methodological quality
Two review authors will independently assess the methodological quality of each included study and resolve their disagreements by discussion or by consulting a third author. We will assess the following methodological data:
1. Randomization (according to the criteria set out in Ryan 2007)
2. Allocation concealment (according to the criteria set out in Ryan 2007)
3. Outcome type; we will use the following categorization:
-
Objective (example: actual use of mammography)
-
Self‐reported past/present (example: self‐reported use of mammography, personal attitudes towards mammography)
-
Self‐reported future (example: self‐reported intention to use mammography)
-
Hypothetical (example: preferences of non‐cancer patients for chemotherapy versus none)
4. Follow‐up (we will record the percent follow‐up for each study).
Assessment of methodological quality will be reported in an additional table.
Data extraction
We will develop a data extraction form. Two review authors will independently extract data from each included study and resolve their disagreements by discussion or by consulting a third author. We will extract data relating to study design, intervention type, number and type of participants, outcomes assessed and study results. We will contact the study authors for incompletely reported data.
Analysis
We will analyze the results of included studies for each of the comparisons that compare identical information. Because outcomes in these studies are typically scaled responses to survey questions, we will standardize the effects using Hedges adjusted standardized mean difference (SMD). For comparisons where we would not be able to calculate the SMD directly, we will estimate t‐values for the study and the corresponding SMD using SMD=2t/sqrt(N) (Cooper 1994) and adjust it using the same adjustment factor; in all cases, we will calculate the adjusted standard error for the resulting SMD. We will pool multiple outcome measures for a single trial ‐ for example, three different questions about attitude or responses to three different scenarios by the same participants‐ using fixed‐effect models into a single SMD for that comparison. We will pool data from different studies when appropriate using random‐effects models with the inverse variance approach.
If a paper reports the results of two or more separate comparisons enrolling different participants, we will treat these as such. If a study uses a between‐subjects factorial design to compare an intervention of interest across another factor, we will treat these as separate comparisons. When practical, we will conduct subgroup analyses for different types of participants. We will conduct sensitivity analyses by comparing the results of studies of lower methodological quality with those of higher methodological quality.
We will create inverted funnel plots of individual study results plotted against inverse of the variance in order to check for possible publication bias.
| Database | Search strategy |
| MEDLINE (starting January 1966) | 1‐randomized controlled trial.pt. |
| EMBASE (starting January 1980) | 1‐randomized controlled trial.pt. |
| PsycINFO (starting January 1887) | 1‐randomi#ed controlled trial$.tw. |
