Intervenciones con medios masivos de comunicación para la reducción del estigma relacionado con la salud mental
Resumen
Antecedentes
El estigma relacionado con la salud mental es generalizado y tiene efectos adversos importantes sobre las vidas de los pacientes con problemas de salud mental. Sus dos componentes principales son la discriminación (ser tratado injustamente) y el prejuicio (estigmatizar actitudes). Las iniciativas antiestigma a menudo incluyen intervenciones con medios masivos de comunicación, y dichas intervenciones pueden ser costosas. Es importante saber si las intervenciones con medios masivos de comunicación son efectivas.
Objetivos
Evaluar los efectos de las intervenciones con medios masivos de comunicación sobre la reducción del estigma (discriminación y prejuicio) relacionado con las enfermedades mentales en comparación con controles inactivos, y comparar la efectividad sobre la base de la naturaleza de la intervención (p.ej., número de componentes de medios masivos de comunicación), el contenido de la intervención (p.ej., tipo de mensaje primario) y el tipo de medio (p.ej., material impreso, Internet).
Métodos de búsqueda
Se hicieron búsquedas en 11 bases de datos: Registro Cochrane Central de Ensayos Controlados (Cochrane Central Register of Controlled Trials) (CENTRAL, The Cochrane Library, número 7, 2011); MEDLINE (OvidSP),1966 hasta el 15 agosto 2011; EMBASE (OvidSP),1947 hasta el 15 agosto 2011; PsycINFO (OvidSP), 1806 hasta el 15 agosto 2011; CINAHL (EBSCOhost) 1981 hasta el 16 agosto 2011; ERIC (CSA), 1966 hasta el 16 agosto 2011; Social Science Citation Index (ISI), 1956 hasta el 16 agosto 2011; OpenSIGLE (http://www.opengrey.eu/), 1980 hasta el 18 agosto 2012; Worldcat Dissertations and Theses (OCLC), 1978 hasta el 18 agosto 2011; metaRegister of Controlled Trials (http://www.controlled‐trials.com/mrct/mrct_about.asp), 1973 hasta el 18 agosto 2011; e Ichushi (OCLC), 1903 hasta el 11 noviembre 2011. Se verificaron las referencias de los artículos y revisiones, y las citas de los estudios incluidos. También se buscó en los resúmenes de congresos y sitios web, y se contactó con los investigadores.
Criterios de selección
Ensayos controlados aleatorios (ECA), ECA por grupos o estudios de series de tiempo interrumpido de las intervenciones con medios masivos de comunicación en comparación con controles inactivos en miembros de la población en general o cualquiera de los grupos integrantes (con la exclusión de los estudios en los que todos los participantes eran personas con problemas de salud mental), con la salud mental como objeto de la intervención y con medidas de resultado que incluyeran la discriminación o el prejuicio.
Obtención y análisis de los datos
Dos autores, de forma independiente, extrajeron los datos y evaluaron el riesgo de sesgo de los estudios incluidos. Se estableció contacto con los autores de los estudios para obtener la información faltante. La información acerca de los efectos adversos se recopiló de los informes de los estudios. Los resultados primarios fueron la discriminación y el prejuicio, y los resultados secundarios fueron el conocimiento, el costo, el alcance, el recuerdo y la conciencia sobre las intervenciones, la duración/sostenibilidad de los efectos de los medios de comunicación, las reacciones de las audiencias al contenido del medio de comunicación y los efectos adversos imprevistos. Se calcularon las diferencias de medias estandarizadas y los odds ratios. Se realizó una síntesis principalmente narrativa debido a la heterogeneidad de los estudios incluidos. Se realizaron análisis de subgrupos para examinar los efectos de la naturaleza, el contenido y el tipo de intervención con medios masivos de comunicación.
Resultados principales
Se incluyeron 22 estudios con 4490 participantes. Todos eran ensayos aleatorios (tres eran ECA por grupos), y 19 de los 22 estudios presentaron datos de resultado analizables. Diecisiete de los estudios incluyeron poblaciones de estudiantes. La mayoría de los estudios estuvieron en riesgo incierto o alto de sesgo para todas las formas de sesgo excepto el sesgo de detección.
Los hallazgos de los cinco ensayos con resultados de la discriminación (n= 1196) fueron contradictorios, con efectos que mostraron reducciones, aumentos o que fueron compatibles con ninguna prueba del efecto. La diferencia de medias estandarizada (DME) mediana para los tres ensayos (n= 394) con resultados continuos fue de ‐0,25; y la DME varió de ‐0,85 (intervalo de confianza [IC] del 95%: ‐1,39 a ‐0,31) a ‐0,17 (IC del 95%: ‐0,53 a 0,20). Los odds ratios (OR) para los dos estudios (n= 802) con resultados de discriminación dicotómicos no mostraron pruebas del efecto: los resultados fueron de 1,30 (IC del 95%: 0,53 a 3,19) y 1,19 (IC del 95%: 0,85 a 1,65).
Los 19 ensayos (n= 3176) con resultados del prejuicio tuvieron DME medianas que favorecieron la intervención, en los tres períodos a continuación: ‐0,38 (inmediato), ‐0,38 (una semana a dos meses) y ‐0,49 (seis a nueve meses). Las DME para los resultados del prejuicio a través de todos los estudios variaron de ‐2,94 (IC del 95%: ‐3,52 a ‐2,37) a 2,40 (IC del 95%: 0,62 a 4,18). Las DME medianas indican que las intervenciones con medios masivos de comunicación pueden tener un efecto pequeño a medio en cuanto a la reducción del prejuicio y son equivalentes en la reducción del nivel de prejuicio, desde el asociado con la esquizofrenia hasta el asociado con la depresión grave.
Los estudios fueron muy heterogéneos, estadísticamente, en las poblaciones, las intervenciones y los resultados, y sólo se justificó la realización de dos metanálisis dentro de dos subgrupos. Los datos de los resultados secundarios fueron escasos. Los datos de los costos fueron proporcionados a petición para tres estudios (n= 416); fueron sumamente variables y no consideraron el costo‐efectividad. Dos estudios (n= 455) incluyeron información acerca de los efectos adversos y los dos informaron que no encontraron ninguno.
Conclusiones de los autores
Las intervenciones con medios masivos de comunicación pueden reducir el prejuicio, aunque no hay pruebas suficientes para determinar sus efectos sobre la discriminación. Se conoce muy poco acerca de los costos, los efectos adversos u otros resultados. La revisión encontró pocos estudios en países de ingresos bajos y medios, o con empleadores o profesionales de la salud como grupo destinatario, y ninguno se dirigió a los niños o a los adolescentes. Los hallazgos son limitados por la calidad de las pruebas, que fue baja para los resultados primarios de la discriminación y el prejuicio, baja para los efectos adversos y muy baja para los costos. Se necesita más investigación para establecer los efectos de las intervenciones con medios masivos de comunicación en la discriminación, para comprender mejor qué tipos de intervención con medios masivos de comunicación funciona mejor, para aportar pruebas acerca del costo‐efectividad y para completar las brechas en las pruebas acerca de los tipos de medios masivos de comunicación que no fueron cubiertos en esta revisión. Dicha investigación debe utilizar métodos consistentes, informar datos de forma más sistemática con guías de informe y basarse menos en las poblaciones de estudiantes.
PICO
Resumen en términos sencillos
Intervenciones con medios masivos de comunicación para la reducción del estigma hacia los pacientes con problemas de salud mental
Los pacientes definen el estigma de diversas maneras. La revisión se centró en dos aspectos clave del estigma: la discriminación (tratar a las personas injustamente debido al grupo al que pertenecen) y el prejuicio (actitudes y emociones negativas hacia determinados grupos). Los pacientes con problemas de salud mental a menudo experimentan estigma. Puede tener efectos terribles sobre sus vidas. Los medios masivos de comunicación son medios destinados a comunicarse con un gran número de personas sin usar el contacto cara a cara. Algunos ejemplos incluyen periódicos, anuncios publicitarios, folletos, DVD, televisión, radio, cine y la Internet. Las campañas antiestigma a menudo incluyen intervenciones con medios masivos de comunicación, y pueden ser costosas, de manera que es importante determinar si el uso de intervenciones con medios masivos de comunicación puede reducir el estigma.
Se examinaron los estudios que comparaban a pacientes que habían visto u oído una intervención con medios masivos de comunicación acerca de los problemas de salud mental con personas que no habían visto ni oído ninguna intervención, o que habían visto una intervención que no contenía nada acerca de las enfermedades mentales o el estigma. Se intentó determinar qué efectos podrían tener las intervenciones con medios masivos de comunicación en la reducción del estigma hacia los pacientes con problemas de salud mental.
Se encontraron 22 estudios con 4490 personas. Cinco de estos estudios tuvieron datos acerca de la discriminación y 19 tuvieron datos acerca del prejuicio. Se encontró que las intervenciones con medios masivos de comunicación pueden reducir, aumentar o no tener ningún efecto sobre la discriminación. Se encontró que las intervenciones con medios masivos de comunicación pueden reducir el prejuicio. La cantidad de reducción puede considerarse de pequeña a media, y es similar desde la reducción del nivel de prejuicio asociado con la esquizofrenia hasta el asociado con la depresión grave. La calidad de las pruebas acerca de la discriminación y el prejuicio fue baja, de manera que no es posible tener gran certeza acerca de estos hallazgos. Sólo tres estudios proporcionaron información acerca de los costos económicos y dos acerca de los efectos adversos, y hubo limitaciones en la forma en que se evaluaron los mismos, de manera que no es posible establecer conclusiones acerca de estos aspectos.
Authors' conclusions
Summary of findings
| Mass media compared with inactive control for reducing mental health‐related stigma | ||||
| Patient or population: General public or any of its constituent groups (excluding groups comprising solely of people with mental ill health) Settings: Any Intervention: Mass media Comparison: Inactive control | ||||
| Outcomes | Relative effect | No of Participants | Quality of the evidence | Comments |
| Discrimination (i) not registering to attend focus group with people with schizophrenia (immediate)1 (ii) not visiting sheltered workshop for people with mental illness (9 months)2 (iii) distance placed between chairs when anticipating meeting with person with Tourette's syndrome3 | ORs (802 participants, 2 studies) (i) 1.30 (0.53 to 3.19) (i) 1.19 (0.85 to 1.65) SMDs (394 participants, 3 studies) (iii) median SMD ‐0.25 Range of SMDs ‐0.85 (‐1.39 to ‐0.31) to ‐0.17 (‐0.53 to 0.20) | 1196 (5) | ⊕⊕⊝⊝ | Quality of evidence downgraded for study limitations (risk of bias) and indirectness of measures (see Quality of the evidence for full details) |
| Prejudice Multiple scales4 Follow‐up (i) Immediate5 (ii) 1 week to 2 months6 (iii) 6 to 9 months7 | Median SMD (i) ‐0.38 (ii) ‐0.38 (iii) ‐0.49 Range of SMDs ‐2.94 (‐3.52 to ‐2.37) to 2.40 (0.62 to 4.18) Median SMDs are equivalent to reductions of 0.22, 0.22 and 0.29 points on the 1 to 4 point SDS scale [7], which is equivalent to reducing the level of prejudice from that associated with symptoms of schizophrenia to the level associated with symptoms of major depression8 | 3176 (19) | ⊕⊕⊝⊝ | Quality of evidence downgraded for study limitations (risk of bias) and indirectness of populations (see Quality of the evidence for full details) |
| Cost (i) Audiovisual public service announcement (ii) a) CD‐ROM b) printed manual (iii) magazine article | Relative costs (i) 100 US dollars (equivalent to £64 GBP) vs. nil (ii) a) 35,000 Australian dollars (£22,404 GBP) for 250 e‐learning CDs @ 140 dollars per CD (ii) b) 7,140 Australian dollars(£4,570 GBP) for 238 manuals @ 30 dollars per manual vs. nil (iii) 'printing costs' vs 'printing costs' | 416 (3) | ⊕⊝⊝⊝ | Quality of evidence downgraded for inconsistency of results, imprecision, and 'other' (data only being available on request, lack of cost‐effectiveness data) (see Quality of the evidence for full details) |
| Unforeseen adverse effects | Statements in two studies: (i) ‘Given that this was an educational intervention with a non‐clinical sample, there was no formal enquiry about adverse events. Informally, no adverse events were reported'; (ii) ‘No adverse effects, such as an increase in stigma as a result of the intervention, were identified’ | 2 (455) | ⊕⊕⊝⊝ | Quality of evidence downgraded for study limitations (risk of bias) and imprecision (see Quality of the evidence for full details) |
| CI: Confidence interval; OR: Odds ratio; SMD: Standardised mean difference | ||||
| GRADE Working Group grades of evidence | ||||
| [1] Penn 2003 [2] Yoshida 2002 [3] Woods 2002; Woods 2003; Woods 2005 [4] 28 different measures were used (see Characteristics of included studies) [5] Brown 2010; Bunn 2009; Coleman 2005; Corrigan (submitted); Finkelstein 2008; Iobst 2008; Matthews 2009; Morgan Owusu 2002; Penn 2003; Smith 2007; Woods 2002; Woods 2003; Woods 2005; Yoshida 2002 [6] Russell 1988; Demyan 2009; Brown 2010; Jorm 2010a [7] Yoshida 2002; Finkelstein 2008; Jorm 2010a [OR:1 indicated decreased stigma, > 1 indicates increased stigma; SMD < 0 indicates decreased stigma, > 0 indicates increased stigma] [8] Link 1999 This reference for the scale also reports a population standard deviation of 0.59 for social distance in relation to schizophrenia using the SDS from the observational study, General Social Survey 1996, USA. It gives SDS scores for different mental health conditions which are used for further interpretation. | ||||
Background
Description of the condition
Stigma has been defined and conceptualised in a number of different ways. The conceptual framework used in this review is that stigma comprises ignorance (lack of knowledge), prejudice (stigmatising attitudes) and discrimination (being treated unfairly, a behaviour concept) (Thornicroft 2007). Our review focused on the latter two concepts: prejudice and discrimination, with knowledge as a secondary outcome. This is because what constitutes de‐stigmatising knowledge is a contested issue. For example, Slade has described the paradigmatic arguments about fundamental ways (clinical models, disability models, diversity models) to understand experiences labelled as mental illness, and highlights the lack of agreement over these (Slade 2009). There is also significant disagreement between different professional groups about what treatments are helpful (Lauber 2005). Our focus on discrimination and prejudice was also because these are central to most conceptualisations of stigma, e.g. (Link 2001a, Corrigan 2005), but these other models only contain one aspect of knowledge, namely stereotype awareness. A further pragmatic consideration was that because the review covered interventions which did not have a stated aim of reducing stigma, having a stigma‐related outcome was a key aspect in inclusion decisions. Whilst prejudice and discrimination are always stigma‐related, knowledge is a much broader concept and may or may not be related to stigma, and therefore would have been problematic as a primary outcome.
Discrimination and prejudice were relevant concepts for this review because they focus on stigmatisers (the targets of the mass media interventions reviewed here) rather than stigmatised people. Some commentators focus on aspects of prejudice, viewing stigma as a social process of 'othering', blaming and shaming (Deacon 2006), whereas others have argued for a purely discrimination‐based conceptual framework (Sayce 1998). Phelan and colleagues have investigated the possible similarity between the concepts of stigma and prejudice, and concluded that the two models have much in common, with most differences being a matter of focus and emphasis (Phelan 2008). Discrimination and prejudice are key elements in Rüsch's discussion of Link's (Link 2001a) conceptualisation of the stigma process as labelling, separation, stereotype awareness, stereotype endorsement, prejudice, and discrimination in a context in which social, economic, or political power is exercised (Rüsch 2005). Discrimination and prejudice are also core elements in Corrigan's framework (Corrigan 2005). In this review, in line with the Thornicroft 2007 model, we used the term 'discrimination' to refer to behavioural elements such as observed discriminatory behaviour and discrimination experiences reported by people with mental health problems, although we recognise that discrimination can also operate at the structural level, for example in discriminatory media reporting, policy and legislation (Corrigan 2004c). Following the same model, we used the term 'prejudice' to encompass concepts such as attitudes towards, stereotypes about, emotional reactions to, and desire for social distance from, people with mental ill health.
Mental health‐related stigma is widespread. A recent survey of public reactions to case descriptions of people with schizophrenia and major depression, involving nationally‐representative samples in 15 countries in Africa, Asia, Australasia, Europe, and in both North and South America, found significant levels of public stigma in all countries studied, although there was some variation between them (Pescosolido 2009). A US study using the same methodology found that in 2006, 62% of the public reported being unwilling to work closely with people with schizophrenia, and 52% were unwilling to socialise with them (Pescosolido 2010). The figures for depression were 47% and 30% respectively (Pescosolido 2010). Furthermore, some studies have reported a worsening of certain attitudes in recent years (Angermeyer 2005; Mehta 2009). A 2009 study investigating the discrimination experiences of 739 people with schizophrenia in 27 countries found that negative discrimination was experienced by 47% in making or keeping friends, by 43% from family members, by 29% in finding a job, 29% in keeping a job, and by 27% in intimate or sexual relationships (Thornicroft 2009). Stigma can be compounded by other axes of difference. For example people with mental ill health who belong to other groups facing stigma and discrimination, such as those from black and ethnic minority groups, lesbian and gay individuals, and asylum seekers, may be particularly disadvantaged (e.g. Gary 2005). Furthermore, both mental ill health itself and mental health‐related discrimination and prejudice can make people more likely to become members of other groups subject to stigma, such as those experiencing homelessness, unemployment and poverty.
Stigma has major adverse effects on the lives of people with mental health problems (McDaid 2008). Public attitudes commonly include stereotypes of incompetence, beliefs about dangerousness, attributions of blame, expectations of poor prognosis, negative emotional responses, and a desire for social distance (Hinshaw 2000). Each of these can directly affect the well‐being and quality of life of people with mental ill health. People with mental health problems experience significant discrimination which spans all major domains of life (Thornicroft 2006; Thornicroft 2009) and includes exclusion from employment (Stuart 2006b) with consequent poverty, negative impacts on intimate relationships and parenting (Hinshaw 2005), reduced access to and engagement with mental health services (Corrigan 2004b), and poorer physical health care (Jones 2008). Discrimination and prejudice can also have significant negative effects on the way that people with mental ill health feel about themselves, such as inducing internalised stigma (Corrigan 2002b; Ritsher 2003). In addition, the anticipation of discrimination can lead people to use strategies of avoidance and concealment, which may further contribute to social exclusion and poor quality of life (Thornicroft 2009). Mental health‐related stigma also affects families and others close to the person with mental ill health, and these people can experience 'courtesy stigma' or 'stigma by association' (Corrigan 2004a). In addition stigma has damaging effects at the societal level, robbing the community of the contributions that people with mental ill health could make were it not for stigma, and helping to maintain fear about mental illness (Corrigan 2005). Negative media reporting ‐ a form of discrimination in itself ‐ also shapes attitudes and influences behaviour, thereby producing or reinforcing stigma (Wahl 1995).
Description of the intervention
Mass media has the potential to de‐stigmatise as well as to stigmatise (Philo 2010). This review focused on mass media interventions, rather than on other types of intervention, because such interventions are able to reach large numbers of people and so have the potential for achieving population‐level change. Large scale change may be difficult with other types of intervention.
Following Bala 2008 and Brinn 2010, we defined mass media as channels of communication intended to reach large numbers, which are not dependent on person‐to‐person contact. A mass media intervention is one that uses such channels. There are many different forms of mass media, for example: print (e.g. newspapers, magazines, billboards, pamphlets, flyers, coasters); recordings (e.g. audio cassettes, videos, CDs, DVDs); radio; television; cinema; mobile phones (e.g. mobile device applications); and the Internet (e.g. websites, blogs, podcasts, viral messaging, social networking sites) (Donovan 2003).
Not all mass media interventions that may reduce stigma have an explicit intention to do so. Examples may include the positive portrayal of a person with a mental health problem on television without a planned intention, or media coverage of a celebrity’s diagnosis with a mental illness. Some health promotion campaigns may also reduce stigma, even though this is not their primary purpose.
Interventions vary in the extent to which they target particular groups. Some are directed at the general population and some are targeted at specific groups, for example young people or employers. Mass media interventions may come from various sources, including governments, community groups and organisations. An intervention may focus on stigma in relation to mental ill health in general, a specific mental health condition, or all forms of disability including mental health disabilities. Interventions may be based, implicitly or explicitly, on diverse conceptualisations of stigma or mental health problems, and may use different theories to underpin the design of the interventions (see How the intervention might work). Interventions sometimes take place at a single time point, or may be short‐term or sustained over a long period. Furthermore they vary in intensity (e.g. extent and frequency of advertising) and reach (e.g. proportion of intended population who see the advertisements).
How the intervention might work
In many respects, mass media interventions to combat stigma work using the same mechanisms operating in advertising and marketing. When these techniques are applied to address social issues rather than to sell commercial products or to promote a particular organisation, this is referred to as social marketing (Donovan 2003). However, it is recognised that social and commercial marketing differ in significant ways, most markedly in that the attitudes and behaviours which social marketing seeks to change are often more complex and hence more challenging to change than commercial behaviour (Donovan 2003).
Social marketing draws on several models of communication and persuasion, and uses various behaviour change theories. A number of these derive from, or overlap with, those from the health psychology, social psychology, public health or health promotion fields. Some of the major theories include: the theory of reasoned action; the health belief model; the transtheoretical (stages of change) model; the theory of planned behaviour; social learning theory; the Rossiter‐Percy motivational model; the diffusion theory model; and the elaboration likelihood model (Donovan 2003; Noar 2006). Symbolic communication and modelling are also processes thought to be important in mass media interventions (Bandura 2001). The mass media operates by potentially influencing not only individuals but also communities and policy makers (Andreasen 2006).
It is not uncommon for mass media material to contain some form of personal narrative from people who have experienced mental health problems, such as celebrities, or members of the public sharing stories about themselves and their lives. These may reduce stigma because they are an indirect form of social/interpersonal contact with people with mental health problems, and this form of contact has been theorised, and demonstrated, to reduce stigma (Couture 2003; Pettigrew 2006; Corrigan 2012). Such narratives may also reduce stigma by increasing awareness of the variation amongst members of out‐groups and in‐groups, increasing social identity complexity, and increasing tolerance (Schmid 2009). Alternatively, narratives may act as 'mediated associations' in which an individual feels empathy towards the suffering of another without the other's physical presence, elicited through language (stories, films) or pictorial representation (e.g. photographs), with this empathy then being translated into a commitment to social justice (Kumagai 2008).
Our conceptualisation does not necessarily imply a linear mode of action with changes in prejudice leading to changes in discrimination. For example, a communication which imparts the message that it is unlawful to discriminate on the basis of mental health could change behaviour (discrimination) outcomes without necessarily changing attitudes (prejudice). Social marketing theory (Donovan 2003) states the importance of including a clear call to action. In England’s national anti‐stigma programme (Henderson 2009), the current call to action is ‘It’s time to talk’ including directions to keep in touch with people with mental health problems. As the loss of friends and being shunned are common reported experiences of discrimination, this is an example of how mass media messages may directly address one form of discrimination. Additionally mass media may change perceptions of social norms, with the change in social norms leading to behavior changes, leaving individual attitudes untouched (Wakefield 2010). Furthermore, subtle factors in communication can influence social behaviour without necessarily being mediated by conscious choice (Bargh 1996) and so mass media may affect behaviour directly. It is also recognised that changes in attitudes may not necessarily translate into changes in behaviour (Marcus 1998). Equalities and human rights legislation have a significant potential to reduce discrimination (Callard 2012). However research in this area is limited and it is not currently known how legislative approaches compare to mass media approaches in their effectiveness in reducing discrimination against people with mental health problems. Protest is another approach for countering discriminatory behaviour. Corrigan has compared protest and other approaches including indirect contact (one form of mass media intervention), and found that protest‐based interventions were rarely studied, and when they were they did not yield significant reductions in stigma, whilst indirect contact significantly reduced discriminatory intentions (Corrigan 2012).
Many variables are believed to influence the effectiveness of mass media interventions, including: whether an intervention is based on formative research; whether it has a theoretical basis; the degree of targeting; campaign intensity; the media channel (Noar 2006); and the 'ad creative' (the creative design and content of the intervention). In addition, whether the mass media element is part of a multi‐faceted campaign (Link 2001b) and which particular messages are conveyed (Clement 2010) are likely to be important. Reviews of mass media interventions in other fields have reported that the duration of campaigns appears to be important, with campaigns of longer duration being more effective (e.g. Friend 2002). Furthermore, interventions that are effective in reducing stigma in high‐income countries may not necessarily be effective if exported without modification to low‐ or middle‐income countries (Rosen 2003) for reasons relating to both available resources and culture. Within one country an anti‐stigma intervention may be received differently by different ethnic groups (Glasgow Anti Stigma Partnership 2007). We took many of these variables into consideration in planning the comparisons and subgroup analyses that were undertaken, as well as the data extraction for this review.
Why it is important to do this review
Stigma is highly prevalent and has serious adverse effects on the lives of people with mental ill health (as described above). Consequently there is a need to find effective ways to reduce mental health‐related stigma. Mass media interventions are one of the most commonly used types of intervention, and they are being carried out throughout the world (Sartorius 2005; Callard 2008). National programmes aiming to reduce mental health‐related stigma and containing mass media components are taking place in a number of countries, such as New Zealand (Vaughn 2004), England (Henderson 2009) and Scotland (Dunion 2005). Local and regional interventions are also widespread. Mass media interventions can be scaled‐up with relative ease to the population‐level and hence, if effective, are a feasible intervention for large‐scale change. If mass media interventions were to produce only a small magnitude of change, this may translate into important impacts at the population level (Noar 2006). Although other types of interventions, such as direct social contact (Couture 2003), have occasionally been used on a large scale (Corrigan 2006; Evans‐Lacko 2012a), this is unusual and presents greater implementation challenges than mass media approaches.
There is a recognised evidence gap in this field (Weiss 2006; Callard 2008). This systematic review synthesises what is currently known to enable future research to be appropriately focused. Such systematic investigation provides guidance for those who are planning initiatives, about whether mass media interventions are worthwhile; about optimal intervention design; and about any possible harm. As mass media interventions may be expensive (Austin 1998), evidence of ineffectiveness will free anti‐stigma resources for other approaches.
A number of non‐systematic reviews of mass media and other interventions to reduce mental health‐related stigma have been undertaken, (for example Warner 2001; Pinfold 2005; Rüsch 2005; Sartorius 2005; Warner 2005; Callard 2008; Hinshaw 2008; McDaid 2008; Thornicroft 2008). Recently four systematic reviews of interventions to reduce mental health‐related stigma have been conducted, but none focus on mass media interventions (Holzinger 2008; Schachter 2008; Yamaguchi 2011; Corrigan 2012). Our review adds to the growing body of systematic review evidence about the effectiveness of mass media interventions in other fields (Grilli 2002; Vidanapathirana 2005; Bala 2008; Brinn 2010). The systematic review of mass media anti‐stigma interventions in mental ill health is likely to create a greater understanding of this vital area, and to help to underpin the development of future population‐level interventions to combat mental health‐related stigma.
Objectives
The primary objective was to assess the effects of mass media interventions on reducing stigma related to mental ill health in terms of discrimination and prejudice compared to inactive controls. The secondary objective was to make comparisons of effectiveness based on the nature of the intervention (e.g. number of mass media components), the content of the intervention (e.g. type of primary message), and the type of media (e.g. print, internet).
Methods
Criteria for considering studies for this review
Types of studies
Two types of study were eligible: randomised controlled trials (RCTs), including cluster trials; and interrupted time series (ITS) analyses. In ITS studies the intervention required a defined start and end point, and at least three data points before the intervention was introduced and at least three after its end point.
RCTs were selected as these provide the strongest level of evidence on effectiveness. ITS analyses were included because this study design is commonly used to assess the effectiveness of mass media interventions (Grilli 2002; Vidanapathirana 2005). The specific criteria for ITS studies were based on Cochrane Consumers and Communication Review Group (CCCRG) study design guidance (Ryan 2009) which advocate using the criteria proposed by the Cochrane Effective Practice and Organisation of Care Review Group (EPOC) (EPOC undated) to minimise bias.
Types of participants
Participants were members of the general public or any of its constituent groups (e.g. occupational or sociodemographic groups or any other target group), including children. We excluded studies in which the whole sample are people with mental health problems. This is because a separate Cochrane review addressing this topic is registered with the Cochrane Schizophrenia Review Group.
Types of interventions
We included interventions if they met all of the following criteria:
-
It was a mass media intervention, defined as an intervention that uses a channel of communication intended to reach large numbers, and is not dependent on person‐to‐person contact. Such channels include newspapers, billboards, pamphlets, DVDs, television, radio, cinema, some web and mobile phone‐based media, street art and ambient media. Interventions may have been undertaken at international, national, regional or local level. Studies that used mass media interventions on a small scale in experimental contexts were also eligible for inclusion, as it is the nature of the intervention and its potential for scaling‐up that are the requisite factors. The mass media component(s) must have been substantial, in that it comprised more than 50% of the total intervention (e.g. in terms of time). Interventions with non‐mass media components were eligible, as long as this criterion was met. The mass media intervention may have used one, two or more types of mass media.
-
An intervention may have taken place at a single time point, may have been short‐term or sustained over a long period.
-
The content of the intervention may have taken any form including: factual material, fiction, persuasive material, personal narratives, slogans, symbols, images, quizzes and games.
-
Mental health was the subject (or one of the subjects) of the intervention. For the purpose of this review, mental health included all conditions listed in the Diagnostic and Statistical Manual of Mental Disorders (DSM‐IV‐TR) (APA 2000), including developmental disorders, dementia, learning disability and substance abuse. Interventions that did not specify a particular condition were also eligible, e.g. interventions that referred to psychological or emotional problems, mental well‐being, etc. Interventions that were not exclusive to mental ill health, but encompassed it, such as disability interventions, were eligible as long as outcomes were reported that related specifically to people with mental ill health.
-
The comparator was an inactive control, e.g. the control group received an intervention with no messages or other content likely to reduce mental health‐related stigma, or received no intervention.
There was no requirement for an intervention to have any intention to reduce stigma. However, we excluded media reports of violent acts committed by people with mental ill health, as these have no potential to reduce stigma. We also excluded clinical mental ill health education interventions directed at health or social care professionals.
Types of outcome measures
We did not exclude studies for failing to use validated outcome measures. However, we reported any validation of outcome measures.
The main outcomes (reported at Quality of the evidence) were: discrimination towards people with mental ill health; prejudice towards people with mental ill health; cost and unforeseen adverse effects.
Primary outcomes
There were two primary outcomes: (i) discrimination and (ii) prejudice; reflecting two of the three elements in Thornicroft’s conceptualisation of stigma (Thornicroft 2007). Discrimination, being a behavioural outcome, is more difficult to measure, but is of greater importance than prejudice for improving the lives of people with mental health problems; consequently data for this outcome are presented before those for prejudice outcomes. For the same reason, in our assessment of the strength of evidence (Quality of the evidence) we defined discrimination as a ‘critically important’ outcome and prejudice as an ‘important’ one.
To have been eligible for inclusion, a study must have included at least one of the discrimination or prejudice outcome measures.
-
Discrimination towards people with mental ill health, including: reports of discrimination personally experienced by people with mental ill health; observed discriminatory behaviour towards people with mental ill health, such as avoidance and negative interaction observed in experimental settings; and reported behavioural discrimination towards people with mental ill health. We had originally planned to include discriminatory behavioural intentions under discrimination, but elected to report these as under prejudice as an intention is more akin to an attitude than a behaviour.
-
Prejudice towards people with mental ill health, including: attitudes towards people with mental ill health; stereotyping of people with mental ill health; desire for social distance from people with mental ill health; emotional responses towards people with mental ill health; empathy for people with mental ill health; and implicit associations regarding people with mental ill health.
Secondary outcomes
The secondary outcomes were:
-
knowledge (any type);
-
cost of the mass media and comparator interventions (cost charged, or cost incurred if cost charged data were unavailable, in pounds sterling);
-
reach, recall, and awareness of intervention(s);
-
duration/sustainability of media effects;
-
audience reactions to media content (generally and by specific groups within sample e.g. favourability and information/message communicated); and
-
unforeseen adverse effects (other than increases in discrimination and prejudice).
Search methods for identification of studies
Electronic searches
We searched eleven electronic databases, each from its earliest date.
-
Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library, Issue 7, 2011) (Appendix 1)
-
MEDLINE (OvidSP),1966 to 15 August 2011 (Appendix 2)
-
EMBASE (OvidSP),1947 to 15 August 2011 (Appendix 2)
-
PsycINFO (OvidSP), 1806 to 15 August 2011 (Appendix 3)
-
CINAHL (EBSCOhost) (nursing and allied health database) 1981 to 16 August 2011 (Appendix 4)
-
ERIC (CSA) (educational database), 1966 to 16 August 2011 (Appendix 5)
-
Social Science Citation Index (ISI), 1956 to 16 August 2011 (Appendix 6)
-
OpenSIGLE (http://www.opengrey.eu/) (grey literature),1980 to 2005 (latest date of database, searched 18 August 2012) (Appendix 7)
-
Worldcat Dissertations and Theses (OCLC), 1978 to 18 August 2011 (Appendix 8)
-
metaRegister of Controlled Trials (http://www.controlled‐trials.com/mrct/mrct_about.asp), 1973 to 18 August 2011 (Appendix 9)
-
Ichushi (Japanese medical database) (OCLC), 1903 to 11 November 2011 (Search strategies are available on request from [email protected]).
A MEDLINE search strategy was developed (see Appendix 2) and tailored to the other databases. There were no language restrictions.
Peer review procedures raised concerns about the translation of searches including:
-
the lack of exploded MeSH terms in the CENTRAL strategy; and
-
limited translation of the MEDLINE strategy to EMBASE, PsycINFO and CINAHL databases.
The impact of these deficiencies is difficult to ascertain. We hope that by searching other resources (see below) we have mitigated the risk of having missed relevant studies. We welcome contact from any authors who believe their studies may be relevant to this review.
Searching other resources
Other search methods included: searching abstracts of World Psychiatric Association Stigma Conferences; reference checking of included studies and reviews; personal communication with experts in the field, including stigma researchers and media scientists; searching websites of governmental and non‐governmental organisations known to be running anti‐stigma campaigns in mental ill health; and citation forward checking from included studies using the Science Citation Index and the Social Science Citation Index via the Web of Science database.
Data collection and analysis
Selection of studies
Two authors independently screened the initial 20% of items. If agreement on whether to exclude studies between the two authors was greater than 95%, we planned for one author to screen the remaining references. This level of agreement was exceeded, so we proceeded in this way. Full papers were ordered for all items identified as potentially relevant by at least one author. Full papers were also ordered when there was insufficient information from the title and/or abstract to indicate possible relevance.
Two authors independently considered whether each full paper obtained met the inclusion criteria and disparities in inclusion decisions were resolved through discussion, with arbitration by a third author where necessary. Review authors did not contribute to inclusion decisions regarding studies in which they had been involved.
Data extraction and management
We used Endnote software to store and manage all located studies. We extracted data into data extraction tables, based on the CCCRG Data Extraction Template. The table format was piloted before use. The format is as follows:
Methodological details of study: aim of study; study design; details of cluster RCTs (number of clusters, size of each cluster, description of the clusters and the intra class correlation coefficient); details of ITS studies (number of time points, the length of time between points, the exact dates and duration of the intervention and the method of statistical analysis used); methods of recruiting participants; inclusion/exclusion criteria for participation; funding; statistical methods; power calculation; and consumer involvement in study design or intervention.
Assessment of risk of bias: Using standard tools (as detailed at Assessment of risk of bias in included studies).
Participants: description of sample measured; geographic location; setting; number; age; gender; ethnicity; and income level of participants' country (World Bank Index A, B or C).
Details of intervention: aim of intervention; content of intervention; type(s) of mass media used; number of mass media components; whether mass media component was combined with non‐mass media components; group(s) targeted by intervention; whether intervention involved personal narratives; whether celebrities were included; whether it was a fictional portrayal of mental ill health; type of message(s) in intervention (based on categories in Clement 2010); mental health condition(s) addressed; intervention providers (who designed the intervention, who funded it, who oversaw its delivery).
Details of control condition(s).
Details of co‐interventions in all groups (non‐mass media elements in interventions).
Delivery of intervention: stages, timing, frequency, duration (specifically and whether < 3 months or 3+ months), reach, recall, awareness.
Intervention quality and fidelity: whether intervention had a theoretical basis and details of theoretical basis; formative research undertaken in the development of intervention; evidence‐base for intervention; whether intervention was delivered as intended; quality information assessed by study authors, others, review team.
Outcomes: primary and secondary outcome measures (as identified by study authors); any validation of outcome measures; methods of assessing outcomes (e.g. phone survey); methods of follow‐up of non‐respondents; timing of outcome assessment (frequency and duration); adverse events.
Notes: contact with authors; if study was translated; if a duplicate publication; and other information.
Results (numerical data): effect estimates, standard errors (these may be calculated from other presented statistics). See also Measures of treatment effect.
Data were extracted independently by two authors. Disparities were resolved through discussion, with arbitration with a third author where necessary. Review authors did not contribute to data extraction of any studies in which they were involved.
We contacted study authors for further information when data relating to any of the fields in the data extraction table were missing.
Assessment of risk of bias in included studies
For RCTs we used the Cochrane Collaboration's 'Risk of bias' tool (Higgins 2011, section 8.5). For ITS studies we used the Cochrane 'Risk of bias' tool for ITS studies adapted using EPOC's criteria for ITS studies and input from the CCCRG (Ryan 2011). Two authors independently assessed the risk of bias. For 'Other sources of bias' we specifically considered: lack of evidence for reliability or validity of primary outcomes; for cluster trials with few clusters, whether there was a difference in baseline measures or participant characteristics; any evidence of counter‐discourse (e.g. high‐profile violence by a person with mental illness during study period); outcomes between audience members and non‐audience members of the media content not compared where relevant (e.g. in population‐level studies); and the risks of bias from other sources as listed in Higgins 2011 (section 8.15.1).
We had not specified methods for classifying levels of risk for outcome measures in our protocol, therefore, through discussion (SC, EB, SEL and FL) we established the following pragmatic criteria. We rated as ' high risk' measures developed by the study authors with no psychometric data reported; measures for which the authors reported a Cronbach's alpha of < 0.7; and un‐referenced measures. We rated as 'unclear' referenced measures with no psychometric data reported; referenced measures with no statement that the measure was reliable or valid; and validated measures being used for the first time in a different type of population. We rated as 'low risk' measures which study authors reported had a Cronbach's alpha of 0.7 or greater, or referenced the measure as being reliable or valid. Disparities in any 'Risk of bias' ratings were resolved through discussion, with arbitration with a third author where necessary, although in the event no arbitration was needed. We planned that review authors would not contribute to 'Risk of bias' assessment for any studies in which they were involved, but this situation did not arise. We incorporated the results of the 'Risk of bias' assessment into the review through narrative description about each of the 'Risk of bias' items, leading to an overall assessment of the risk of bias in the included studies. Studies considered at high risk of bias were removed as part of a sensitivity analysis (see Sensitivity analysis).
Measures of treatment effect
Subject to data availability, for RCTs with continuous outcome measures we had planned to report the mean differences with 95% confidence intervals (CIs), and for dichotomous outcome measures to calculate odds ratios with 95% CIs. However, as the same outcome was generally measured by different scales, we calculated standardised mean differences (SMDs) instead of mean differences.
In cluster RCTs, when the cluster size, number of clusters and the intra‐class correlation coefficient (ICC) (or estimate equivalent) could be successfully obtained for a study, we inflated the variances for clustering.
For ITS studies we planned to proceed as follows: where the risk of bias for all criteria was low, the study authors’ results would be used. If any ITS study failed to meet this criterion, raw data would be requested for reanalysis using autoregressive interrupted moving average (ARIMA) models as suggested in Ramsay (Ramsay 2003) when there are a large number of time points; otherwise by using time series regression as suggested by Grilli (Grilli 2002). When ARIMA models were used, we would obtain both point estimates and change in slope estimates for each study, as both of these are important in the interpretation of the intervention effect. When time series regression was used, regression coefficients would be used to measure intervention effects. In the event no ITS studies were included in the review.
We did not pre‐specify actions if data were skewed data. When this was the case the data were transformed into the logarithmic scale using methods described by Higgins and colleagues (Higgins 2008).
Unit of analysis issues
In cluster trials, where reported we used effect estimates and standard errors that were adjusted in the analysis for clustering, and combined the studies using the generic inverse variance method. If the analysis did not take account of clustering, we approximated the cluster adjusted effect size and standard error based on available data if the unadjusted effect estimate, the number or size of clusters and the ICC were provided. If the ICC could not be obtained then we used an estimate from similar studies.
In cross‐over trials, we planned to use the effect estimate and standard deviation based on a paired t‐test, and combine the studies using the generic inverse variance method (Higgins 2011, section 16.3). However, no appropriate cross‐over trials were identified (see Included studies).
If studies had more than two groups we combined all relevant experimental intervention groups of the study into a single group, and combined all relevant control intervention groups into a single control group (Higgins 2011, section 16.5.4). Where intervention arms fell into different subgroups each intervention arm was compared to the control group, and the possibility of meta‐analysis was only considered within each subgroup, thereby avoiding potential unit of analysis errors. Data from irrelevant intervention groups were ignored (e.g. live presentations). We had not anticipated studies having two control groups, so we made a post‐hoc decision to select the control group most similar to the intervention.
Dealing with missing data
We contacted study authors where any data were missing. Where studies did not state that results were reported using an intention‐to‐treat analysis for primary outcomes, we contacted study authors to request data to enable us to conduct such an analysis, and in the event of non‐response we analysed results as reported.
When there were missing summary data in a study, we contacted authors and asked them to provide the required summary data, or failing that, any data to derive the required summary data. If authors were unable to provide this, we attempted to derive the specific data from other reported statistics in the study. If we could not obtain such data, we analysed the particular study narratively.
Assessment of heterogeneity
Statistical measures of heterogeneity were ascertained visually, and using the Cochrane's Q and the I2 statistic, with I2 > 50% representing substantial heterogeneity (Higgins 2011, section 9.5.2). We also considered the clinical heterogeneity of the studies (for example in participants, interventions and outcomes) and methodological heterogeneity (such as in the quality of the studies, and in study design).
Assessment of reporting biases
Where we found at least 10 studies and an appropriate range of sample sizes, we assessed the possibility of reporting bias using funnel plots to examine the relationship between studies' risk of bias and effect size estimates. This was quantified using Egger's test of symmetry. Where reporting bias was identified, we investigated the impact in a sensitivity analysis.
Data synthesis
Whether a narrative synthesis or meta‐analysis was conducted, we planned to produce a 'Summary of findings' table from the included studies for each type of study design (i.e. RCT and ITS) using GRADEprofiler (GRADEpro) software. In the event the 'Summary of findings' table was produced using the template in RevMan, but still following the GRADE approach (Guyatt 2008).
For RCTs, for each comparison (mass media intervention versus control) we reported tables of summary statistics for each of the included studies. For each primary and secondary outcome, we reported outcome measure, follow‐up summary statistics, and effect estimates and their statistical significance. In the protocol (Clement 2011) we stated that we would also report baseline summary statistics, but this is no longer applicable now that SMDs rather than mean differences are being reported (see Measures of treatment effect). We also reported our assessment of risk of bias. For cluster randomised trials we noted whether there were unit of analysis issues. We also reported details concerning potential effect moderators (as specified under Subgroup analysis and investigation of heterogeneity) e.g. nature of the intervention, content of the intervention, type of media.
For ITS studies we had planned to follow the approach outlined by Brennan (Brennan 2009), and present results from these studies in tables for each comparison with summary statistics for each of the included studies, change in level of the outcome at the first point after the introduction of the intervention, post‐intervention slope minus the pre‐intervention slope, and information on effect modifiers. We would have also presented this graphically using, for example, scatter plots of change in level versus change in slope with combinations of statistical significance denoted by different symbols. In the event, we found no ITS studies.
In a narrative synthesis, for each comparison (e.g. mass media intervention versus control) we stated: the number of comparisons showing a positive direction of effect; the median effect size across all comparisons; the median effect size across comparisons without unit of analysis errors; and the number of comparisons showing statistically significant effects. This approach was recommended by Grimshaw 2003 as it “allows the reader to assess the likely effect size and consistency of effects across all included studies and whether these effects differ between studies, with and without unit of analysis errors”.
In the narrative synthesis and in any statistical synthesis, we synthesised first according to the different types of interventions (grouping similar interventions together), second according to the types of outcomes (with discrimination outcomes reported first, then prejudice outcomes, then secondary outcomes), and third according to the strength of evidence.
Preliminary scoping of the field indicated considerable heterogeneity in the types of intervention, participants and outcome measures, therefore we anticipated that we would be unlikely to find sufficient homogeneity to warrant meta‐analysis. However a review author group discussion (originally planned as a face‐to‐face meeting but altered to an email discussion, given the disparate locations of the authors) took place to judge the appropriateness of meta‐analysis in the light of the heterogeneity assessments. For any meta‐analysis undertaken we used a random‐effects model, as planned, as we had predicted there would be a high level of heterogeneity across the studies.
In the event of multiple outcomes reported in a study, the outcome selected for analysis was the primary outcome as defined by the authors of that particular study. If there was no specified primary outcome, or if a specific primary outcome could not be deduced from the study, we chose the outcome from which the power equation for the study was provided. In the case where this was not reported, we chose the outcome which had the median reported effect size (Grimshaw 2003). Where there was an even number of outcomes, we made a post‐hoc decision that, following Brennan 2009, we would select the outcome with the n/2 ranked effect size (using data from the final follow‐up point when there were two or more follow‐up points). A post‐hoc decision was also needed about which outcome to select when multiple outcomes were used in studies with median data. In these cases an adapted version of the methods proposed by Brennan 2009 was used whereby, after checking that the interquartile ranges were similar, we examined medians at the latest time point and selected the one ranked (n+1)/2 when there was an odd number of outcomes and the one ranked n/2 when there was an even number.
Subgroup analysis and investigation of heterogeneity
We planned to undertake the following subgroup analyses (by narrative methods and also by meta‐analysis if appropriate) to explore possible explanations for observed heterogeneity:
-
Short‐term interventions (up to three months) versus long‐term interventions (three months or longer). This was not undertaken as all included studies had short‐term interventions.
-
Studies in high‐income countries (band A, World Bank Index) versus middle‐/low‐income (band B and C) countries.
We intended to conduct the following comparisons:
Comparisons relating to nature of the intervention:
-
Interventions with one mass media component versus those with two or more mass media components.
-
Interventions in which the mass media component(s) was combined with non‐mass media components, versus interventions with a mass media component only.
Comparisons relating to the content of the interventions:
-
Interventions involving personal narratives (indirect 'social contact') versus those not involving personal narratives. In the event a post‐hoc decision was made to use three subgroups here: first‐person narratives, third‐person narratives and no narratives, as we had not anticipated the middle group.
-
Interventions with the primary message being biomedical, psychosocial, recovery‐oriented, 'see the person', high prevalence of mental disorders, anti‐dangerousness, valuing difference, social inclusion/human rights, continuum or negative impact of mental illness (Clement 2010). A post‐hoc decision was made not to use the 'see the person' message type as this message type only arose when interventions contained personal narratives and if we had categorised these as having a 'see the person' primary message, we would have missed messages contained in what the narrators said (or other aspects of the intervention). We also decided post‐hoc to include commonly‐used categories of primary message that were not in Clement 2010. We had not pre‐specified the method of deciding which message was primary, and decided this would be undertaken independently by two authors who would resolve disparities by discussion, and with arbitration if necessary.
-
Interventions that included personal narratives by celebrities versus interventions that included personal narratives and included no celebrities.
-
Interventions that included fictional narratives versus interventions with non‐fictional narratives.
Comparisons relating to the type of media:
-
Interventions that used broadcast media (television, radio) versus print media versus cinema/recordings versus Internet/mobile phone versus other media. As no mobile phone, broadcast media or cinema interventions were found, we did not refer to these and they did not appear in the type of media subgroup analysis. There was just one intervention ‐ a CD‐ROM ‐ that fell in the 'other' category for media type, and we decided to group this in the Internet category, as Internet‐delivery would not have materially changed participants' experience of the intervention.
Sensitivity analysis
Where meta‐analysis was possible, we conducted sensitivity analysis to examine the effects of excluding studies at higher risk of bias. If bias was discovered we used two methods as a sensitivity analysis:
-
removed the less precise studies, and
-
used the ‘trim and fill’ method. This method was only performed on the prejudice (immediate) outcome, for which there were more than 10 studies.
We had intended to test for small study effects of binary outcomes by performing the arcsine‐Thompson test, as this has been shown to perform well in simulations and it allows for substantial between‐study heterogeneity (Rücker 2008). However this was precluded because we found only two studies with binary primary outcomes (Yoshida 2002; Penn 2003, discrimination outcome) and these had very different timings of outcome (immediate and 9 months), rendering this test inappropriate.
We included a sensitivity check of a fixed‐effect model. A sensitivity analysis for plausible variations in estimated ICCs was performed when unit of analysis errors arose in cluster randomised trials and the ICCs were estimated for these studies from studies of similar populations, that is from university students (Campbell 2011).
As we found that three of the multi‐arm studies included arms that the study authors considered unlikely to reduce stigma (Reinke 2004; Brown 2010; Corrigan (submitted)), we undertook a post‐hoc sensitivity analysis to examine the effects of removing these studies.
Stakeholder participation
One of the authors of the review uses mental health services, has experience of mental health‐related stigma, and has close family members who have used mental health services, and draws on these perspectives in this review.
A consultation group was set up to provide additional relevant perspectives. The role of the consultation group members was to comment and provide feedback on the draft protocol, draft review and plain language summary. Members of this group who have worked in anti‐stigma campaigning/research were also included in the request for additional studies that may meet the inclusion criteria at the search stage. Those with personal experience of mental ill health were asked to give their view on the importance of the degree of change in stigma found. The group included the following members: a researcher from the Service User Research Enterprise, Institute of Psychiatry, Kings College London; the Deputy Director of Knowledge and Learning, Rethink (charity for people affected by severe mental illness); a medical doctor; an advertising executive; the Service User Lead for an organisation working to reduce mental health‐related stigma and discrimination, focusing particularly on employment; and the ex co‐chair of a mental health service user advocacy organisation. Four of these members were also stigma researchers.
The plain language summary was written by the review author with experience of mental health service use in collaboration with a person who has used mental health services but who is not involved in research.
In addition, the standard peer review process of the Cochrane Consumers and Communication Review Group includes review of the protocol and review by at least one consumer.
Results
Description of studies
See: Characteristics of included studies; Characteristics of excluded studies. '(a)' indicates information provided by study authors.
Results of the search
See Figure 1. The database searches yielded 22,895 records, as follows:
Study flow diagram.
-
CENTRAL (315);
-
MEDLINE (3303);
-
EMBASE (9530);
-
PsycINFO (1803);
-
CINAHL (401);
-
ERIC (1782);
-
Social Science Citation Index (3663);
-
OpenSIGLE (46);
-
Worldcat Dissertations and Theses (80);
-
metaRegister of Controlled Trials (0); and
-
Ichushi (1972).
We found a further 24 records from:
-
reference checking of reviews (19);
-
conference abstracts of World Psychiatric Association Stigma Conferences (1);
-
citation checking from included studies (3); and
-
communication with experts (1).
No records were identified from website checking, reference checking of included studies or communication with authors of included studies.
Two authors independently screened 20% of the 19,031 non‐duplicate records and achieved an agreement level of 99.9%. Consequently a single author screened the remainder of the records as per our protocol (Clement 2011). We deemed 131 records to be potentially relevant, and for each of these full‐text articles were obtained and assessed independently for inclusion by two authors. Disparities were resolved through discussion and 108 papers were excluded. We initially identified 23 studies (24 papers) as eligible for inclusion in the review, but one was excluded after contact with the author (Bayar 2009).
Included studies
We included 22 studies in the review. Three studies contained no analysable outcome data (Dias‐Vieira 2005; Han 2006; Varughese 2010), therefore 19 studies contribute to the qualitative synthesis. Meta‐analysis was appropriate for three studies and for one outcome only (see Effects of interventions). The 22 studies had a total of 4490 participants randomised to relevant study arms (median = 150, range = 46 to 739). For the 19 studies with analysable outcome data, data were available for a total of 1196 participants (five studies) for discrimination outcomes, 3176 (19 studies) for prejudice outcomes, 1213 (4 studies) for knowledge outcomes, 381 (3 studies) for audience reactions to the interventions, 416 (3 studies) for cost data, 727 (1 study) for awareness data, 1225 (5 studies) for duration of effects data, and 455 (2 studies) for statements about adverse effects.
We sought contact with authors of all included studies. Authors of nine studies responded with additional information, with one conducting additional analyses (Jorm 2010a) and one providing the data set (Yoshida 2002).
Study designs
The 22 included studies had a high level of clinical and methodological heterogeneity (described in this section), and were also statistically heterogeneous (see Effects of interventions). All included studies were randomised controlled trials, including three cluster trials (Yoshida 2002; Penn 2003; Coleman 2005) and one had a cross‐over design (Varughese 2010). In none of the cluster trials was there any adjustment for the design effect, although one study author (Yoshida 2002) provided his data set and the review team statistician (PW) analysed this accounting for clustering. Because there is no established 'wash‐out' period for stigma interventions, we concluded that the cross‐over design was inappropriate and contacted the author to request summary statistics for a post‐intervention comparison of those allocated to receive the intervention and control initially. As no reply was received, this study (Varughese 2010) had no analysable data, as did two other studies (Dias‐Vieira 2005; Han 2006).
Ten of the studies were multi‐arm studies having between two or more mass media intervention groups. Eleven of the studies had some level of involvement from people with mental health problems in the design of the study or intervention, although this was often indirect. Four studies had such a person as a study author and in each of these the person was also involved in developing the intervention itself (Reinke 2004; Kerby 2008; Matthews 2009; Jorm 2010a). A further three studies used or adapted interventions which had been developed with consumer involvement (Russell 1988; Bunn 2009; Brown 2010). Three used interventions endorsed by a consumer organisation (Woods 2002; Woods 2003; Woods 2005) and one drew upon messages endorsed by a consumer organisation in developing the intervention (Smith 2007). Eight of the studies were unpublished (seven were dissertations and one was a paper in submission). Twenty‐one studies were reported in English and one in Japanese (Yoshida 2002).
Settings and populations
The studies took place mainly in the United States (n = 15), or other high‐income countries, including two studies in the United Kingdom (Kerby 2008; Varughese 2010), and one study each in Canada (Matthews 2009), Australia (Jorm 2010a), Japan (Yoshida 2002) and Taiwan (Han 2006). One study took place in Russia (Finkelstein 2008), a country classified as 'upper middle income'. The majority (n = 13) of studies had samples comprised of college or university students studying psychology (usually introductory) or a variety of courses. Other studies involved students on professional courses: medical students (Kerby 2008; Bunn 2009); nursing students (Coleman 2005) and special education students (Finkelstein 2008). Four of the studies had general population samples (Yoshida 2002; Jorm 2010a; Varughese 2010; Corrigan (submitted)) and one involved employers (Russell 1988).
Intervention and control conditions
The interventions were all short‐term (defined a priori as up to three months). The majority (n = 16) involved a single presentation of a single‐component mass media intervention. However, in two studies participants could complete the multi‐component educational materials interventions over a number of sittings in up to four weeks (Finkelstein 2008; Jorm 2010a); in one study participants received three pamphlets mailed weekly (Russell 1988); in another they received material by email sent over four weeks (Smith 2007); in another they were handed three booklets (Yoshida 2002); and in one watched two films at one sitting (Kerby 2008). No interventions combined two or more types of mass media. In two studies the mass media component was combined with a non‐mass media component, that is a class discussion (Reinke 2004), or a meeting with people with mental health problems (Yoshida 2002). None of the interventions used broadcast media (television, radio); none were delivered through public cinemas, and no mobile phone interventions were found in the eligible studies. The interventions therefore fell into three of the predefined categories: Internet, recordings and print.
Internet interventions included an Internet‐delivered interactive computer education programme (Finkelstein 2008); web‐delivered newspaper articles (Corrigan (submitted); and educational adverts delivered by email (Matthews 2009). An educational CD‐ROM (Jorm 2010a) was classified as Internet, as this was the category it most closely fitted. Post‐hoc, we subdivided the recordings into 'audiovisual' and 'audio' because the audio recordings were very different from the audiovisual ones. The former were all DVDs or videos containing personal narratives (n = 7), educational material (Morgan Owusu 2002), a combination of the two, or a public service announcement (Demyan 2009). The latter were recordings of simulated hallucinations delivered through headphones. The print interventions included educational manuals (Finkelstein 2008; Jorm 2010a); a magazine article (Matthews 2009); brochures (Russell 1988; Yoshida 2002; Dias‐Vieira 2005); written text (Han 2006; Iobst 2008); and a photograph (Varughese 2010). Conditions covered were: schizophrenia/psychosis/serious mental illness (n = 8), a combination of conditions (n = 5), depression (Dias‐Vieira 2005; Han 2006); exam stress (Demyan 2009); Tourette's syndrome (Woods 2002; Woods 2003; Woods 2005); intellectual disability (Russell 1988; Varughese 2010); and autism (Iobst 2008). Reduction in stigma was not the main objective of all the interventions, for example some aimed to encourage healthcare‐seeking (Morgan Owusu 2002; Demyan 2009) and some had a primary educational aim (Coleman 2005; Jorm 2010a). Three studies included one intervention arm they believed was unlikely to reduce stigma (Reinke 2004; Brown 2010; Corrigan (submitted)).
The most common types of primary message included in the studies were recovery‐oriented messages (n = 7) and 'not to blame' messages (n = 4). Four studies had multiple primary messages. Intervention arms also included primary messages of the following types: biomedical (Coleman 2005; Han 2006; Iobst 2008), social inclusion (Russell 1988), 'seek professional care' (Morgan Owusu 2002; Demyan 2009; Jorm 2010a), 'negative impact of mental illness' (Reinke 2004; Corrigan (submitted), and 'this is hallucinatory experience' (Bunn 2009; Brown 2010). Some interventions had been subject to extensive developmental work (e.g. Jorm 2010a) and others were based on extensive theoretical considerations (e.g. Finkelstein 2008), whereas others were developed for the study (e.g. Smith 2007). For 12 studies the control was no intervention, 1 had a waiting list control (Jorm 2010a), and 9 used the same mass media containing material unrelated to mental health problems or stigma (e.g. a film about wildlife). One study (Penn 2003) had both a 'no intervention' and an irrelevant material control group, and for this study we used the latter as it was more closely matched and controlled better for non‐specific effects.
Outcomes
In the majority of studies outcomes were measured immediately post‐intervention only (n = 14); four had final follow‐up at 1 to 2 weeks, two at 1 to 2 months (Kerby 2008; Jorm 2010a); one at 6 months (Finkelstein 2008); and one at 9 months (Yoshida 2002). Discrimination outcomes were measured in only 5 of the 22 studies. Measures included the distance between chairs arranged by participants anticipating meeting a person with Tourette's syndrome (Woods 2002; Woods 2003; Woods 2005); written expression of interest in attending a focus group with people with schizophrenia and providing contact details (Penn 2003); and reported visits to community mental health facilities during the follow‐up period (Yoshida 2002). All studies measured prejudice outcomes with the majority (n = 12) using multiple measures for this outcome. In total 28 different measures were used, with the Social Distance Scale (Link 1987) being the most common, and used in 6 of the studies. Data on secondary outcomes were sparse, with four studies measuring knowledge (each of these assessed knowledge of mainstream models of mental illness), four measuring audience reactions to the interventions, one assessing awareness of the intervention (Yoshida 2002), and none assessing reach or recall. No studies assessed cost effectiveness or reported cost data, however two authors were able to provide some specific information about the costs of their interventions (Demyan 2009; Jorm 2010a).
Excluded studies
See Characteristics of excluded studies. The main reasons why the 108 potentially‐relevant studies were excluded were:
-
not having an RCT or interrupted time series design (n = 38);
-
there being no inactive comparator (n = 27);
-
the intervention not being mass media (n = 19);
-
the mass media component not comprising more than 50% of the intervention (n = 10); and
-
there being no discrimination or prejudice outcome (n = 9).
One study (Bayar 2009) was initially included, however when we requested further details about randomisation methods the author informed us that alternation had been used, so we excluded this study as it did not meet the criteria for being an RCT. We located one simulated hallucinations intervention study (Kalyanaraman 2010) other than Bunn 2009 and Brown 2010, but this used a hand‐held headset to deliver both audio and visual simulated hallucinations and was excluded as, unlike headphone‐delivered interventions, it does not currently have the potential to be scaled up for use at the population level.
Risk of bias in included studies
For detection bias all studies were considered as being at high risk, because they had at least one primary outcome that was assessed by self‐report. For all other types of bias, a minority of studies were at low risk of bias. Selective reporting, incomplete outcome data and lack of participant blinding were the types of bias with fewest low risk studies, each including some high risk studies. High risk studies were also found for allocation concealment and other forms of bias (mainly relating to lack of validity of outcome measures). No studies had no high risk of bias ratings. See: Characteristics of included studies, Figure 2 and Figure 3.
Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
Risk of bias summary: review authors' judgements about each risk of bias item for each included study.
Allocation
Thirteen of the 22 studies had an unclear risk of bias in relation to random sequence generation, generally because the particular method of generation was unspecified preventing us from being sure that the allocation sequence was truly random. The same number had an unclear risk for allocation concealment and a further two (Smith 2007; Brown 2010) were at high risk, both on the basis of further information provided by the authors. Only 6 of the 22 studies were at low risk on both indicators of selection bias.
Blinding
With mass media interventions participants will always know that they are receiving some sort of intervention and it will often be clear to them that they are experiencing something intended to alter their views or behaviour. Consequently the vast majority of studies (18/22) were at high risk for non‐blinding of participants. However four studies (Morgan Owusu 2002; Dias‐Vieira 2005; Demyan 2009; Matthews 2009) used cover stories to help blind participants by informing them that the purpose of the research was to rate or improve extraneous features of the mass media material (e.g. music, length), sometimes embedding the intervention and control within a battery of materials to further blind the participants (Demyan 2009; Matthews 2009). For the five studies with discrimination outcomes, two were at high risk of detection bias as measures were assessed by written self‐report (Yoshida 2002; Penn 2003), however three had an unclear risk of detection bias because these involved investigator measurements of the distance between two chairs and it is not stated whether or not the researcher was aware of group allocation when making this measurement (Woods 2002; Woods 2003; Woods 2005). All prejudice outcomes and the secondary outcomes were assessed by self‐completed questionnaire and therefore were considered to be at high risk of detection bias.
Incomplete outcome data
Fourteen studies had an unclear risk of attrition bias and two were considered to be at high risk. In Woods 2002 there were significant incomplete data for the discrimination measure as this was only assessed on the second half of the sample. This was also the case in Smith 2007 which had a post‐randomisation exclusion of, and no data available for, those who had previous contact with mental health services or had someone in their immediate family who had had such contact (n = 71). Only a minority (5/22) of studies were at low risk of attrition bias.
Selective reporting
For the majority of studies (17/22) there was an unclear risk of selective reporting as no study protocol or similar material was available to enable us to check that all variables measured had been reported. Two studies were deemed to be at high risk of selective reporting bias (Han 2006; Kerby 2008) as data for some measures were only partially reported. Three studies were at low risk because: their protocol appeared on a trials register (Jorm 2010a); the author provided the data set (Yoshida 2002); or the author provided a research ethics board application (Matthews 2009), and all measures in these materials were reported.
The group of studies including at least 10 studies and an appropriate range of sample sizes was the non‐clustered RCT studies with prejudice outcomes and immediate follow‐up. A funnel plot was created which was relatively symmetrical and therefore provided no evidence of publication bias. Likewise, Egger’s test of symmetry was not significant (P value = 0.444) providing no evidence of small study effects.
Other potential sources of bias
The most common potential other source of bias in the included studies was lack of reported evidence for the reliability and validity of outcome measures: two studies were rated as high risk for this (Morgan Owusu 2002; Kerby 2008), with an unclear risk in a further ten studies. Other high risks of bias came from baseline imbalance (Yoshida 2002; Reinke 2004; Coleman 2005); and evidence of counter‐discourse during the follow‐up period through the occurrence of serious crimes committed by people reported to have mental illness during the study period, which may have had a differential effect on those in the two study arms (Yoshida 2002). Finally two studies had an unclear risk of bias due to failure to adjust for study design effects in the analysis of cluster trials (Penn 2003; Coleman 2005).
Effects of interventions
See: Summary of findings for the main comparison
See summary of findings Table for the main comparison.
Primary outcomes
Narrative synthesis of effects of interventions
Discrimination
Main comparison
Five studies (n = 1196) assessed discrimination outcomes. Three studies had continuous outcomes, all of which were the distance placed between two chairs when a participant was anticipating a meeting with a person with Tourette's syndrome (Woods 2002; Woods 2003; Woods 2005, see Figure 4). One study reported a reduction in discriminatory behaviour (Woods 2002) with a standardised mean difference (SMD) of ‐0.85 (95% confidence interval (CI) ‐1.39 to ‐0.31), however the two other studies by the same author (Woods 2003; Woods 2005) involving the same type of population and methodology but with larger sample sizes, both failed to provide evidence of an effect of the intervention on discrimination: SMDs ‐0.25 (95%CI ‐0.51 to 0.02); ‐0.17 (95%CI ‐0.53 to 0.20). Meta‐analysis of these three studies was precluded due to statistical heterogeneity (I2 = 0.57, see meta‐analysis section). Two studies had a dichotomous discrimination outcome: failure to register to attend a focus group which was to include people with schizophrenia (Penn 2003), and self‐report of not visiting sheltered workshops for people with mental illness during the follow‐up period (Yoshida 2002). These studies provided no evidence of effect: odds ratios (OR) 1.30 (95% CI 0.53 to 3.19); and 1.19 (95% CI 0.85 to 1.65). See Figure 5. Here meta‐analysis was inappropriate due to clinical and methodological heterogeneity (see meta‐analysis section).
Forest plot of comparison: 1 Mass media vs. control (main comparison), outcome: 1.1 Discrimination (Immediate).
Forest plot of comparison: 1 Mass media vs. control (main comparison): outcome 1.2 Discrimination.
We constructed a narrative synthesis table following Grimshaw 2003 (first four columns of Table 1). We added a fifth column to aid the interpretation of the findings, and classified the effect sizes as 'small'; 'medium' or 'large' derived from the work of Cohen (Cohen 1988). Further interpretation using transformation into a common scale is given in Summary of main results. Examination of Table 1 (top section, main comparison) indicates that overall there is no evidence to support or refute mass media interventions being able to reduce discrimination.
| Comparison | No. of comparisons with positive direct of effect1 | Median effect size across all comparisons2 | Median effect size across all comparisons without unit of analysis errors3 | No. of comparisons showing statistically significant effects4 | Interpretation of median effect size across all comparisons5 |
| Mass media vs. control | |||||
| Immediate follow‐up | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| 6 to 9 months follow‐up | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| Mass media vs. control by if 2+ mass media components | |||||
| 2+ mass media components | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| One mass media component | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| Mass media vs. control by presence of narratives | |||||
| First‐person narratives | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| Third‐person narratives | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| Mass media vs. control by type of primary message | |||||
| Recovery‐oriented | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| Not to blame | 3/3 | SMD = ‐0.25 | SMD = ‐0.25 | 1/3 | Small |
| Multiple | 0/1 | OR = 1.30 | ‐ | 0/1 | Negligible |
| Mass media vs. control by fictional narratives | |||||
| Fictional narratives | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| Non‐fictional narratives | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| Mass media vs. control by type of media | |||||
| Audiovisual | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
[1] Stigma‐reducing
[2] When there was an even number of effect sizes the median reported is the mean of the two middle values.
[3] This excludes cluster trials not adjusted for study design by the authors and without a study‐derived intra‐class correlation co‐efficient (Penn 2003)
[4] (+) means in stigma‐reducing direction, (‐) means in stigma increasing direction
[5] SMDs large ≥ 0.8; medium ≥ 0.5, small ≥ 0.2, negligible < 0.2; and for ORs large ≥ 4.3; medium ≥ 2.5, small ≥ 1.5, negligible < 1.5, based on Cohen’s (Cohen 1988) rule of thumb, and using a pragmatic decision about how to treat intermediate values
Subgroup comparisons
Several planned subgroup comparisons were not possible as all studies with discrimination outcomes were in the same subgroup. The findings from the subgroup analyses that were possible for this outcome are shown in Table 1. As in the main comparison, all subgroup effect sizes were found to be small/negligible apart from the 'not to blame' messages subgroup which had a small discrimination‐reducing effect size. See Data and analyses; Figure 4; Figure 5; Table 1.
Prejudice
Main comparison
All 19 studies with analysable data reported prejudice outcomes (n = 3176). The results of the main comparison can be seen in two forest plots (Figure 6 and Figure 7). In addition, for one study (Kerby 2008) with skewed outcomes, the medians (interquartile ranges) for the intervention and control groups post‐intervention were 13 ((10 to 15) n = 20) and 12 ((10 to 14) n = 22) respectively. At eight weeks follow‐up the data were 14 ((11 to 15) n = 20) and 12 ((10 to 14) n = 21) respectively. The study authors stated that there were no significant differences between the groups at any time point, but provided no statistical information. We constructed a narrative synthesis combining all these data for prejudice outcomes; see Table 2. Inspection of the forest plots (Figure 6 and Figure 7) and the top section of Table 2 indicates that there is evidence of benefit (reduction in prejudice) at all three follow‐up time points. For example, at immediate follow‐up the median effect size across all studies was SMD = ‐0.38. At 1 week to 2 months follow‐up, the all‐study median effect size was ‐0.38. The median effect size across all three studies with outcomes at 6 to 9 months was ‐0.49. Two studies (Russell 1988; Finkelstein 2008) stood out for having much larger prejudice‐reducing effect sizes: SMD ‐2.94 (95% CI ‐3.52 to ‐2.37) and ‐1.69 (95% CI ‐2.06 to ‐1.32) (see Figure 6), and one (Coleman 2005) for having a very large prejudice‐increasing effect (SMD 2.40, 95% CI 0.62 to 4.18) (see Figure 7). Only one meta‐analysis within one subgroup of studies (Finkelstein 2008; Jorm 2010a) ‐ non‐cluster trials with prejudice outcomes at 6 to 9 months ‐ had an I2 value indicating reasonable statistical homogeneity. Meta‐analysis was warranted (see meta‐analysis section) and these studies had a combined affect size of SMD =‐0.36 (95% CI ‐0.63 to ‐0.10) demonstrating a reduction in prejudice.
Forest plot of comparison: 1 Mass media vs. control (main comparison), outcome: 1.3 Prejudice.
Forest plot of comparison: 1 Mass media vs. control (main comparison), outcome: 1.4 Prejudice.
| Comparison | No. of comparisons with positive direct of effect1 | Median effect size across all comparisons2 | Median effect size across all comparisons without unit of analysis errors3 | No. of comparisons showing statistically significant effects4 | Interpretation of median effect size across all comparisons ( Cohen 1988)5 |
| Mass media vs. control | |||||
| Immediate follow‐up | 12/15 | ‐0.38 | ‐0.38 | 6/15 (+), 1/15 (‐) | Small |
| 1 to 8 weeks follow‐up | 3/5 | ‐0.38 | ‐0.38 | 2/5 (+) | Small |
| 6 to 9 months follow‐up | 3/3 | ‐0.49 | ‐0.49 | 1/3 (+) | Small‐to‐medium |
| Mass media vs. control by income of country | |||||
| Upper middle income | 1/1 | ‐1.69 | ‐1.69 | 1/1 | Large |
| High income | 14/18 | ‐0.42 | ‐0.42 | 7/18 (+) 1/18 (‐) | Small |
| Mass media vs. control by if 2+ mass media components | |||||
| 2+ mass media components | 4/6 | ‐0.49 | ‐0.49 | 3/6 (+) | Small‐to‐medium |
| One mass media component | 12/13 | ‐0.34 | ‐0.34 | 5/13 (+) 1/13 (‐) | Small |
| Mass media vs. control by if combined with non‐mass media | |||||
| With non‐mass media | 2/2 | ‐0.42 | ‐0.42 | 0/2 | Small |
| Mass media alone | 13/17 | ‐0.43 | ‐0.43 | 8/17 (+) 1/17 (‐) | Small |
| Mass media vs. control by presence of narratives | |||||
| First‐person narratives | 8/10 | ‐0.51 | ‐0.51 | 6/10 (+) 1/10 (‐) | Medium |
| Third‐person narratives | 4/5 | ‐0.03 | ‐0.03 | 2/5 (+) | Negligible |
| No narratives | 5/7 | ‐0.27 | ‐0.27 | 3/7 (+) 1/7 (‐) | Small |
| Mass media vs. control by celebrity narratives | |||||
| Celebrity narratives | 1/1 | ‐0.23 | ‐0.23 | 0/1 | Small |
| Non‐celebrity narratives | 11/15 | ‐0.48 | ‐0.44 | 7/15 (+) 1/15 (‐) | Small |
| Mass media vs. control by fictional narratives | |||||
| Fictional narratives | 2/3 | ‐0.42 | ‐0.42 | 1/3 (+) | Small |
| Non‐fictional narratives | 10/12 | ‐0.47 | ‐0.47 | 6/12 (+) 1/12 (‐) | Small |
| Mass media vs. control by type of message | |||||
| Biomedical | 1/2 | 0.99 | ‐0.42 | 1/2 (+) 1/2 (‐) | Large |
| Social inclusion/human rights | 1/1 | ‐2.94 | ‐2.94 | 1/1 (+) | Large |
| Recovery‐oriented | 5/5 | ‐0.30 | ‐0.30 | 1/5 (+) | Small |
| Negative impact of mental illness | 1/2 | ‐0.13 | ‐0.13 | 0/2 | Negligible |
| Seek professional care | 2/3 | ‐0.43 | ‐0.43 | 2/3 (+) | Small |
| Not to blame | 3/3 | ‐0.51 | ‐0.51 | 3/3 (+) | Medium |
| Hallucinatory experience | 1/2 | 0.22 | 0.22 | 1/2 (‐) | Small |
| Multiple messages | 3/4 | ‐0.70 | ‐0.71 | 1/4 (+) | Medium |
| Mass media vs. control by type of media | |||||
| Internet | 3/4 | ‐0.30 | ‐0.30 | 2/4 (+) | Small |
| Audiovisual | 7/10 | ‐0.47 | ‐0.47 | 5/10 (+) 1/10 (‐) | Small |
| Audio | 1/2 | 0.22 | 0.22 | 1/2 (‐) | Small |
| | 6/6 | ‐0.46 | ‐0.46 | 4/5 (+) | Small |
[1] Stigma‐reducing
[2] Excluding study with median and IQR data only (Kerby 2008). When there was an even number of effect sizes the median reported is the mean of the two middle values.
[3] This excludes cluster trials not adjusted for study design by the authors and without a study‐derived intra‐class correlation co‐efficient (Penn 2003; Coleman 2005)
[4] (+) means in stigma‐reducing direction, (‐) means in stigma increasing direction. Excluding study with median and IQR data only (Kerby 2008)
[5] SMDs large ≥ 0.8; medium ≥ 0.5, small ≥ 0.2, negligible < 0.2, based on Cohen’s (Cohen 1988) rule of thumb, and using a pragmatic decision about how to treat intermediate values
Subgroup analyses
See Data and analyses. We were unable to undertake the planned subgroup analysis of short‐term (up to three months) versus long‐term (three months or longer) interventions, as no intervention in the review lasted longer than four weeks. All the studies with analysable outcome data (n = 19) had a prejudice outcome and contributed to the narrative synthesis for prejudice (Table 2). Forest plots are not reported for the subgroup analyses as this would create an excessive number of figures, although all were created and examined and used to derive the data in Table 2.
All but one study were undertaken in high‐income countries and these had median SMD of ‐0.42, indicating a small effect size (Cohen 1988). The Kerby 2008 study is excluded from all the subgroup analyses as it used medians and therefore no SMD was calculable. The individual effect sizes for the studies contributing to the subgroup analyses can be seen in Figure 6 and Figure 7 unless otherwise stated. In contrast to the small effect in high‐income countries, a large effect size (SMD ‐1.69, 95% CI ‐2.06 to ‐1.32) was found in the one study undertaken in an upper‐middle income country (Russia) (Finkelstein 2008).
Five studies (Russell 1988; Yoshida 2002; Smith 2007; Finkelstein 2008; Jorm 2010a) had two or more mass media components. In each case the multiple components were within one media type (e.g. three printed booklets) rather than using two or more different types of media in one intervention. The multiple component interventions had a higher median effect size than those with one mass media component (median SMDs ‐0.49 and ‐0.34 respectively). Two studies combined mass media with non‐mass media elements, one by adding a 5‐minute facilitated discussion after a 10‐minute DVD presentation (Reinke 2004), and the other by inviting participants to attend a community event where they would meet people with mental illness and hear about their experiences, and receive information about mental health services (Yoshida 2002). These interventions had a similar small effect size to purely mass media interventions (median SMDs ‐0.42 and ‐0.43 respectively).
The nine interventions which included first‐person narratives and provided SMD data had a medium median effect size (‐0.51); whereas for five studies with third‐person narratives the median effect size was negligible at ‐0.03; and for the seven interventions with no narratives there was a small effect size (median SMD = ‐0.27). Data and studies contributing to these medians can be seen in Figure 8 for non‐cluster trials. For cluster trials the data can be seen in Figure 7 with two studies (Penn 2003; Coleman 2005) containing first‐person narratives and one (Yoshida 2002) containing third‐person narratives. Only one of the narratives involved a celebrity (an emailed newspaper article about Nobel Prize‐winning mathematician John Nash who was diagnosed with schizophrenia) (Corrigan (submitted) intervention A) and this had a small effect size (SMD ‐0.23, 95% CI ‐0.63 to 0.16), whereas the interventions with narratives by/about people who were not celebrities had a median SMD of ‐0.48. For data and studies contributing to this median, see Figure 9 and all studies in Figure 7. When we compared fictional and non‐fictional narratives, there was little difference with median SMDs for each group respectively being ‐0.42 and ‐0.47. Data and studies contributing to these medians can be seen in Figure 10 for non‐cluster trials. For cluster trials the data can be seen in Figure 7 with two studies (Penn 2003; Coleman 2005) containing non‐fictional narratives and one (Yoshida 2002) containing fictional narratives.
Forest plot of comparison: 5 Mass media vs. control by presence of narratives, outcome: 5.1 Prejudice (at earliest follow‐up time point).
Forest plot of comparison: 6 Mass media vs. control by celebrity narratives, outcome: 6.1 Prejudice (at earliest follow‐up time point).
Forest plot of comparison: 7 Mass media vs. control by fictional narratives, outcome: 7.1 Prejudice (at earliest follow‐up time point).
As the interventions were categorised into eight different subgroups for type of message, there were few (one to three) interventions within most subgroups. One subgroup ‐ 'recovery‐oriented' messages ‐ had effect size data from five interventions (Yoshida 2002; Reinke 2004 interventions A and B combined; Matthews 2009; Brown 2010 intervention A; Corrigan (submitted) intervention A) and found a small prejudice‐reducing median SMD (‐0.30). There was a large reduction in prejudice for a single study with a social inclusion/human rights message (SMD ‐2.94, 95% CI ‐3.52 to ‐2.37) (Russell 1988). Medium prejudice‐reducing effects were found for interventions with multiple primary messages (median SMD = ‐0.70) (Penn 2003; Smith 2007; Finkelstein 2008) and those with 'not to blame' messages (median SMD = ‐0.51) (Woods 2002; Woods 2003; Woods 2005). 'Seek professional care' messages had small (median SMD = ‐0.43) prejudice‐reducing effects (Morgan Owusu 2002; Demyan 2009; Jorm 2010a); and negligible effects (median SMD = ‐0.13) were found for 'negative impact of mental illness' messages (Reinke 2004 intervention C; Corrigan (submitted) intervention B). Some types of message were increased levels of prejudice. A small prejudice‐increasing effect (median SMD = 0.22) was found for 'this is hallucinatory experience' messages (Bunn 2009; Brown 2010 intervention B); and a large (median SMD = 0.99) prejudice‐increasing effect for biomedical messages (Coleman 2005; Iobst 2008 interventions A and C combined). The studies and data contributing to the medians for the messages subgroup analysis can be seen in Figure 11 for non‐cluster trials. For cluster trials the data can be seen in Figure 7 with Penn 2003 having multiple primary messages; Coleman 2005 having a biomedical primary message, and Yoshida 2002 having a recovery‐oriented primary message.
Forest plot of comparison: 8 Mass media vs. control by type of message, outcome: 8.1 Prejudice (at earliest follow‐up time point).
When the interventions were examined by type of media, audiovisual and print interventions were found to have similar median SMDs of ‐0.47 and ‐0.46 respectively. A small prejudice‐reducing effect was also found for Internet interventions (median SMD = ‐0.30). For the two audio interventions (Bunn 2009; Brown 2010 intervention B) the median effect size represented a small increase in prejudice at 0.22. The studies and data contributing to these medians can be seen in Figure 12 for non‐cluster trials. For cluster trials the data can be seen in Figure 7 with two (Penn 2003; Coleman 2005) being audiovisual and one (Yoshida 2002) being a print intervention.
Forest plot of comparison: 9 Mass media vs. control by type of media, outcome: 9.1 Prejudice (at earliest follow‐up time point)).
Meta‐analysis
See Data and analyses. We examined the values of the Q (Chi2) statistic and I2 index for studies with forest plots for overall effects by follow‐up time point and for each subgroup in the subgroup analyses for both primary outcomes. No meta‐analysis was appropriate for the secondary outcomes due to the disparate and sparse nature of the data. There was substantial statistical heterogeneity (I2 > 50%) (Higgins 2011, section 9.5.2) in all groups apart from the following:
-
Discrimination outcome: main analysis, cluster trials, dichotomous outcomes:.Tau² = 0.00; Chi² = 0.03, df = 1 (P = 0.85); I² = 0% (Yoshida 2002; Penn 2003)
-
Prejudice outcome: main analysis, non‐cluster trials, 6 to 9 months follow‐up: Tau² = 0.01; Chi² = 1.50, df = 1 (P = 0.22); I² = 33% (Finkelstein 2008; Jorm 2010a)
-
Prejudice outcome: Third person narratives: Tau² = 0.03; Chi² = 5.67, df = 3 (P = 0.13); I² = 47% (Iobst 2008; Demyan 2009; Matthews 2009; Corrigan (submitted))
-
Prejudice outcome: Non‐celebrity narratives: Tau² = 0.00; Chi² = 0.48, df = 1 (P = 0.49); I² = 0% (Matthews 2009; Corrigan (submitted), intervention B)
-
Prejudice outcome: 'Not to blame' messages: Tau² = 0.00; Chi² = 0.27, df = 2 (P = 0.87); I² = 0% (Woods 2002; Woods 2003; Woods 2005)
Although there was statistical homogeneity regarding the dichotomous discrimination outcomes, the two studies (Yoshida 2002; Penn 2003) were very dissimilar (different media, primary messages, and number of components and one intervention being fictional and the other non‐fictional), and were also heterogenous methodologically, one assessing outcomes immediately post‐intervention and the other only at nine months follow‐up. Consequently, meta‐analysis was considered inappropriate.
For prejudice outcomes in the main analysis there was also one group of studies for which I2 was < 50%. (non‐cluster trials, Finkelstein 2008; Jorm 2010a at 6 to 9 months follow‐up). These studies were considered to be clinically and methodologically homogenous, both being three‐arm studies of Internet versus print versus control multi‐component with the Internet arms involving first‐person non‐fictional narratives. Therefore meta‐analysis was deemed appropriate and the combined effect size was SMD = ‐0.36 (95% CI ‐0.63 to ‐0.10) demonstrating a reduction in prejudice.
We concluded that meta‐analysis was not warranted for the studies in the first subgroup with I2 below 50% (third‐person narratives, prejudice outcome) due to substantial clinical heterogeneity. For example the third‐person narrative interventions portray disparate conditions (autism, psychosis and exam stress), varied media types (Internet, audiovisual and print) and different types of primary message (biomedical, seek help, recovery‐oriented, and multiple).
There is more clinical similarity in the second subgroup above (non‐celebrity narratives, prejudice outcome) in that both address psychosis and are print journalism (delivered in print format or by the Internet), although their primary messages are very different, with one being recovery‐oriented and the other focusing on the negative impact of mental illness. The former is a magazine article recounting the story of a young woman successfully recovering from an episode of psychosis with the help of mental health services. The latter is a newspaper piece telling the story of a man who stabbed himself to death in prison during an acute psychotic episode whilst guards looked on, and focuses on poor mental healthcare provision. We therefore concluded that meta‐analysis was not appropriate here due to clinical heterogeneity.
For the third subgroup with I2 < 50% (interventions with 'not to blame' primary messages) the studies are highly homogenous with identical outcomes, interventions, and population types and similar methodologies, and all were conducted by the same author. For the prejudice but not the discrimination outcome, statistical homogeneity was also confirmed by visual inspection of the forest plot and by the heterogeneity statistics cited above. The combined effect size was SMD = ‐0.52 (95% CI ‐0.71 to ‐0.33) demonstrating, in line with the findings of the individual studies, a reduction in prejudice, but providing a more precise estimate of the effect.
Sensitivity analysis
Much of the sensitivity analysis planned was precluded since so few data were appropriate for meta‐analysis.
In the meta‐analysis of prejudice outcomes in the studies by Woods (Woods 2002; Woods 2003; Woods 2005) we examined the effects of removing Woods 2002 as this was at high risk of attrition (the studies did not differ on any other type of risk). This marginally increased the combined effect size from ‐0.52 (95% CI ‐0.71 to ‐0.33) to ‐0.55 (95% CI ‐0.76 to ‐0.33). Removing the less precise studies (Woods 2002; Woods 2005) also increased the effect size marginally from ‐0.52 (95% CI ‐0.71 to ‐0.33) to ‐0.56 (95% CI ‐0.83 to ‐0.30). Using a fixed‐effect model did not alter the findings with the combined effect size remaining at ‐0.52 (95% CI ‐0.71 to ‐0.33).
We undertook a sensitivity analysis to investigate the impact of using estimated ICCs in the cluster trials which did not account for the design effect in their analysis and for which no raw data were available (Penn 2003; Coleman 2005). The SMD between intervention and control arms was insignificant in both studies after adjusting standard errors for clustering. Any extra adjustments to penalise the studies further would result in the outcomes still remaining insignificant at the 5% significance level. Effect estimates of these studies unadjusted for the clustered data remained insignificant.
Removing the multi‐arm studies which included arms that the study authors considered unlikely to reduce stigma (Reinke 2004; Brown 2010; Corrigan (submitted)) in a post hoc sensitivity analysis changed the median SMD for prejudice outcomes at immediate follow‐up from ‐0.38 to ‐0.44 (range unchanged). As both values represent a small effect size using Cohen's (Cohen 1988) rule of thumb, we considered that the findings were not substantially altered by this removal.
Secondary outcomes
Four studies (n = 1213) (Morgan Owusu 2002; Yoshida 2002; Finkelstein 2008; Jorm 2010a) examined knowledge and each used a different type of outcome and statistic (see Table 3). A narrative synthesis (Grimshaw 2003) of the knowledge data shows that for all seven interventions in the four studies the direction of effect was positive (increasing knowledge) and for five of the seven interventions the effect was statistically significant.
| Study ID
| Measure | Intervention arm | Intervention group summary statistic | N(I) | Control group summary statistic | N(C) | Effect estimate/test statistic and P value/95% CI | |
|
| Psychiatric Knowledge Survey | A (Internet education) | Immediate follow‐up: 90.5% items correct Six‐months follow‐up: 65.6% items correct | 69 | 43.9% items correct | 48 | Pre‐post Χ2 P value < 0.0001 | |
| B (Printed documents) | Immediate follow‐up: 56.3% items correct Six‐months follow‐up: 49.6% items correct | 76 | 43.9% items correct | 48 | Pre‐post Χ2 P value < 0.0001 | |||
| Four knowledge items, beliefs about depression treatments selected | A (CD‐ROM) | 65.8% | 73 | 52.4%
| 82 | OR for pre‐post intervention interactions = 2.11 | ||
| B (manual) | 67.9% | 83 | 52.4% | 82 | OR for pre‐post intervention interactions = 3.64 | |||
| Seven knowledge items, item 1 (what is mental illness) selected | A (standard video) | 96.6% (29/30) | 30 | 76.7% (23/30) | 30 | OR = 8.83 (95% CI 1.01 to 76.96) | ||
| B (culturally specific video) | 96.6% (29/30) | 30 | 76.7% (23/30) | 30 | OR = 8.83 (95% CI 1.01 to 76.96) | |||
| Knowledge of Mental Illness scale | Booklet | 3.22 (1.48) | 301 | 3.03 (1.41) | 391 | Adjusted mean difference = 0.20, 95% CI ‐0.06 to 0.46, P = 0.133) | ||
[1] Outcome measure used in two previous studies (Jorm 1997; Griffiths 2004); control group same for both intervention arms; statistic from paper, OR or SMD not calculable.
[2] Four items, if recognised depression/schizophrenia from vignettes of person portrayed as meeting DSM‐IV‐TR criteria for the conditions DSM (Jorm 1997); two items developed for study: beliefs about treatments for schizophrenia/depression (% of all type treatments deemed by authors to be helpful endorsed by respondent; outcome for review as specified per protocol (Brennan 2009); data from earliest follow‐up time point (1 month). N assumed from consort diagram as not stated.
[3] Items developed for study, dichotomous (correct or not); item 1 selected as knowledge outcome for review as specified per protocol [Grimshaw 2003]
[4] Six‐item scale developed for study; continuous scale; linear regression model fitted including a random effect for clustering by sampling design (19 areas, average observations per cluster 36); from authors dataset; results presented are unadjusted means (SD) by arm along with the adjusted mean difference between arms, corresponding 95% confidence interval and P value.
None of the publications provided any information about cost, although three authors of studies involving 416 participants provided intervention cost data on request. In an audio visual intervention the costs were estimated to be 100 US dollars (equivalent to £64 GBP) for the telecommunications students who developed the intervention (Demyan 2009). For a mental health first‐aid intervention the purchase costs were 35,000 Australian dollars (£22,404 GBP) for 250 e‐learning CDs at 140 dollars per CD and 7140 Australian dollars (£4570 GBP) for 238 manuals at 30 dollars per manual (Jorm 2010a). A third author stated on request that the only costs involved were for printing (Matthews 2009). No data were presented on cost effectiveness.
No studies examined reach or recall. One study (Yoshida 2002) assessed awareness of the mass media intervention with data on 727 participants and found that at nine months follow‐up 33% (106/321) of the intervention group reported 'having seen the booklets', as did 6.6% (27/406) of the control group.
Regarding the duration/sustainability of media effects, in 14 studies outcomes were measured immediately post‐intervention only. Five studies with analysable outcome data (n = 1225) had longer follow‐up and so could provide some information about duration or sustainability of effects. Discrimination was assessed at 9 months post‐intervention in one study (Yoshida 2002) and there was no evidence of effect at this time point (see Figure 5). It can be seen from Table 2, and as reported in the prejudice main comparison section, that there is evidence of prejudice reduction 1 week to 2 months, and 6 to 9 months after the interventions. It is also informative to consider studies which assessed outcomes at two post‐intervention time points. One study (Finkelstein 2008) assessed prejudice immediately post‐intervention and at six months post‐intervention. This study reported a large reduction in prejudice at immediate follow‐up and a medium reduction at six months (see Figure 6). One study assessed effects at one and six months post‐intervention (Jorm 2010a) and found a medium effect size at one month which was reduced to a small one at six months. One study (Kerby 2008) assessed prejudice immediately post‐intervention and at 8 weeks, and, although no data were available to calculate SMDs, in the paper authors reported a within‐group comparison of baseline to immediate follow‐up of z = ‐2.78 P = 0.005 and stated that the same test for immediate to 8 weeks follow‐up found 'no significant differences', suggesting a lasting effect. Brown 2010 measured prejudice immediately post‐intervention and one week later, and our analyses, which combined interventions, found no evidence of effect at either time point, although these analyses combined two interventions which had different effects. In the paper (Brown 2010) the authors reported that at one week follow‐up the prejudice‐reducing effects of the filmed contact (intervention A) partially persisted, however for simulated hallucinations (intervention B) at one week there was a small effect size indicating a continued increase in prejudice. However, the latter finding, as with Kerby 2008, should be viewed with caution as it is a within‐group comparison.
Three studies (n = 381) presented numerical data on the audience reactions to the interventions (Yoshida 2002; Finkelstein 2008; Jorm 2010a). These data were solely on favourability and none addressed information or message communicated. See Table 4. Reactions were positive to all five interventions in the three studies.
| Intervention A (Internet education) %, n = 69 | Intervention B (printed articles) %, n = 76 | |
| Liked educational experience in general (yes/probably yes) | 97.1 | 89.3 |
| Topic is important (yes/probably yes) | 100 | 97.4 |
| Content is useful (y) (yes/probably yes) | 98.6 | 90.7 |
| Content is interesting (y) (yes/probably yes) | 100 | 86.8 |
| Content is unpleasant (no/probably no) | 81.2 | 73.3 |
| Content is difficult to understand (no/probably no) | 89.9 | 61.8 |
| Liked way information is presented (y) (yes/probably yes) | 87.0 | 60.5 |
| Intervention A (CD‐ROM) %, n = 63 to 64 | Intervention B (manual) %, n = 62 to 66 | |
| Read most/all of material | 95.3 | 100.0 |
| Material was easy/very easy to understand | 96.9 | 97.0 |
| Learnt a great deal/a fair bit from material | 90.6 | 90.1 |
| Material was useful/very useful | 93.8 | 98.4 |
| Will use intervention in future | 73.0 | 90.3 |
| Would probably/definitely recommend intervention to others | 96.9 | 98.4 |
| Yoshida 2002 (N = 106)1,2 | Mean (SD) |
|
| Favourability rating (1 to 5) | 3.67 (0.85) | |
| Looking forward to next issue of booklet rating (1 to 5) | 3.67 (1.04) | |
| Recommendation to others rating (1 to 5) | 3.29 (0.78) |
[1] Among participants who were aware of the intervention
[2] High rating indicates a positive audience reaction
Only two studies (n = 455) made any mention of unforeseen adverse effects (other than increases in discrimination and prejudice). One (Jorm 2010a) stated ‘Given that this was an educational intervention with a non‐clinical sample, there was no formal enquiry about adverse events. Informally, no adverse events were reported'. Another (Finkelstein 2008) stated ‘No adverse effects, such as an increase in stigma as a result of the intervention, were identified’.
Discussion
Summary of main results
Our review identified 22 eligible studies with a total of 4490 participants, and 19 of these studies had primary outcome data. All studies were randomised controlled trials (RCTs) and were highly heterogeneous. Participants included the general population, employers, and students training for health/educational roles, but the majority (n = 13) involved other types of students. Interventions spanned Internet, audiovisual, audio and print media. Discrimination was assessed in only five studies, prejudice in all. Data on secondary outcomes were relatively sparse. Outcomes were most often assessed only immediately post‐intervention (n = 14), although three studies had follow‐up beyond six months (Yoshida 2002; Finkelstein 2008; Jorm 2010a).
For discrimination outcomes, one study (Woods 2002) found evidence of reduced discrimination but this was not replicated in two larger nearly identical studies by the same author (Woods 2003; Woods 2005), which found no evidence of effect, as did two further studies (Yoshida 2002; Penn 2003). The effect size was small or negligible for discrimination, and we conclude that these findings are compatible with mass media having either a positive effect, a negative effect or no effect on discrimination.
Our review showed that, overall, mass media interventions reduce prejudice and that the size of this effect can be considered small‐to‐medium (Cohen 1988) with standardised mean differences (SMDs) of ‐0.38, ‐0.38 and ‐0.49 for the three follow‐up time‐point categories (see Table 2). To further aid interpretation, we transformed effect sizes into the equivalent number of points on Link's Social Distance Scale (SDS) (Link 1999) using a population standard deviation of 0.59 for social distance in relation to schizophrenia using the SDS from the observational study, General Social Survey 1996, USA (Link 1999). On this scale, respondents can score a minimum of 1 and a maximum of 4. The overall median effect sizes for prejudice outcomes immediately post‐intervention, at 1 week to 2 months and at 6 to 9 months are the equivalent of reductions of 0.22, 0.22 and 0.29 points on the SDS. We can further extrapolate from data about SDS scores for different mental health conditions (Link 1999) that the overall effects of mass media interventions found in our review are similar to reducing the level of prejudice from that associated with symptoms of schizophrenia (mean SDS 2.75) to the level associated with symptoms of major depression (mean SDS score 2.54) as the difference between the two is 0.21. The four members of the stakeholder group and review team with declared personal experience of mental ill health were asked to rate the importance of this degree of reduction, and three rated it as 'quite important' and one as 'slightly important'.
This overall finding of a small‐to‐medium reduction in prejudice conceals the considerable variation in the effects of different individual interventions. The prejudice subgroup findings help to explain some of this variation. The clearest pattern of evidence is for the presence of first‐person narratives, which were found to be effective in reducing prejudice. Interventions with two or more components tended to reduce prejudice more than those with only one. Other findings are tentative due to the small number of interventions in subgroups..
The individual studies with large effect sizes merit comment. The two studies (Russell 1988; Finkelstein 2008) with large prejudice‐reducing effects took place in atypical settings. The former was dated 1988 and was a much earlier study than the others; the latter was the only study in a non‐high‐income country. It is possible that there may have been higher baseline prejudice in these contexts, making it easier to achieve large change. Alternatively the large effects may be due to factors inherent in the interventions, the former using three mailings of postal booklets with social inclusion/human rights messages and the latter a computer‐assisted educational programme containing personal narratives. The study with a large prejudice‐increasing effect (Coleman 2005) was the only first‐person narrative intervention to have a primarily biomedical message and the film’s title, 'Fires of the Mind: Dark Voices: Schizophrenia', emphasises acute illness. This highlights the importance of the type of narrative content.
Overall completeness and applicability of evidence
Completeness of evidence
There were a number of areas where the evidence was incomplete, thereby limiting the external validity of the review findings. Although two types of study design were eligible for inclusion, we located very few ITS studies and none that met the inclusion criteria of having defined start and end points and at least three time points before and after the intervention. This is surprising, as ITS designs are common in studies evaluating other types of mass media intervention (Grilli 2002; Vidanapathirana 2005). Only five included studies examined our primary outcome of discrimination, but all included studies examined prejudice, our other primary outcome. Discrimination, being a behavioural outcome, is more difficult to measure, but is of greater importance than prejudice for improving the lives of people with mental health problems. Data on all secondary outcomes were sparse. We found no studies conducted in lower‐middle, middle or low income countries, and none in which the populations were children or adolescents. The populations studied did not cover the full range of target groups who may stigmatise people with mental ill health. Four involved the general public and a further four targeted students training for health‐related professions. However, none targeted practising health professionals, and only one study included employers despite these groups being identified as common sources of mental health‐related stigma (Schulze 2007; Brohan 2010).
None of the studies tested long‐term (over three months) interventions and none investigated interventions combining more than one type of mass media, and yet these approaches are typical of several national and regional anti‐stigma programmes, e.g.Vaughn 2004; Dunion 2005; Henderson 2009. These programmes also often combine mass media with non‐mass media components. Although we only included studies where the mass media component comprised more than 50% of the intervention, we only located two studies combining mass media and non‐mass media, so the evidence remains relatively incomplete in this regard. Some types of mass media were not investigated by any of the included studies, specifically television, radio, cinema and mobile phone media. Only one study included a celebrity narrative (Corrigan (submitted), intervention A), whereas this is relatively common in national mass media campaigns (e.g. Vaughn 2004; Henderson 2009). No studies addressed stigma arising from mental ill health at the same time as addressing stigma due to other attributes.
The data were incomplete in several studies, as many authors did not provide important details, e.g. about randomisation methods (see Characteristics of included studies); attrition (see Figure 2); and about numbers of participants contributing data to each study arm (e.g. see footnotes to Figure 6). Furthermore, means and standard deviations were not always reported in papers and sometimes had to be requested from the study authors, or estimated or derived from other sources.
Applicability and transferability of evidence
The findings cannot necessarily be transferred to other contexts, populations and interventions not covered in the review (described in the section above on the completeness of the evidence). There are also issues around feasibility and resources. Some interventions are likely to be too costly to implement in many settings, although not enough is known about costs to state which are most likely to be problematic to implement for this reason. The majority of the print and Internet interventions were text‐based, and required some degree of literacy, which may preclude their use with some populations (Clement 2009). The availability of technology is a further factor which may limit the transferability of the findings. As anti‐stigma interventions are often led by people with personal experience of mental health‐related stigma or are conducted in partnership with them, their views will frequently shape the nature of mass media interventions used. For example, mental health professionals and people with mental ill health differ in their views about which types of messages should be included in anti‐stigma interventions (Clement 2010). The values and preferences of people with mental health problems may also influence whether mass media interventions are used at all; they may prefer face to face interventions as these provide greater opportunities for personal empowerment and employment (Clement 2012).
Quality of the evidence
See summary of findings Table for the main comparison. We had pre‐specified in the protocol (Clement 2011) that the main outcomes for assessment of the quality of the evidence were: discrimination towards people with mental ill health; prejudice towards people with mental ill health; cost; and unforeseen adverse effects. We made a post‐hoc decision to categorise discrimination as a critically‐important outcome and the remainder as important. Although all of the studies were randomised controlled trials which are considered to provide a high quality of evidence due to their design, for each outcome in this review we considered study limitations, inconsistency of results, the indirectness of the evidence, imprecision or other considerations, and downgraded the quality where appropriate (Guyatt 2008).
Critically‐important outcomes
Discrimination
Discrimination was assessed in five of the included studies. The evidence is affected by study limitations (risk of bias) as the majority (30/35) of 'Risk of bias' items were rated as unclear or high. We did not downgrade for inconsistency as the studies were fairly consistent in showing no evidence of effect. The discrimination measures were somewhat indirect, in that although four studies assessed behaviour in situations in which participants believed they would be interacting with a person with mental ill health, no observation of an actual interaction was made, and one study used participants' reports of their behaviour in real‐life settings as its measure (Yoshida 2002). Also, for each type of discrimination outcome the behaviour is not an indicator of discrimination in every instance. Consequently we downgraded the evidence for indirectness. The studies were not considered imprecise as only a minority (2/5) had outcome data from less than 100 participants (Woods 2002; Penn 2003). No other limits were noted (lack of validity of outcome measures was already included in our 'Risk of bias' ratings). Consequently the quality of evidence for discrimination was downgraded by two levels to 'low' for this outcome.
Important outcomes
Prejudice
All included studies assessed prejudice. There were study limitations due to risk of bias, as the majority (105/133) of the 'Risk of bias' indicators were classified as unclear or high risk. Overall the results were fairly consistent for 16 of the 19 studies with analysable outcome data (see Figure 6 and Figure 7); therefore the evidence was not downgraded for inconsistency. The evidence was limited by indirectness of study populations. with generic student samples commonly used as proxies for the general population. Imprecision was not a large problem as only a minority (5/19) of studies had outcome data from less than 100 participants. No other limits were noted. Consequently the quality of evidence for prejudice was also downgraded by two levels to 'low'.
Cost
No studies included any information on cost or cost effectiveness in papers. Three study authors provided data on the costs of the interventions on request (Demyan 2009; Matthews 2009; Jorm 2010a). These studies were not considered to be compromised by study limitations as the majority (12/21) of 'Risk of bias' items were rated as low risk, and there were only four high risk ratings. There was inconsistency of results, with costs varying widely. The data did not suffer from indirectness, but there was imprecision in the costs for two (Demyan 2009; Matthews 2009) of the three studies. Other considerations also limited the quality of the evidence such as the lack of cost‐reporting in papers and lack of cost‐effectiveness data. Consequently we downgraded the quality of the evidence by three levels to 'very low'.
Unforeseen adverse outcomes
Two studies contained statements about unforeseen adverse outcomes (Finkelstein 2008; Jorm 2010a). These studies were considered to be compromised by study limitations as the majority (9/14) of 'Risk of bias' items were rated as unclear or high risk. There was consistency of results and the data did not suffer from indirectness. However there was imprecision, in that one study did not formally assess adverse outcomes and neither specified their methods. No other limits were noted. Consequently we downgraded the quality of the evidence by two levels to 'low'.
Potential biases in the review process
We searched 11 databases, including grey literature databases and a non‐English language database. Limited translation of the MEDLINE strategy to other databases (notably CINAHL, CENTRAL, EMBASE and PsycINFO) will have impaired sensitivity and specificity. The impact of these shortcomings is difficult to determine; it may have resulted in the loss of relevant studies and influenced the results of the review. Resource limitations precluded the conduct of new searches before publication. However the electronic searching was supplemented by checking of references, citations and websites and communication with experts, which yielded eight further studies, all unpublished. It is therefore feasible that all relevant studies have been identified. We welcome contact from any authors who believe their studies may be relevant to this review.
The restriction to RCTs and ITS studies meant that we were unable to consider data from other types of evaluation of mass media interventions such as qualitative evaluations and before and after studies. Although RCTs are the highest form of evidence for evaluating interventions, the RCTs in our review were subject to a number of biases and other limitations (see Figure 2; Quality of the evidence).
As 10 of the 22 studies had 2 or more active arms, in our main analysis we combined these, as per our protocol (Clement 2011). However the arms were often quite different and had different effects. This had the effect of making our overall findings more conservative than they might otherwise have been. A further issue was the multiplicity of prejudice outcome measures evident in 12 of the studies. In most of these, no primary outcome was defined by the study author and no power calculation was undertaken (our second basis for selecting the outcome for analysis). Consequently we used methods which select the outcome for analysis as being the one with the median effect size (Grimshaw 2003; Brennan 2009). This obscures larger effects found for some outcome measures.
Our pre‐specified classification of all attitudinal and emotional outcomes into the single category of prejudice precluded the examination of possible differential effects on different types of prejudice. For example, some interventions may increase empathy but also increase social distance (Brown 2010 intervention B), or may reduce blame but increase assumptions about poor prognosis (Penn 2003).
It is possible that particular combinations of characteristics of a mass media intervention are important to its effectiveness, e.g. type of message within, for example, a personal narrative, using a particular medium. A hierarchical subgroup analysis might have been illuminating in this regard, but would have required substantially more data than were available in this review.
Agreements and disagreements with other studies or reviews
Our findings are broadly in line with other studies and reviews, whilst making an important additional contribution to the existing knowledge base. Several systematic reviews have investigated the effects of anti‐stigma interventions which have included mass media interventions. A systematic review of 71 outcome studies using contact, education or protest‐based approaches to reduce mental health‐related stigma (Corrigan 2012) has recently reported findings relevant to our review. Corrigan and colleagues' review did not specifically address the effectiveness of mass media interventions of all types, but did compare outcomes for face‐to‐face and video‐based contact, the latter being the equivalent of the mass media first‐person narrative interventions in our review. They found that both approaches were beneficial (Corrigan 2012). The effect sizes found for video‐based contact were smaller than those found in our review, with a mean d = ‐0.296 for attitudes and ‐0.197 for behavioural intentions (Corrigan 2012) compared to our finding of a median effect size of ‐0.51 on prejudice. This difference may be attributable to Corrigan's findings being based on all types of study design, not just RCTs. Corrigan and colleagues found video‐based contact to be less effective than face‐to‐face contact (Corrigan 2012), although two RCTs have found the delivery modes to be equivalent (Reinke 2004; Clement 2012).
A systematic review of school‐based interventions to reduce mental health‐related stigma noted a dearth of RCT evidence, poor methods, considerable clinical heterogeneity, and inconsistent or null results (Schachter 2008). It recommend the development, implementation and evaluation of a curriculum which fosters the development of empathy and, in turn, an orientation toward social inclusion and inclusiveness. These effects may be achieved largely by bringing especially but not exclusively the youngest children into direct, structured contact with an infant, and likely only the oldest children and youth into direct contact with individuals experiencing mental health difficulties. Similarly, a narrative review of educational interventions for young people concluded that direct contact with people with mental health problems seems to be key in reducing stigmatisation, while the components of education and video‐based contact conditions are still arguable (Yamaguchi 2011). A systematic review of target group‐oriented interventions to reduce mental health‐related stigma did not report any findings on the efficacy of mass media interventions, although some of the studies included in the review did have mass media elements (Holzinger 2008).
These reviews report that discrimination outcomes are rarely measured (Yamaguchi 2011; Corrigan 2012). Evaluations of national anti‐stigma programmes which have mass media components have sometimes examined reports of discrimination experienced by people with mental ill health, and have found reductions in discrimination (Wyllie 2011; Henderson 2012). These studies did not use the types of study design eligible for inclusion in this review, but their findings suggest that anti‐stigma programmes may have the potential to reduce discrimination. We found very little information about costs. McCrone and colleagues have applied economic modelling to data from Scotland's anti‐stigma campaign (McCrone 2010). They report that If the campaign caused 10% of changed attitudes then it was estimated to cost £35 per one less person who felt that people with mental health problems were dangerous, and £186 per one less person who felt the public needs protection from people with mental health problems.
The problematic nature of biomedical messages apparent in our review has also been demonstrated in two recent systematic reviews on public attributions for mental illness and levels of stigmatisation (Angermeyer 2011; Schomerus 2012), and in a consensus development study (Clement 2010). The latter study recommended the use of ‘see the person’ messages, and our finding about the benefits of interventions echoes this. A systematic review of simulated hallucination interventions to reduce stigma (Ando 2011) was in accordance with this review's finding that such interventions can be problematic with regard to stigma, especially when not combined with pre‐ and post‐simulation activities. However, the review's qualitative findings from studies which had included a pre‐intervention DVD developed by a researcher with personal experience of hearing voices, and a post‐simulation debriefing discussion, suggest that when delivered in this way, simulated hallucinations may have a number of beneficial effects.
When we compare our review findings with systematic review evidence about the effectiveness of mass media interventions in other fields, we find that studies on promoting HIV testing (Vidanapathirana 2005) and increasing health service utilisation (Grilli 2002) provided more consistent evidence for the effectiveness of mass media interventions than did the studies in our review. Our preliminary findings about the benefits of multi‐component interventions echo Brinn and colleagues' finding that the more effective interventions tended to be more intensive (Brinn 2010). Our review is also in agreement with one on mass media and HIV testing (Vidanapathirana 2005) in highlighting the need for more research on different types of media, cost effectiveness and on the characteristics of messages.
Study flow diagram.
Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
Risk of bias summary: review authors' judgements about each risk of bias item for each included study.
Forest plot of comparison: 1 Mass media vs. control (main comparison), outcome: 1.1 Discrimination (Immediate).
Forest plot of comparison: 1 Mass media vs. control (main comparison): outcome 1.2 Discrimination.
Forest plot of comparison: 1 Mass media vs. control (main comparison), outcome: 1.3 Prejudice.
Forest plot of comparison: 1 Mass media vs. control (main comparison), outcome: 1.4 Prejudice.
Forest plot of comparison: 5 Mass media vs. control by presence of narratives, outcome: 5.1 Prejudice (at earliest follow‐up time point).
Forest plot of comparison: 6 Mass media vs. control by celebrity narratives, outcome: 6.1 Prejudice (at earliest follow‐up time point).
Forest plot of comparison: 7 Mass media vs. control by fictional narratives, outcome: 7.1 Prejudice (at earliest follow‐up time point).
Forest plot of comparison: 8 Mass media vs. control by type of message, outcome: 8.1 Prejudice (at earliest follow‐up time point).
Forest plot of comparison: 9 Mass media vs. control by type of media, outcome: 9.1 Prejudice (at earliest follow‐up time point)).
Comparison 1 Mass media vs. control: main comparison, Outcome 1 Discrimination (Immediate).
Comparison 1 Mass media vs. control: main comparison, Outcome 2 Discrimination.
Comparison 1 Mass media vs. control: main comparison, Outcome 3 Prejudice.
Comparison 1 Mass media vs. control: main comparison, Outcome 4 Prejudice.
Comparison 2 Mass media vs. control by income of country, Outcome 1 Prejudice (at earliest follow‐up time point).
Comparison 3 Mass media vs. control by number of mass media components, Outcome 1 Prejudice (at earliest follow‐up time point).
Comparison 4 Mass media vs. control by whether combined with non‐mass media, Outcome 1 Prejudice (at earliest follow‐up time point).
Comparison 5 Mass media vs. control by presence of narratives, Outcome 1 Prejudice (at earliest follow‐up time point).
Comparison 6 Mass media vs. control by celebrity narratives, Outcome 1 Prejudice (at earliest follow‐up time point).
Comparison 7 Mass media vs. control by fictional narratives, Outcome 1 Prejudice.
Comparison 8 Mass media vs. control by type of message, Outcome 1 Prejudice (at earliest follow‐up time point).
Comparison 9 Mass media vs. control by type of media, Outcome 1 Prejudice (at earliest follow‐up time point)).
| Mass media compared with inactive control for reducing mental health‐related stigma | ||||
| Patient or population: General public or any of its constituent groups (excluding groups comprising solely of people with mental ill health) Settings: Any Intervention: Mass media Comparison: Inactive control | ||||
| Outcomes | Relative effect | No of Participants | Quality of the evidence | Comments |
| Discrimination (i) not registering to attend focus group with people with schizophrenia (immediate)1 (ii) not visiting sheltered workshop for people with mental illness (9 months)2 (iii) distance placed between chairs when anticipating meeting with person with Tourette's syndrome3 | ORs (802 participants, 2 studies) (i) 1.30 (0.53 to 3.19) (i) 1.19 (0.85 to 1.65) SMDs (394 participants, 3 studies) (iii) median SMD ‐0.25 Range of SMDs ‐0.85 (‐1.39 to ‐0.31) to ‐0.17 (‐0.53 to 0.20) | 1196 (5) | ⊕⊕⊝⊝ | Quality of evidence downgraded for study limitations (risk of bias) and indirectness of measures (see Quality of the evidence for full details) |
| Prejudice Multiple scales4 Follow‐up (i) Immediate5 (ii) 1 week to 2 months6 (iii) 6 to 9 months7 | Median SMD (i) ‐0.38 (ii) ‐0.38 (iii) ‐0.49 Range of SMDs ‐2.94 (‐3.52 to ‐2.37) to 2.40 (0.62 to 4.18) Median SMDs are equivalent to reductions of 0.22, 0.22 and 0.29 points on the 1 to 4 point SDS scale [7], which is equivalent to reducing the level of prejudice from that associated with symptoms of schizophrenia to the level associated with symptoms of major depression8 | 3176 (19) | ⊕⊕⊝⊝ | Quality of evidence downgraded for study limitations (risk of bias) and indirectness of populations (see Quality of the evidence for full details) |
| Cost (i) Audiovisual public service announcement (ii) a) CD‐ROM b) printed manual (iii) magazine article | Relative costs (i) 100 US dollars (equivalent to £64 GBP) vs. nil (ii) a) 35,000 Australian dollars (£22,404 GBP) for 250 e‐learning CDs @ 140 dollars per CD (ii) b) 7,140 Australian dollars(£4,570 GBP) for 238 manuals @ 30 dollars per manual vs. nil (iii) 'printing costs' vs 'printing costs' | 416 (3) | ⊕⊝⊝⊝ | Quality of evidence downgraded for inconsistency of results, imprecision, and 'other' (data only being available on request, lack of cost‐effectiveness data) (see Quality of the evidence for full details) |
| Unforeseen adverse effects | Statements in two studies: (i) ‘Given that this was an educational intervention with a non‐clinical sample, there was no formal enquiry about adverse events. Informally, no adverse events were reported'; (ii) ‘No adverse effects, such as an increase in stigma as a result of the intervention, were identified’ | 2 (455) | ⊕⊕⊝⊝ | Quality of evidence downgraded for study limitations (risk of bias) and imprecision (see Quality of the evidence for full details) |
| CI: Confidence interval; OR: Odds ratio; SMD: Standardised mean difference | ||||
| GRADE Working Group grades of evidence | ||||
| [1] Penn 2003 [2] Yoshida 2002 [3] Woods 2002; Woods 2003; Woods 2005 [4] 28 different measures were used (see Characteristics of included studies) [5] Brown 2010; Bunn 2009; Coleman 2005; Corrigan (submitted); Finkelstein 2008; Iobst 2008; Matthews 2009; Morgan Owusu 2002; Penn 2003; Smith 2007; Woods 2002; Woods 2003; Woods 2005; Yoshida 2002 [6] Russell 1988; Demyan 2009; Brown 2010; Jorm 2010a [7] Yoshida 2002; Finkelstein 2008; Jorm 2010a [OR:1 indicated decreased stigma, > 1 indicates increased stigma; SMD < 0 indicates decreased stigma, > 0 indicates increased stigma] [8] Link 1999 This reference for the scale also reports a population standard deviation of 0.59 for social distance in relation to schizophrenia using the SDS from the observational study, General Social Survey 1996, USA. It gives SDS scores for different mental health conditions which are used for further interpretation. | ||||
| Comparison | No. of comparisons with positive direct of effect1 | Median effect size across all comparisons2 | Median effect size across all comparisons without unit of analysis errors3 | No. of comparisons showing statistically significant effects4 | Interpretation of median effect size across all comparisons5 |
| Mass media vs. control | |||||
| Immediate follow‐up | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| 6 to 9 months follow‐up | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| Mass media vs. control by if 2+ mass media components | |||||
| 2+ mass media components | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| One mass media component | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| Mass media vs. control by presence of narratives | |||||
| First‐person narratives | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| Third‐person narratives | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| Mass media vs. control by type of primary message | |||||
| Recovery‐oriented | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| Not to blame | 3/3 | SMD = ‐0.25 | SMD = ‐0.25 | 1/3 | Small |
| Multiple | 0/1 | OR = 1.30 | ‐ | 0/1 | Negligible |
| Mass media vs. control by fictional narratives | |||||
| Fictional narratives | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| Non‐fictional narratives | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| Mass media vs. control by type of media | |||||
| Audiovisual | 3/4 | SMD = ‐0.25 OR = 1.30 | SMD = ‐0.25 | 1/4 (+) | Small/negligible |
| | 0/1 | OR = 1.19 | OR = 1.19 | 0/1 | Negligible |
| [1] Stigma‐reducing [2] When there was an even number of effect sizes the median reported is the mean of the two middle values. [3] This excludes cluster trials not adjusted for study design by the authors and without a study‐derived intra‐class correlation co‐efficient (Penn 2003) [4] (+) means in stigma‐reducing direction, (‐) means in stigma increasing direction [5] SMDs large ≥ 0.8; medium ≥ 0.5, small ≥ 0.2, negligible < 0.2; and for ORs large ≥ 4.3; medium ≥ 2.5, small ≥ 1.5, negligible < 1.5, based on Cohen’s (Cohen 1988) rule of thumb, and using a pragmatic decision about how to treat intermediate values | |||||
| Comparison | No. of comparisons with positive direct of effect1 | Median effect size across all comparisons2 | Median effect size across all comparisons without unit of analysis errors3 | No. of comparisons showing statistically significant effects4 | Interpretation of median effect size across all comparisons ( Cohen 1988)5 |
| Mass media vs. control | |||||
| Immediate follow‐up | 12/15 | ‐0.38 | ‐0.38 | 6/15 (+), 1/15 (‐) | Small |
| 1 to 8 weeks follow‐up | 3/5 | ‐0.38 | ‐0.38 | 2/5 (+) | Small |
| 6 to 9 months follow‐up | 3/3 | ‐0.49 | ‐0.49 | 1/3 (+) | Small‐to‐medium |
| Mass media vs. control by income of country | |||||
| Upper middle income | 1/1 | ‐1.69 | ‐1.69 | 1/1 | Large |
| High income | 14/18 | ‐0.42 | ‐0.42 | 7/18 (+) 1/18 (‐) | Small |
| Mass media vs. control by if 2+ mass media components | |||||
| 2+ mass media components | 4/6 | ‐0.49 | ‐0.49 | 3/6 (+) | Small‐to‐medium |
| One mass media component | 12/13 | ‐0.34 | ‐0.34 | 5/13 (+) 1/13 (‐) | Small |
| Mass media vs. control by if combined with non‐mass media | |||||
| With non‐mass media | 2/2 | ‐0.42 | ‐0.42 | 0/2 | Small |
| Mass media alone | 13/17 | ‐0.43 | ‐0.43 | 8/17 (+) 1/17 (‐) | Small |
| Mass media vs. control by presence of narratives | |||||
| First‐person narratives | 8/10 | ‐0.51 | ‐0.51 | 6/10 (+) 1/10 (‐) | Medium |
| Third‐person narratives | 4/5 | ‐0.03 | ‐0.03 | 2/5 (+) | Negligible |
| No narratives | 5/7 | ‐0.27 | ‐0.27 | 3/7 (+) 1/7 (‐) | Small |
| Mass media vs. control by celebrity narratives | |||||
| Celebrity narratives | 1/1 | ‐0.23 | ‐0.23 | 0/1 | Small |
| Non‐celebrity narratives | 11/15 | ‐0.48 | ‐0.44 | 7/15 (+) 1/15 (‐) | Small |
| Mass media vs. control by fictional narratives | |||||
| Fictional narratives | 2/3 | ‐0.42 | ‐0.42 | 1/3 (+) | Small |
| Non‐fictional narratives | 10/12 | ‐0.47 | ‐0.47 | 6/12 (+) 1/12 (‐) | Small |
| Mass media vs. control by type of message | |||||
| Biomedical | 1/2 | 0.99 | ‐0.42 | 1/2 (+) 1/2 (‐) | Large |
| Social inclusion/human rights | 1/1 | ‐2.94 | ‐2.94 | 1/1 (+) | Large |
| Recovery‐oriented | 5/5 | ‐0.30 | ‐0.30 | 1/5 (+) | Small |
| Negative impact of mental illness | 1/2 | ‐0.13 | ‐0.13 | 0/2 | Negligible |
| Seek professional care | 2/3 | ‐0.43 | ‐0.43 | 2/3 (+) | Small |
| Not to blame | 3/3 | ‐0.51 | ‐0.51 | 3/3 (+) | Medium |
| Hallucinatory experience | 1/2 | 0.22 | 0.22 | 1/2 (‐) | Small |
| Multiple messages | 3/4 | ‐0.70 | ‐0.71 | 1/4 (+) | Medium |
| Mass media vs. control by type of media | |||||
| Internet | 3/4 | ‐0.30 | ‐0.30 | 2/4 (+) | Small |
| Audiovisual | 7/10 | ‐0.47 | ‐0.47 | 5/10 (+) 1/10 (‐) | Small |
| Audio | 1/2 | 0.22 | 0.22 | 1/2 (‐) | Small |
| | 6/6 | ‐0.46 | ‐0.46 | 4/5 (+) | Small |
| [1] Stigma‐reducing [2] Excluding study with median and IQR data only (Kerby 2008). When there was an even number of effect sizes the median reported is the mean of the two middle values. [3] This excludes cluster trials not adjusted for study design by the authors and without a study‐derived intra‐class correlation co‐efficient (Penn 2003; Coleman 2005) [4] (+) means in stigma‐reducing direction, (‐) means in stigma increasing direction. Excluding study with median and IQR data only (Kerby 2008) [5] SMDs large ≥ 0.8; medium ≥ 0.5, small ≥ 0.2, negligible < 0.2, based on Cohen’s (Cohen 1988) rule of thumb, and using a pragmatic decision about how to treat intermediate values | |||||
| Study ID
| Measure | Intervention arm | Intervention group summary statistic | N(I) | Control group summary statistic | N(C) | Effect estimate/test statistic and P value/95% CI | |
|
| Psychiatric Knowledge Survey | A (Internet education) | Immediate follow‐up: 90.5% items correct Six‐months follow‐up: 65.6% items correct | 69 | 43.9% items correct | 48 | Pre‐post Χ2 P value < 0.0001 | |
| B (Printed documents) | Immediate follow‐up: 56.3% items correct Six‐months follow‐up: 49.6% items correct | 76 | 43.9% items correct | 48 | Pre‐post Χ2 P value < 0.0001 | |||
| Four knowledge items, beliefs about depression treatments selected | A (CD‐ROM) | 65.8% | 73 | 52.4%
| 82 | OR for pre‐post intervention interactions = 2.11 | ||
| B (manual) | 67.9% | 83 | 52.4% | 82 | OR for pre‐post intervention interactions = 3.64 | |||
| Seven knowledge items, item 1 (what is mental illness) selected | A (standard video) | 96.6% (29/30) | 30 | 76.7% (23/30) | 30 | OR = 8.83 (95% CI 1.01 to 76.96) | ||
| B (culturally specific video) | 96.6% (29/30) | 30 | 76.7% (23/30) | 30 | OR = 8.83 (95% CI 1.01 to 76.96) | |||
| Knowledge of Mental Illness scale | Booklet | 3.22 (1.48) | 301 | 3.03 (1.41) | 391 | Adjusted mean difference = 0.20, 95% CI ‐0.06 to 0.46, P = 0.133) | ||
| [1] Outcome measure used in two previous studies (Jorm 1997; Griffiths 2004); control group same for both intervention arms; statistic from paper, OR or SMD not calculable. [2] Four items, if recognised depression/schizophrenia from vignettes of person portrayed as meeting DSM‐IV‐TR criteria for the conditions DSM (Jorm 1997); two items developed for study: beliefs about treatments for schizophrenia/depression (% of all type treatments deemed by authors to be helpful endorsed by respondent; outcome for review as specified per protocol (Brennan 2009); data from earliest follow‐up time point (1 month). N assumed from consort diagram as not stated. [3] Items developed for study, dichotomous (correct or not); item 1 selected as knowledge outcome for review as specified per protocol [Grimshaw 2003] [4] Six‐item scale developed for study; continuous scale; linear regression model fitted including a random effect for clustering by sampling design (19 areas, average observations per cluster 36); from authors dataset; results presented are unadjusted means (SD) by arm along with the adjusted mean difference between arms, corresponding 95% confidence interval and P value. | ||||||||
| Intervention A (Internet education) %, n = 69 | Intervention B (printed articles) %, n = 76 | |
| Liked educational experience in general (yes/probably yes) | 97.1 | 89.3 |
| Topic is important (yes/probably yes) | 100 | 97.4 |
| Content is useful (y) (yes/probably yes) | 98.6 | 90.7 |
| Content is interesting (y) (yes/probably yes) | 100 | 86.8 |
| Content is unpleasant (no/probably no) | 81.2 | 73.3 |
| Content is difficult to understand (no/probably no) | 89.9 | 61.8 |
| Liked way information is presented (y) (yes/probably yes) | 87.0 | 60.5 |
| Intervention A (CD‐ROM) %, n = 63 to 64 | Intervention B (manual) %, n = 62 to 66 | |
| Read most/all of material | 95.3 | 100.0 |
| Material was easy/very easy to understand | 96.9 | 97.0 |
| Learnt a great deal/a fair bit from material | 90.6 | 90.1 |
| Material was useful/very useful | 93.8 | 98.4 |
| Will use intervention in future | 73.0 | 90.3 |
| Would probably/definitely recommend intervention to others | 96.9 | 98.4 |
| Yoshida 2002 (N = 106)1,2 | Mean (SD) |
|
| Favourability rating (1 to 5) | 3.67 (0.85) | |
| Looking forward to next issue of booklet rating (1 to 5) | 3.67 (1.04) | |
| Recommendation to others rating (1 to 5) | 3.29 (0.78) | |
| [1] Among participants who were aware of the intervention [2] High rating indicates a positive audience reaction | ||
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Discrimination (Immediate) Show forest plot | 3 | Std. Mean Difference (IV, Random, 95% CI) | Totals not selected | |
| 2 Discrimination Show forest plot | 2 | Odds Ratio (Random, 95% CI) | Totals not selected | |
| 2.1 Immediate | 1 | Odds Ratio (Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 2.2 6 ‐ 9 months | 1 | Odds Ratio (Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 3 Prejudice Show forest plot | 15 | Std. Mean Difference (IV, Random, 95% CI) | Totals not selected | |
| 3.1 Immediate | 12 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 3.2 1 week ‐ 2 months | 4 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 3.3 6 ‐ 9 months | 2 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 4 Prejudice Show forest plot | 3 | Std. Mean Difference (Random, 95% CI) | Totals not selected | |
| 4.1 Immediate | 2 | Std. Mean Difference (Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 4.2 6 ‐ 9 months | 1 | Std. Mean Difference (Random, 95% CI) | 0.0 [0.0, 0.0] | |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Prejudice (at earliest follow‐up time point) Show forest plot | 15 | Std. Mean Difference (IV, Random, 95% CI) | Totals not selected | |
| 1.1 Upper‐middle income | 1 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.2 High income | 14 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Prejudice (at earliest follow‐up time point) Show forest plot | 15 | Std. Mean Difference (IV, Random, 95% CI) | Totals not selected | |
| 1.1 Multiple mass media components | 4 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.2 One mass media component | 11 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Prejudice (at earliest follow‐up time point) Show forest plot | 15 | Std. Mean Difference (IV, Random, 95% CI) | Totals not selected | |
| 1.1 Mass media combined with non‐mass media | 1 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.2 Mass media alone | 14 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Prejudice (at earliest follow‐up time point) Show forest plot | 15 | Std. Mean Difference (IV, Random, 95% CI) | Totals not selected | |
| 1.1 First‐person narratives | 7 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.2 Third‐person narratives | 4 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.3 No narratives / none stated | 7 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Prejudice (at earliest follow‐up time point) Show forest plot | 11 | Std. Mean Difference (IV, Random, 95% CI) | Totals not selected | |
| 1.1 Celebrity | 1 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.2 Non‐celebrity | 11 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Prejudice Show forest plot | 11 | Std. Mean Difference (IV, Random, 95% CI) | Totals not selected | |
| 1.1 Fictional narrative | 2 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.2 Non‐fictional narrative | 9 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Prejudice (at earliest follow‐up time point) Show forest plot | 15 | Std. Mean Difference (IV, Fixed, 95% CI) | Totals not selected | |
| 1.1 Biomedical | 1 | Std. Mean Difference (IV, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.2 Social inclusion / human rights | 1 | Std. Mean Difference (IV, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.3 Recovery‐oriented | 4 | Std. Mean Difference (IV, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.4 Negative impact of mental illness | 2 | Std. Mean Difference (IV, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.5 Seek professional care | 3 | Std. Mean Difference (IV, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.6 Not to blame | 3 | Std. Mean Difference (IV, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.7 This is hallucinatory experience | 2 | Std. Mean Difference (IV, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.8 Multiple primary messages | 2 | Std. Mean Difference (IV, Fixed, 95% CI) | 0.0 [0.0, 0.0] | |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Prejudice (at earliest follow‐up time point)) Show forest plot | 15 | Std. Mean Difference (IV, Random, 95% CI) | Totals not selected | |
| 1.1 Internet (also includes CD‐ROM) | 4 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.2 Recorded ‐ audiovisual | 7 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.3 Recorded ‐ audio | 2 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
| 1.4 Print | 5 | Std. Mean Difference (IV, Random, 95% CI) | 0.0 [0.0, 0.0] | |
