Rehabilitación vocacional para pacientes con enfermedades mentales graves
Resumen
Antecedentes
Las tasas de desempleo son altas entre los pacientes con enfermedades mentales graves, aunque las encuestas muestran que la mayoría quiere trabajar. Los servicios de rehabilitación vocacional existen para ayudar a los enfermos mentales a encontrar trabajo. Tradicionalmente estos servicios han ofrecido un período de preparación (adiestramiento prevocacional), antes de intentar colocar a los pacientes en un empleo competitivo (es decir, abierto). Más recientemente, algunos servicios han comenzado a colocar a los pacientes en un empleo competitivo de inmediato, al tiempo que proporcionan apoyo en el trabajo (Empleo con Apoyo). No está claro qué enfoque es el más efectivo.
Objetivos
Evaluar los efectos de la adiestramiento prevocacional y el Empleo con Apoyo (para los pacientes con enfermedades mentales graves) entre sí y con respecto a la atención estándar (en el hospital o en la comunidad). Además, evaluar los efectos de: a) variedades especiales de Adiestramiento Prevocacional (modelo Clubhouse) y de Empleo con Apoyo (modelo de colocación y apoyo individual); y b) técnicas para mejorar cualquiera de los dos enfoques, por ejemplo, el pago o la intervención psicológica.
Métodos de búsqueda
Se realizaron búsquedas en CINAHL (1982 a 1998), The Cochrane Library (Número 2, 1999), EMBASE (1980 a 1998), MEDLINE (1966 a 1998) y PsycLIT (1887 a 1998). Se inspeccionaron las listas de referencias de los estudios y revisiones elegibles y se estableció contacto con los investigadores en el campo para identificar los estudios no publicados.
Criterios de selección
Ensayos controlados aleatorizados de enfoques de rehabilitación vocacional para pacientes con enfermedades mentales graves.
Obtención y análisis de los datos
Un equipo de dos calificadores seleccionaron de manera fiable los ensayos incluidos. Dos autores de la revisión obtuvieron los datos por separado y los verificaron de forma cruzada. Se estableció contacto con los autores de los estudios para obtener información adicional. Se calcularon las medidas de riesgo relativo (RR) con los intervalos de confianza (IC) del 95% para los datos dicotómicos homogéneos. Se utilizó un modelo de efectos aleatorios para los datos dicotómicos heterogéneos. Los datos continuos se presentaron en tablas (no hubo datos continuos suficientes para un metanálisis formal). Se realizó un análisis de sensibilidad que excluyó los ensayos de calidad más deficiente.
Resultados principales
Se identificaron 18 ensayos controlados aleatorizados de calidad razonable. El hallazgo principal fue que en el resultado primario (número en un empleo competitivo) el Empleo con Apoyo fue significativamente más efectivo que el Adiestramiento Prevocacional; por ejemplo, a los 18 meses el 34% de los pacientes en Empleo con Apoyo estaban empleados versus el 12% en el Adiestramiento Prevocacional (RR de efectos aleatorios [desempleo] 0,76; IC del 95%: 0,64 a 0,89; NNT 4,5). Los pacientes en Empleo con Apoyo también ganaban más y trabajaban más horas por mes que los que estaban en Adiestramiento Pre‐vocacional. No hubo evidencia de que el Adiestramiento Prevocacional fuera más efectivo para ayudar a los pacientes a obtener un empleo competitivo, en comparación con la atención comunitaria estándar.
Conclusiones de los autores
El Empleo con Apoyo es más efectivo que el Adiestramiento Prevocacional para ayudar a los pacientes con enfermedades mentales graves a obtener un empleo competitivo. No hay evidencia clara de que el Adiestramiento Prevocacional sea efectivo.
Resumen en términos sencillos
Rehabilitación vocacional para pacientes con enfermedades mentales graves
A la mayoría de los pacientes con enfermedades mentales graves les gustaría trabajar y existen razones éticas, sociales y clínicas de peso para ayudarles a conseguir este objetivo. El adiestramiento prevocacional y el empleo con apoyo son dos enfoques diferentes para ayudar a los pacientes con enfermedades mentales graves a conseguir un empleo. El principio clave del adiestramiento prevocacional es que es necesario un período de preparación antes de entrar en un empleo competitivo. Por el contrario, el principio clave del Empleo con Apoyo es que la colocación en un empleo competitivo se debe producir lo más rápidamente posible, seguida de apoyo y adiestramiento en el trabajo. Esta revisión sistemática encontró que los pacientes que recibieron Empleo con Apoyo tuvieron significativamente más probabilidad de tener un empleo competitivo que los que recibieron Adiestramiento Prevocacional (a los 12 meses, el 34% de los empleados en Empleo con Apoyo, en comparación con el 12% en Adiestramiento Entrenamiento Prevocacional).
Authors' conclusions
Background
People who suffer from severe mental disorder experience high rates of unemployment. In the United States unemployment rates amongst such people are estimated at 75‐85% (Lehman 1995, Ridgeway 1998), whilst in the UK rates of 61‐73% have been reported (McCreadie 1992, Meltzer 1995). These high rates reflect the disability caused by severe mental illness, but they also reflect discrimination (unemployment rates are higher than in other disabled groups ‐ ONS 1998) and the low priority given to employment by psychiatric services (Lehman 1998). Despite high unemployment rates amongst the severely mentally ill, surveys have consistently shown that most want to work (Hatfield 1992, Lehman 1995, Shepherd 1994).
There are compelling ethical, social and clinical reasons for helping mentally ill people to work. From an ethical standpoint, the right to work is enshrined in the Universal Declaration of Human Rights 1948 and has been incorporated into national disability acts in Europe and America. From a social standpoint, high unemployment rates are an index of the social exclusion of severely mentally ill people, which many governments, including that of the UK, are committed to reducing (DoH 1998). Finally, from a clinical standpoint, employment may lead to improvements in the outcome of severe mental illness, through increasing self‐esteem, alleviating psychiatric symptoms, and reducing dependency and relapse (Lehman 1995).
Helping mentally ill people to work is not a new idea. The value of therapeutic work was recognised by the pioneers of the asylum movement, and in their latter days many large asylums depended on the labour of their inmates in farms, workshops, or work‐crews (Jones 1993). As asylums closed down, work experience played an important role in the preparation of patients for discharge. Patients who performed well on graded tasks within the hospital were gradually reintroduced to working in the community, often through special arrangements with local employers. As community care developed, these arrangements evolved into enterprises or workshops providing sheltered employment within a segregated work setting (Gervey 1994). Such sheltered workshops aimed to place clients in competitive employment after a period of pre‐vocational training, but follow up studies showed a success rate of only five to ten percent (Bond 1988, Connors 1987).
The Clubhouse movement arose in the 1950s as a reaction to traditional sheltered employment and to the lack of emphasis on work within mental health services (Macias 1995). The Clubhouse movement proposed that better employment outcomes could be achieved by fostering patient autonomy in a non‐psychiatric setting (known as a Clubhouse). The Clubhouse is a building run by clients and staff along egalitarian lines, where clients meet for social activity, mutual support and graded work experience. Like traditional pre‐vocational training, the Clubhouse approach involves a period of preparation before clients attempt to return to competitive employment. This period of preparation consists of two stages: the work ordered day and Transitional Employment (Beard 1982). The work‐ordered day refers to a process whereby clients join work crews (working side‐by‐side with staff) that take responsibility for managing and maintaining the Clubhouse. Work crews are seen as a means of preparing for Transitional Employment. Transitional Employment refers to the placement of clients in a series of paid but temporary jobs controlled by the Clubhouse, in order to help them develop the skills and confidence required to cope with competitive employment (Bond 1998a). Whilst there are no rigid guidelines for length of time on work crews, clients are discouraged from seeking competitive employment until they have achieved success in Transitional Employment, and are free to return to work crews at any time (Bilby 1992). Cross‐fertilisation between the Clubhouse and traditional approaches led to a number of hybrid approaches (or stepwise eclectic approaches), combining for example, transitional employment with pre‐employment training (Bond 1998a).
In the mid‐1980s a new approach to vocational rehabilitation emerged, known as Supported Employment. Supported Employment involved trying to place clients in competitive jobs without any extended preparation (Bond 1992). Originally developed for people with learning disabilities, Supported Employment has been defined as paid work that takes place in normal work settings with provision for ongoing support services (Becker 1994, Bond 1998a). Proponents of Supported Employment had two objections to Pre‐vocational Training (Bilby 1992, Bond 1997a). First, they argued that it promoted dependency and deterred clients from finding competitive employment. Second, they argued that Pre‐vocational Training was not effective in developing work skills. Instead of Pre‐vocational Training, they proposed trying to place clients as quickly as possible in competitive employment positions, where they would receive intensive on‐the‐job support and training from personnel known as Job Coaches (Anthony 1987).
Individual Placement and Support is a carefully specified variant of Supported Employment distinguished by six key principles: (1) the goal is competitive employment in work settings integrated into a community's economy; (2) clients are expected to obtain jobs directly, rather than following lengthy pre‐employment training (rapid job search); (3) rehabilitation is an integral component of mental health treatment rather than a separate service; (4) services are based on clients' preferences and choices; (5) assessment is continuous and based on real work experiences; and (6) follow‐on support is continued indefinitely (Bond 1998b). Adherence to Individual Placement and Support guidelines may be measured using a fidelity scale (Bond 1997b).
Natural experiments have suggested that Supported Employment is an acceptable intervention that helps sustain people in work. For example, Drake‐New Hampshire2 studied a Community Mental Health Centre that was forced to eliminate its day care program because of budget cuts. As a replacement for the day care services, a small Supported Employment program was started. Drake‐New Hampshire2 compared the day centre conversion site to a second site, which continued to offer day care alongside traditional Pre‐vocational services. Clients at the Supported Employment site showed increased rates of competitive employment, whilst no change was found for the site not converting. After the completion of the initial conversion, the second site subsequently converted to Supported Employment, with similarly favourable results (Clark 1996). Favourable results have been reported from a third day care centre, which made a partial transition to the Individual Placement and Support model of Supported Employment (Bailey‐New Hampshire).
Both Pre‐vocational Training (traditional, Clubhouse) and Supported Employment are widely practiced. In the US there are 3,000 'psychiatric rehabilitation providers' offering traditional pre‐vocational training and about 230 Clubhouses. There are also around 36,000 mentally ill people in Supported Employment schemes (Bond 1998a, Wehman 1997). In the UK there are around 135 organisations offering Pre‐vocational Training and 77 offering Supported Employment (ERMIS 1998). It remains unclear if Pre‐vocational Training and Supported Employment are equally effective.
Objectives
The main objective was to determine how far Pre‐vocational Training and Supported Employment were effective in helping people with severe mental illness to obtain competitive (i.e. open) employment. The review also examined how far Pre‐vocational Training and Supported Employment affected other work and clinical outcomes. The main comparisons in the review were as follows:
1. Pre‐vocational Training versus standard hospital care;
2. Pre‐vocational Training (in addition to standard community care) versus standard community care;
3. Supported Employment (in addition to standard community care) versus standard community care;
4. Pre‐vocational Training versus Supported Employment.
In addition, the review examined the effectiveness of modifications designed to enhance approaches to vocational rehabilitation (e.g. payment or psychological intervention) and the effectiveness of well‐characterised sub‐types of Pre‐vocational Training (Clubhouse model) and Supported Employment (Individual Placement and Support model). The review did not consider the effectiveness of Assertive Community Treatment or other forms of case management in improving employment outcomes, as these general approaches to enhancing community care have been reviewed elsewhere (Marshall 1999a, Marshall 1999b).
Methods
Criteria for considering studies for this review
Types of studies
Relevant randomised controlled trials that provided data which could be analysed on an intention‐to‐treat basis.
Types of participants
Vocational rehabilitation approaches were not designed for a specific diagnostic group nor are they applied in a diagnostic‐specific way in everyday practice. Therefore, for the purpose of this review, the main requirements of participants were that they were similar to those who typically present to vocational rehabilitation services. Specific inclusion criteria were that a majority of clients in the trial were: (a) aged 18‐65; and (b) suffering from severe mental disorder defined as: schizophrenia and schizophrenia‐like disorders; bipolar disorder; or depression with psychotic features. Substance abuse was not considered a severe mental disorder, but trials were eligible if participants had a problem with substance abuse in addition to a mental disorder. Trials were excluded where a majority of participants were suffering from a learning disability.
Types of interventions
Four interventions were defined: Pre‐vocational Training, Supported Employment, enhanced approaches, and standard care.
1. Pre‐vocational Training
Pre‐vocational Training was defined as any approach to vocational rehabilitation in which participants were expected to undergo a period of preparation, before being encouraged to seek competitive employment. This preparation could involve either work in a sheltered environment (such as a workshop or work unit), or some form of pre‐employment training or transitional employment. Both the traditional (sheltered workshop) and Clubhouse approaches were classified as Pre‐vocational Training.
2. Supported Employment
Supported employment was defined as any approach to vocational rehabilitation that attempted to place clients immediately in competitive employment. It was acceptable for Supported Employment to begin with a short period of preparation, but this had to be of less than one month duration and not involve work placement in a sheltered setting, or training, or transitional employment. Individual Placement and Support was defined as Supported Employment that adhered to the six principles outlined in the Background (see above).
3. Modifications of vocational rehabilitation programs
Modified programs were defined as either Pre‐vocational Training or Supported Employment that had been enhanced by some technique to increase participants' motivation. Typically such techniques consisted of payment for participation in the program, or some form of psychological intervention.
4. Standard care was defined as usual psychiatric care for patients in the trial, without any specific vocational component. In all trials where an intervention is compared against standard care, unless otherwise stated clients will have received the intervention in addition to standard care. Thus, for example, in a trial comparing Pre‐vocational Training against standard community care, patients in the Pre‐vocational Training group will also be in receipt of standard community services, such as out‐patient appointments.
Types of outcome measures
The primary outcome was number of clients in competitive employment at various time points (defined as a full or part time position held by the client in an ordinary work setting, for which they were receiving payment at the market rate).
Secondary outcome measures were grouped into three main categories.
1. Other employment outcomes:
1.1 in any form of employment (defined as competitive employment, transitional employment, sheltered employment or voluntary work);
1.2 in any form of employment or education (defined as above but including people on training courses or full or part‐time education);
1.3 mean hours per month in competitive employment;
1.4 mean monthly earnings.
2. Clinical outcomes:
2.1 numbers lost to follow up (for trials with community or hospital controls only) or numbers not participating in program (for trials comparing different VR approaches);
2.2 admitted to hospital (for trials with a community control) or number living in community at end of study (if a hospital control);
2.3 other clinical outcomes (e.g. symptoms, quality of life and social functioning).
3. Costs:
3.1 mean monthly program costs (direct costs of experimental program versus direct costs of control program);
3.2 mean monthly healthcare costs (including costs of all psychiatric/medical care and program costs, but excluding earnings or transfer costs i.e. benefits obtained).
Search methods for identification of studies
1. Electronic searching
The search began by deriving a list of search terms from reading overviews of the field and consulting experts in vocational rehabilitation.
1.1 CINAHL (January 1982‐December 1998) was searched using the Cochrane Schizophrenia Group's search strategy for randomised controlled trials combined with the phrase:
[(SUPP* EMPLOY*) or (EMPLOYMENT) or (PSYCHOSOCIAL REHAB*) or (PSYCHIATRIC REHAB*) or (OCCUPATIONAL REHAB*) or (SOC* REHAB*) or (WORK REHAB*) or (JOB REHAB*) or (SHELTERED WORK*) or (TRANSITIONAL EMP*) or (REHABILITATION COUNSELLING) or (VOCATION*) or (FOUNTAIN HOUSE*) or (FOUNTAIN‐HOUSE*) or (CLUBHOUSE*) or (CLUB‐HOUSE*). The results of this search were then combined with a search using the major indexing term MENTAL‐DISORDERS.
1.2 The Cochrane Library (Issue 2, 1999) was searched using the phrases:
[(SUPP* EMPLOY*) or (EMPLOYMENT) or (PSYCHOSOCIAL REHAB*) or (PSYCHIATRIC REHAB*) or (OCCUPATIONAL REHAB*) or (SOC* REHAB*) or (WORK REHAB*) or (JOB REHAB*) or (SHELTERED WORK*) or (TRANSITIONAL EMP*) or (REHABILITATION COUNSELLING) or (VOCATION*) or (FOUNTAIN HOUSE*) or (FOUNTAIN‐HOUSE*) or (CLUBHOUSE*) or (CLUB‐HOUSE*) and (MENTAL ILLNESS or SCHIZOPHRENIA)]
1.3 EMBASE (January 1980‐December 1998) was searched using the Cochrane Schizophrenia Group's search strategy for randomised controlled trials combined with the phrase:
[(SUPP* EMPLOY*) or (EMPLOYMENT) or (PSYCHOSOCIAL REHAB*) or (PSYCHIATRIC REHAB*) or (OCCUPATIONAL REHAB*) or (SOC* REHAB*) or (WORK REHAB*) or (JOB REHAB*) or (SHELTERED WORK*) or (TRANSITIONAL EMP*) or (REHABILITATION COUNSELLING) or (VOCATION*) or (FOUNTAIN HOUSE*) or (FOUNTAIN‐HOUSE*) or (CLUBHOUSE*) or (CLUB‐HOUSE*). The results of this search were then combined with a search exploding all sub‐headings of the indexing term MENTAL DISEASE.
1.4 MEDLINE (January 1966‐December 1998) was searched using the Cochrane Schizophrenia Group's search strategy for randomised controlled trials combined with the phrase:
[(SUPP* EMPLOY*) or (EMPLOYMENT) or (PSYCHOSOCIAL REHAB*) or (PSYCHIATRIC REHAB*) or (OCCUPATIONAL REHAB*) or (SOC* REHAB*) or (WORK REHAB*) or (JOB REHAB*) or (SHELTERED WORK*) or (TRANSITIONAL EMP*) or (REHABILITATION COUNSELLING) or (VOCATION*) or (FOUNTAIN HOUSE*) or (FOUNTAIN‐HOUSE*) or (CLUBHOUSE*) or (CLUB‐HOUSE*). The results of this search were then combined with a search exploding all sub‐headings of the indexing term MENTAL DISORDERS.
1.5 PsycLIT (January 1887‐December 1998) was searched using the Cochrane Schizophrenia Group's search strategy for randomised controlled trials combined with the phrase:
[(SUPP* EMPLOY*) or (EMPLOYMENT) or (PSYCHOSOCIAL REHAB*) or (PSYCHIATRIC REHAB*) or (OCCUPATIONAL REHAB*) or (SOC* REHAB*) or (WORK REHAB*) or (JOB REHAB*) or (SHELTERED WORK*) or (TRANSITIONAL EMP*) or (REHABILITATION COUNSELLING) or (VOCATION*) or (FOUNTAIN HOUSE*) or (FOUNTAIN‐HOUSE*) or (CLUBHOUSE*) or (CLUB‐HOUSE*).
This search strategy identified 40 confirmed trials and 13 review articles.
2. Reference searching
The sensitivity of the search strategy was examined by comparing the results of the search with the reference lists of the identified reviews and trials to determine how many cited trials had not been detected. Of three undetected trials cited in the reviews, two were not listed on any of the databases, whilst the third trial was indexed under the term 'DELIVERY OF HEALTH CARE/INTEGRATED'. This term was then added to the search strategy and the search was re‐run, but no further trials were detected. Finally the results of the search were compared against bibliographies from two unpublished PhD theses (Kim 1998, Schneider 1998) but no further trials were detected.
3. Personal contact
Researchers in the field were approached to identify unpublished studies.
Data collection and analysis
1. Selection of studies
The initial electronic search was performed by one reviewer (RC). The list of publications identified by the search strategy was examined by two reviewers working independently (MM, RC). Each reviewer discarded irrelevant publications and retained only those trials in which some form of vocational rehabilitation had been compared against a control treatment. The reviewers then obtained copies of all papers relating to relevant trials. Once these papers had been obtained they were read independently by the two reviewers who decided whether the trials were eligible for the study and allocated them to one of six possible comparisons (Pre‐vocational Training versus hospital control; Pre‐vocational Training versus community control; Supported Employment versus Pre‐vocational Training; Supported Employment versus community control; modifications of vocational rehabilitation programs). Inter‐rater agreement was assessed for overall eligibility and for allocation of trials to comparisons.
2. Quality assessment
MM and RC rated each trial according to the three categories of allocation concealment described in the Cochrane Collaboration Handbook (Clarke 1999): A ‐ adequate (i.e. the method for assigning participants to interventions was robust against patient and clinician bias and clearly described); B ‐ method of allocation concealment unclear; C ‐ inadequate (i.e. the method of assignment was not robust to patient and clinician bias). When the method of allocation concealment was unclear, trialists were contacted for further details. Blinding of patients and treating clinicians is not possible in trials of vocational rehabilitation. It is also difficult for those evaluating outcome to remain blind to group allocation, as they are obliged to collect data that indicate group allocation (such as days in different types of employment). However, trials were rated on independence of evaluators from those providing the intervention.
3. Data extraction
All data were extracted by the two reviewers working alone and then cross‐checked to ensure reliability.
4. Data management
4.1 Missing data
4.1.1 Unacceptable loss to follow‐up: a sensitivity analysis was performed excluding trials where the loss to follow up was greater than 80%. Additionally, amongst included studies, the review did not report data on outcomes where less than 50% of those assessed at baseline failed to be reassessed on the same outcome at follow‐up.
4.1.2 Intention‐to‐treat analysis: it was assumed that patients who were lost to follow up remained unemployed, as suggested by previous research (Bond 1984).
4.2 Dichotomous (i.e. yes/no data)
The relative risk and 95% confidence interval (CI), as well as the number needed to treat (NNT) were calculated for relevant outcomes. The relative risk was chosen over the odds ratio because the latter tends to overestimate effect size when event rates are high (Clarke 1999). The NNT was calculated as the inverse of the absolute risk reduction, and confidence intervals were calculated using the Arcus Quickstat(c) Program.
4.3 Continuous data
Continuous data were reported on MetaView when normally distributed, and when available on the same variable from more than one trial. Otherwise continuous data, including skewed data (see below 4.3.2) and data analysed using non‐parametric methods, were reported in tables or in the text.
4.3.1 Intention‐to‐treat analysis: in the case of continuous data a completer analysis was presented.
4.3.2 Rating scales: data from rating scales were excluded if collected using an unpublished scale, or based on a subset of items from a scale (see Marshall 2000 for justification).
4.3.2 Skewed data: continuous data on mental health outcomes are often not normally distributed (i.e. skewed). It may not be appropriate to analyse such data using parametric methods, such as those used by MetaView. In this review the degree of skew of continuous data was assessed by multiplying the standard deviation by 1.96. If the result was less than the mean then the data were entered on MetaView, otherwise they were reported in the text or in tables (Altman 1996). Data analysed using non‐parametric statistics were also reported in tables.
4.3.3 Conversion of data: data were reported as presented in the original studies, with two exceptions. First, continuous variables such as costs or days in employment were converted to a single common scale (such as mean days in employment per month or mean monthly costs) in order to facilitate comparisons. Second, number of clients not participating in the program was estimated by taking the number of clients who were not re‐interviewed at the final follow‐up assessment, or by taking actual non‐participation rates (where these were given in the trial report and were greater than the number not re‐interviewed). Clients were not counted as not participating if the reason for non‐participation was that they were in an alternative work or educational placement.
5. Sub‐analyses
Two sub‐types of Pre‐vocational Training and Supported Employment (the Clubhouse and IPS models, respectively) have been sufficiently specified to be regarded as approaches in their own right (see above for details). Data from trials using these approaches were included in the main Pre‐vocational Training and Supported Employment comparisons, but were also presented separately as sub‐analyses.
6. Heterogeneity
Heterogeneity between trial results was assessed by inspection of graphical presentations and by calculating a Chi squared test of heterogeneity. If heterogeneity was present (p value of Chi squared <0.1) the data were re‐analysed using a random effects model. If heterogeneity was still present, the summary relative risk was interpreted cautiously, and efforts were made to identify the source of the heterogeneity.
7. Addressing publication bias
Data from all identified and selected trials were entered into a funnel graph (trial effect versus trial size) in an attempt to investigate overt publication bias.
8. Tables and figures
The data were recorded on RevMan so that the area to the left of the 'line of no effect' indicated a 'favourable' outcome for the first intervention mentioned in the title of the comparison. For example, in trials comparing Supported employment to Pre‐vocational Training, an outcome to the left of the 'line of no effect' would indicate a favourable outcome for Supported Employment, whereas an outcome to the right would indicate a favourable outcome for Pre‐vocational Training .
Results
Description of studies
1. Excluded studies
Thirty‐one studies were excluded (please see 'Table of excluded studies'); seventeen of these were not randomised, but fourteen were classified as randomised controlled trials. The non‐randomised studies consisted of: one survey (without comparison group); three cross‐sectional comparisons; five uncontrolled 'before and after' comparisons, and eight quasi‐experimental designs (i.e. comparative trials where no attempt was made to randomise). The excluded randomised controlled trials consisted of: five trials of Pre‐vocational Training versus standard care (in two the number of participants with mental illness was unclear, and in three the data could not be analysed on an intention‐to‐treat basis); four trials of modifications of Pre‐vocational Training versus Pre‐vocational Training (in three the number of participants was unclear, whilst the remaining trial was concerned with increasing productivity rather than helping patients find work); and five trials of approaches to community care that did not involve any specific vocational interventions (three of assertive community treatment and two others), although the trials happened to report employment rates. Of the trials where data could not be analysed on an intention‐to‐treat basis: in Briggs‐Minnesota the number of participants was unclear; in Kaufman‐Pittsburgh the numbers randomised to treatment and control groups were not given; and in Ryan‐Connecticut there was substantial exclusion of treatment group participants post‐randomisation (for example, any client who failed to complete three months in the Pre‐vocational Training group was excluded).
2. Ongoing studies
Two ongoing studies were identified, including one substantial multi‐centre study of Supported Employment versus Pre‐vocational Training (Carey‐US 8 site).
3. Awaiting assessment
There were four studies awaiting assessment: three were published in books that were difficult to obtain (one of which was in Dutch) and one was unpublished.
4. Included studies
Eighteen trials met inclusion criteria for the review (see 'Table of included studies' for full details).
4.1 Participants
People with schizophrenia are well represented in the trials of Pre‐vocational Training versus Supported Employment (weighted means of 52.4 and 78.7%). Women were well represented. There were insufficient data to assess representation of people from ethnic minorities. Women, people from ethnic minorities, and people with schizophrenia were well represented in the trials of Supported Employment versus Pre‐vocational Training (weighted means of 49.8%, 37.9% and 60.2% respectively).
4.2 Interventions
4.2.1 Pre‐vocational training: Becker‐Fort Worth compared care on a specialised rehabilitation ward with an integral vocational program against continuing in‐patient rehabilitation. Kuldau‐California compared a rehabilitation program involving sheltered work, an in‐patient therapeutic community and transitional housing against a control involving standard hospital care with rapid discharge planning. Walker‐Massachusetts compared day placement in an out‐of‐hospital industrial therapy unit with standard hospital care. Beard‐NewYork compared the 'Clubhouse' model of vocational rehabilitation to standard community care. Dincin‐Chicago compared the 'Thresholds' program, involving work crews and transitional employment to standard community care, including six hours of supportive psychotherapy and fortnightly consultations with a psychiatrist. Griffiths‐London compared a rehabilitation programme involving day hospital and industrial workshops against standard community care involving home support and day centres. Okpaku‐Nashville compared employment‐oriented case management involving work assessment and employment preparation against standard case management. Wolkon‐Cleveland compared individual counselling and transitional work to standard community care.
4.2.2 Modifications of pre‐vocational training: Bell‐Connecticut modified pre‐vocational training and examined the effect of payment on uptake of sheltered set‐aside jobs in a hospital. Blankertz‐Philadelph and Kline‐Philadelphia examined the effects of psychological interventions (designed to increase motivation) on the uptake of community vocational rehabilitation services. Control groups received vocational rehabilitation services but no psychological intervention. Bond‐Chicago1 compared a graded approach (of experience in work crews leading to transitional employment), with an accelerated approach involving immediate placement in transitional employment.
4.2.3 Supported employment: Chandler‐LongBeach compared Assertive Community Treatment combined with Supported Employment against standard community care (not involving Assertive Community Treatment). In the studies of Supported Employment versus Pre‐vocational Training, Bond‐Indiana compared Supported Employment with Pre‐vocational work‐readiness training. Drake‐New Hampshire1 compared the Individual Placement and Support model of Supported Employment with a brokered model of Pre‐vocational Training. Drake‐Washington compared the Individual Placement and Support Model with Pre‐vocational counselling and work adjustment training in a sheltered workshop. Gervey‐New York compared Supported Employment with employment training in a sheltered workshop. McFarlane‐New York compared Family‐aided Assertive Community Treatment plus Supported Employment with conventional Pre‐vocational Training from the local vocational rehabilitation service.
4.3 Outcome Scales
Rating scales used to measure clinical outcomes are listed below.
4.3.1 Global Outcome
GAS (Endicott 1976). A clinician rated scale of overall functioning on a scale of 1‐100. Lower scores indicate poorer functioning.
4.3.2 Mental State
The Positive and Negative Symptom Scale ‐ PANSS (Kay 1987)
This schizophrenia scale has 30 items, each of which can be defined on a seven‐point scoring system varying from one ‐ absent to seven ‐ extreme. This scale can be divided into three sub‐scales for measuring the severity of general psychopathology, positive symptoms (PANSS‐P), and negative symptoms (PANSS‐N). A low score indicates lesser severity.
The Brief Psychiatric Rating Scale ‐ BPRS (Lukoff 1986)
This is used to assess the severity of abnormal mental state. The original scale has 16 items, but a revised 18‐item scale is commonly used. Each item is defined on a seven‐point scale varying from 'not present' to 'extremely severe', scoring from 0‐6 or 1‐7. Scores can range from 0‐126, with high scores indicating more severe symptoms.
4.3.2 Others
Self‐confidence scale (Wing 1966)
No details were available on this scale, and the original reference was difficult to obtain.
Self‐esteem (Rosenberg 1969)
This scale is a 10‐item self‐report measure. Each item involves a statement about how the respondent feels about him or herself ('I feel that I have a number of good qualities') or aspects of his or her functioning ('I feel that I can't do anything right'). Respondents rate each item on a Likert scale from 'almost always true' to 'never true'. Lower scores indicate higher self‐esteem.
Quality of Life Scale (Lehman 1983)
This standardised assessment includes areas such as living situation, leisure activities, relationships and finances. Rated on a seven‐point scale, with higher scores indicating a better quality of life.
Risk of bias in included studies
1. Randomisation
The quality of allocation concealment in included trials was as follows: seven trials were in randomisation category A (adequate) (Bond‐Chicago1, Bond‐Indiana, Dincin‐Chicago, Drake‐New Hampshire1, Drake‐Washington, Kuldau‐California, Walker‐Massachusetts); nine were in category B (unclear) (Becker‐Fort Worth, Bell‐Connecticut, Blankertz‐Philadelph, Chandler‐LongBeach, Griffiths‐London, Kline‐Philadelphia, McFarlane‐New York, Okpaku‐Nashville, Wolkon‐Cleveland); and two were category C (inadequate) (Beard‐NewYork, Gervey‐New York). Of the trials in category C: in Beard‐NewYork allocation was by day of referral and in Gervey‐New York allocation was by drawing lots from a hat.
2. Objectivity of rating of outcome
In eight trials outcome was assessed by raters who were not involved in providing the treatment or control interventions (Bell‐Connecticut, Chandler‐LongBeach, Drake‐New Hampshire1, Drake‐Washington, Gervey‐New York, McFarlane‐New York, Okpaku‐Nashville, Wolkon‐Cleveland). In ten trials it was either unclear how far raters were independent (Becker‐Fort Worth, Griffiths‐London, Kline‐Philadelphia, Kuldau‐California), or it was clear that they were not independent (Beard‐NewYork, Blankertz‐Philadelph, Bond‐Chicago1, Bond‐Indiana, Dincin‐Chicago, Walker‐Massachusetts).
3. Description of loss to follow up
Follow up rates were generally good: 16 trials had loss to follow up rates of 20% or less (Beard‐NewYork, Becker‐Fort Worth, Bell‐Connecticut, Blankertz‐Philadelph, Bond‐Chicago1, Bond‐Indiana, Drake‐New Hampshire1, Drake‐Washington, Gervey‐New York, Griffiths‐London, Kline‐Philadelphia, Kuldau‐California, McFarlane‐New York, Okpaku‐Nashville, Walker‐Massachusetts, Wolkon‐Cleveland). Loss to follow up rates of greater than 20% were found in the following trials: Chandler‐LongBeach (21% at one year); Dincin‐Chicago (37% at nine months);
4. Confounding of interventions
There was confounding of the intervention in two trials. In Chandler‐LongBeach experimental patients received Assertive Community Treatment in addition to Supported Employment, whilst in McFarlane‐New York experimental patients received Family‐aided Assertive Community Treatment in addition to Supported Employment.
5. Sensitivity analysis
In the initial analyses data from all included trials, regardless of quality, were analysed within the relevant comparisons. Subsequently, these analyses were repeated excluding data from trials with: (a) allocation concealment in categories B or C; (b) non‐independent evaluators; (c) follow up rates of less than 80%; (d) confounding of interventions. As it turned out, only two trials, both of Supported Employment versus Pre‐vocational Training (Drake‐New Hampshire1, Drake‐Washington), met criteria for inclusion in the sensitivity analysis.
Effects of interventions
1. Reliability of data extraction and funnel plot
There were no disagreements between the raters on which trials should be discarded on the basis of abstracts obtained from the electronic search. Inter‐rater reliability for inclusion of trials in the review, once full text had been obtained, based on a sample of 20 trials, was 0.89. There was full agreement between raters on categorisation of included trials. There were insufficient data to draw funnel plots in most comparisons, however it was possible to draw a funnel plot for the variable 'not in competitive employment at 12 months' in the comparison: 'Supported Employment versus Pre‐vocational Training'. This plot showed evidence of asymmetry attributable to Gervey‐New York, which found a large effect size in favour of Supported Employment. It was not clear whether the asymmetry in the funnel plot indicated publication bias or whether it was due to inadequate allocation concealment in this trial. Gervey‐New York was not eligible for the sensitivity analysis.
2. PRE‐VOCATIONAL TRAINING versus STANDARD HOSPITAL CARE
Three trials, with a total of 200 patients, contributed data to this comparison (Becker‐Fort Worth, Kuldau‐California, Walker‐Massachusetts).
2.1 Not in competitive employment
Few data were available on the primary outcome. One small trial (Becker‐Fort Worth) reported data at eight month follow up which showed a non‐significant trend in favour of clients in Pre‐vocational Training (n=50, RR 0.79 CI 0.63 to 1.00).
2.2 Secondary employment outcomes
Becker‐Fort Worth reported that at eight months, a significantly larger number of clients in Pre‐vocational Training had obtained any form of employment (n=50, RR 0.42 CI 0.26 to 0.68, NNT 1.8). Walker‐Massachusetts, however, reported no difference in hours/month in competitive employment (n=28, Pre‐vocational Training mean 36.8, control 31.6, p=0.92, Mann Whitney). Kuldau‐California reported that Pre‐vocational Training clients earned significantly more dollars per month than controls (Pre‐vocational Training mean $176.2, control mean $97.3, p <.01). There was a non‐significant trend towards better participation amongst Pre‐vocational Training clients (n=78, RR 0.5 CI 0.05 to 5.25).
2.3 Clinical outcomes
The limited data available suggested that clients in Pre‐vocational Training were not more likely to be discharged from hospital (n=50, RR 0.95 CI 0.76 to 1.19).
2.4 Sensitivity analysis
No trials met criteria for inclusion in the sensitivity analysis (see Methodological Quality of Included Studies, section 5 above).
3. PRE‐VOCATIONAL TRAINING (ALL APPROACHES) versus COMMUNITY CARE
Five trials involving a total of 1204 patients contributed data to this comparison (Beard‐NewYork, Dincin‐Chicago, Griffiths‐London, Okpaku‐Nashville, Wolkon‐Cleveland).
3.1 Not in competitive employment
Some limited data (Griffiths‐London and Beard‐NewYork) were available on the primary outcome at 18 and 24 months. These showed no difference between Pre‐vocational Training and control (18 months n=28, RR 1.18 CI 0.87 to 1.61; 24 months n=215, RR 0.95 CI 0.77 to 1.17).
3.2 Secondary employment outcomes
Three trials reported data on number in any form of employment. These data showed no difference between Pre‐vocational Training and control at three, six, nine, 12 and 18 months.
3.3 Clinical outcomes
Data from two trials (Dincin‐Chicago, Okpaku‐Nashville) showed no significant difference in the number of clients participating in the program (n=284, RR random effects 0.95 CI 0.52 to 1.7) between Pre‐vocational Training and control groups. Data from three trials (Beard‐NewYork, Dincin‐Chicago, and Wolkon‐Cleveland) showed that significantly fewer patients were admitted to hospital amongst those receiving Pre‐vocational Training (N= 887, RR 0.79 CI 0.65 to 0.95). However, heterogeneity was present on this outcome and re‐analysis using a random effects model failed to show a significant difference (RR random effects 0.71 CI 0.48 to 1.04). Griffiths‐London reported no difference in self‐esteem (Self‐confidence scale, Wing 1966) between Pre‐vocational Training and control groups (n=28, Pre‐vocational Training mean 25.5, SD 6.6, control mean 23.3, SD 7.3).
3.4 Costs
One trial (Dincin‐Chicago) reported mean monthly total healthcare costs of $417.90 for Pre‐vocational Training and $651.50 for controls, but no statistical analysis was reported.
3.5 Sensitivity Analysis
No trials met criteria for inclusion in the sensitivity analysis (see Methodological Quality section 5 above).
4. SUB‐ANALYSIS 1: CLUBHOUSE APPROACH (TYPE OF PRE‐VOCATIONAL TRAINING) versus STANDARD COMMUNITY CARE
Only one trial (Beard‐NewYork) provided data for this sub‐analysis.
4.1 Not in competitive employment
On the primary outcome at 24 months there was no difference between people allocated to Clubhouse approach in addition to standard care and those in the control group (n=215, RR 0.95 CI 0.77 to 1.17).
4.2 Secondary employment outcomes
Beard‐NewYork showed no significant difference between the Clubhouse approach and control in numbers obtaining any form of employment at three, six and 12 months.
4.3 Clinical outcomes
Beard‐NewYork found significantly fewer admissions to hospital for patients amongst clients in the Clubhouse group (N=215, RR 0.69 CI 0.46 to 0.96, NNT 6.1).
5. MODIFICATION 1. PRE‐VOCATIONAL TRAINING + PAYMENT versus PRE‐VOCATIONAL TRAINING ALONE
One trial (Bell‐Connecticut) provided data for this comparison.
5.1 Not in competitive employment
No data were available on the primary outcome.
5.2 Secondary employment outcomes
At six month follow‐up significantly more clients in the payment group were in any form of employment (n=150, RR 0.40 CI 0.28 to 0.57, NNT 2.2). Clients in the payment group earned significantly more per month (payment group mean $192, non‐payment group mean $32.03, t=7.56, p<0.0001).
5.3 Clinical outcomes
Significantly more clients from the payment group participated in the programme (n=150, RR 0.53 CI 0.39 to 0.71, NNT 2.8). There were also significantly fewer admissions to hospital in the payment group (RR 0.55 CI 0.31 to 0.96, NNT 6.4) and they showed significantly better total symptom scores (PANSS, Kay 1987) (payment mean 66.2, SD 15.1, non‐payment mean 72.6, SD 15.0. p<0.02).
6. MODIFICATION 2. PRE‐VOCATIONAL TRAINING + PSYCHOLOGICAL INTERVENTIONS versus PRE‐VOCATIONAL TRAINING ALONE
Two trials (Kline‐Philadelphia, Blankertz‐Philadelph) provided data for this comparison.
6.1 Not in competitive employment
On the primary outcome at six to nine month follow up there was a difference in favour of Pre‐vocational Training and psychological intervention (n=142, RR 0.86 CI 0.78 to 0.95, NNT 7.1). However, there was evidence of heterogeneity on this variable (Chi squared test 3.12, p=0.077): both trials found an effect in favour of the intervention, but the effect size was larger in the smaller trial (Kline‐Philadelphia n=20, RR 0.56 CI 0.29 to 1.07; Blankertz‐Philadelph n=122, RR 0.90 CI 0.83 to 0.98). Reanalysis using a random effects model found no significant difference (RR 0.76 CI 0.44 to 1.33).
6.2 Secondary employment outcomes
One trial (Blankertz‐Philadelph) found that clients receiving psychological intervention were significantly more likely to be in some form of employment (n=122, RR 0.89 CI 0.81 to 0.97, NNT 8.7) or in some form of employment, training or education at the end of the study (n=122, RR 0.63 CI 0.52 to 0.77, NNT 2.8).
6.3 Clinical outcomes
Both trials reported data on numbers not participating in the programme, but found no significant difference between intervention and control groups (n=142, RR 0.85 CI 0.33 to 2.18).
6.4 Sensitivity Analysis
Neither trial met criteria for inclusion in the sensitivity analysis (see Methodological Quality section 5 above).
7. MODIFICATION 3. ACCELERATED ENTRY TO TRANSITIONAL EMPLOYMENT (A TYPE OF PRE‐VOCATIONAL TRAINING) versus GRADUAL ENTRY TO TRANSITIONAL EMPLOYMENT
One trial (Bond‐Chicago1) provided data for this comparison.
7.1 Not in competitive employment
On the primary outcome there was no difference between groups at nine and 15 months (although there was a result in favour of accelerated placement, that fell just sort of significance at 15 months, n=131, RR 0.88 CI 0.78 to 1.0).
7.2 Secondary employment outcomes
Clients in the accelerated condition were not more likely to be in any form of employment at 15 months (n=131, RR 0.96 CI 0.69 to 1.33), but earned more per month (accelerated condition mean $115.3, control mean $38.9, no statistical analysis).
7.3 Clinical outcomes
There was no difference in participation rates between the two groups at nine or 15 months.
8. SUPPORTED EMPLOYMENT versus STANDARD COMMUNITY CARE
One trial, involving 256 patients, contributed data to this comparison (Chandler‐LongBeach).
8.1 Not in competitive employment
On the primary outcome there was no difference between Supported Employment and control at 12 months (n=256, RR 1.01 CI 0.93 to 1.09), but there was a significant difference favouring Supported Employment at 24 months (n=256, RR 0.92 CI 0.85 to 0.99, NNT 12.6) and 36 months (n=256, RR 0.88 CI 0.82 to 0.96, NNT 9).
8.2 Secondary employment outcomes
Supported Employment clients were significantly more likely to be in any form of employment at 12 months (n=256, RR 0.79 CI 0.70 to 0.90, NNT 5.5) and also earned significantly more per month (Supported Employment mean $60.5, control mean $26.9, p<0.05).
8.3 Clinical outcomes
There was no significant difference in participation rates between Supported Employment and control, although there was a result favouring Supported Employment (n=256, RR 0.74, CI 0.55 to 1.01). There was no difference in the number of hospital admissions between Supported Employment and control (n=256, RR 0.83 CI 0.63 to 1.10).
8.4 Costs
Mean monthly healthcare costs were significantly higher for clients in Supported Employment (Supported Employment mean $1599, control mean $527.30), but this finding was difficult to interpret as Supported Employment clients were also receiving Assertive Community Treatment.
9. SUPPORTED EMPLOYMENT (ALL APPROACHES) versus PRE‐VOCATIONAL TRAINING
Five trials, involving 484 patients, contributed data to this comparison (Drake‐New Hampshire1, Drake‐Washington, Bond‐Indiana, Gervey‐New York, McFarlane‐New York).
9.1 Not in competitive employment
On the primary outcome there was a difference in favour of supported employment at four, six, nine, 12, 15 and 18 months, e.g. at four months n=364, RR random effects 0.75 CI 0.64 to 0.89, and at 12 months, n=484, RR random effects 0.76 CI 0.64 to 0.89, NNT 4.5 CI 4.48 to 4.63. At 12 months 34% of clients were employed in the Supported Employment group, but only 12% in the Pre‐vocational Training group. There was no significant heterogeneity on this variable at any time point.
9.2 Secondary employment outcomes
Three trials found that clients in Supported Employment had significantly more hours per month in competitive employment than those receiving Pre‐vocational Training (Table 01). Three of four trials also found that clients in Supported Employment had higher mean monthly earnings that those in the Pre‐vocational Training (Table 02).
9.3 Clinical outcomes
There was no significant difference in participation rates between Supported Employment and control at six, 12, and 18 months (12 month data analysed using random effects model due to heterogeneity). Drake‐New Hampshire1 reported no difference in overall functioning (General Assessment Scale, Endicott 1976), self‐esteem (Rosenberg scale, Rosenberg 1969) or mental state (BPRS scale, Lukoff 1986), but did not report any raw data. Drake‐Washington reported no significant differences at six, 12 and 18 months in: (a) Global outcome (GAS, e.g. at 18 months Supported Employment 45.8 (SE 1.43), control 46.0 (SE 1.78)); (b) Self‐esteem (Rosenberg Scale, e.g. at 18 months Supported Employment 18.5 (SE 0.7), control 18.1 (SE 0.68), (c) Quality of Life (Lehman's scale, Lehman 1983, e.g. at 18 months Supported Employment 5 (SE 0.17), control 4.8 (SE 0.18)) or (d) psychiatric symptoms (BPRS, e.g. at 18 months Supported Employment 39.2 (SE 1.19), control 41.1(SE 1.54)).
9.4 Costs
Bond‐Indiana reported that the programme costs of Supported Employment were greater than for Pre‐vocational Training, but that other health care costs were reduced (no statistical analysis), so that overall health care costs were less for Supported Employment. Drake‐New Hampshire1 found no significant difference in program costs or overall health care costs between Supported Employment and Pre‐vocational Training (Table 03).
9.5 Sensitivity Analysis
Two trials (Drake‐New Hampshire1, Drake‐Washington) met criteria for inclusion in the sensitivity analysis. As these trials were the only trials included in Sub‐analysis 2 (see item 10, below) the results of the sensitivity analysis were the same as those of Sub‐analysis 2 and were similar to those of the analysis involving all five trials.
10. SUB‐ANALYSIS 2: INDIVIDUAL PLACEMENT & SUPPORT (IPS ‐ TYPE OF SUPPORTED EMPLOYMENT) versus PRE‐VOCATIONAL TRAINING
Two trials (Drake‐New Hampshire1, Drake‐Washington) provided data for this comparison.
10.1 Not in competitive employment
On the primary outcome there was a difference in favour of IPS clients at four, six, nine, 12, 15 and 18 months. For example, at four months, n=295, RR 0.7 CI 0.6 to 0.8, and at 12 months n=295, RR 0.79 CI 0.70 to 0.89, NNT 5.5. At 12 months 30% of people allocated to IPS were employed as against 12% in Pre‐vocational Training.
10.2 Secondary employment outcomes
Both trials reported that IPS clients spent significantly more hours per month in competitive employment (Table 1). One trial (Drake‐New Hampshire1) reported significantly higher mean monthly earnings, but the other (Drake‐Washington) found no difference, (although the IPS grouped earned more from competitive employment ‐ see Table 2).
| Study | Intervention | Mean monthly hrs | t (or F) | p |
| Drake‐NH | IP | 33.7 | 3.7 | <0.001 |
| PVT | 11.4 | |||
| Drake‐Wash | IP | 17.9 | 4.4 | <0.001 |
| PVT | 1.5 | |||
| Gervey | IP | 69 | 3.7 | 0.03 |
| PVT | 9.9 |
| Study | Intervention | Mean earnings | t or F | p |
| Bond‐Indiana | SE | 127.1 | 2.55 | <0.05 |
| PVT | 71.7 | |||
| McFarlane‐New York | SE | 41.9 | 2.35 | 0.019 |
| PVT | 11.8 | |||
| Drake‐NH1 | SE | 188.5 | 3.34 | <0.001 |
| PVT | 59.9 | |||
| Drake‐Wash | SE | 111.1 | 4.29 | NS |
| PVT | 111.4 |
10.3 Clinical outcomes
IPS clients were not significantly more likely to participate than control clients, although confidence intervals were wide and there was a trend favouring Supported Employment (n=295, RR 0.52 CI 0.15 to 1.85, random effects model). There were no significant differences between groups on: self‐esteem; mental state; overall functioning; or quality of life at any time point (see 9.3 above).
10.4 Costs
Drake‐New Hampshire1 found no significant difference in program costs or overall health care costs between IPS and Pre‐vocational Training (Table 3).
| Study | Group | Program costs | Other health costs | Overall costs |
| Bond‐Indiana | Immediate Placement | $251.6 | $263.0 | $514.6 |
| Control | $132.0 | $586.5 | $718.5 | |
| Drake‐NH1 | Immediate Placement | $313.1 | $801.6 | $1114.7 |
| Control | $307.3 | $928.5 | $1235.8 |
Discussion
1. General
1.1 Methodological limitations affecting the ability to detect improved outcomes
The review found little evidence that Supported Employment or Pre‐vocational Training improved symptoms, quality of life or social functioning. This finding is difficult to interpret, however, as only a minority of participants in vocational rehabilitation trials actually find competitive employment (about one third in the most effective Individual Placement and Support trials). Therefore, a large sample would be required to detect clinically significant improvements. There were some indications that this problem was masking symptomatic improvements amongst those people who actually worked. For example, Bell‐Connecticut found a significant improvement in symptoms, after financial inducements had ensured a high participation rate in the treatment group, whilst Drake‐New Hampshire1 reported a sub‐analysis of mental state data showing a significant improvement in clients who obtained competitive work.
1.2 Generalisability
There was no evidence that vocational rehabilitation trials were 'cherry‐picking' clients who were likely to be easy to place in employment. Thus a weighted average of participants in Supported Employment versus Pre‐vocational Training trials showed good recruitment of women and ethnic minorities, with a majority of participants suffering from schizophrenia (see Included Studies table). This suggests that the findings of the review can be applied with confidence to the general population of clients with severe mental disorder. The review is however limited by the fact that all trials (bar one) were conducted in the United States. This limitation makes it uncertain how far the findings can be generalised to countries with less dynamic economies, different welfare structures, or dissimilar cultural attitudes to work.
2. PRE‐VOCATIONAL TRAINING versus STANDARD COMMUNITY CARE OR HOSPITAL CARE
2.1 Employment outcomes
The review found no evidence to suggest that Pre‐vocational Training was more effective on the primary outcome than standard community care or hospital care. This was supported by findings on other secondary employment outcomes (although Pre‐vocational Training performed slightly better on some secondary outcomes when compared against a hospital control group).
2.2 Clinical outcomes and costs
Clients were not more likely to engage in Pre‐vocational Training than standard care. Whilst clients in Pre‐vocational Training programs appeared less likely to be admitted to hospital than clients receiving standard community care, heterogeneity was present on this outcome and reanalysis using a random effects model found no significant difference, although the result is borderline (RR random effects 0.71 CI 0.48 to 1.04). There were only limited data on costs.
2.3 Sub‐analysis
This showed that there was insufficient evidence to judge whether the Clubhouse approach was more effective than other approaches to Pre‐vocational Training.
2.4 Modifications of Pre‐vocational Training
There was some evidence that payment improved engagement in Pre‐vocational Training and enhanced its effectiveness. The effect of psychological interventions to enhance motivation was less certain, although there were some promising indications.
3. SUPPORTED EMPLOYMENT versus PRE‐VOCATIONAL TRAINING
3.1 Employment outcomes
The main finding of the review was that on the primary outcome, finding competitive employment, Supported Employment was superior to Pre‐vocational Training. Evidence supporting this finding was strong: five randomised trials (n=484) showed that people in Supported Employment were significantly more likely to be in competitive employment at six time points across 18 months. There was no evidence of heterogeneity at any time point. A sensitivity analysis excluding all but the two highest quality trials did not substantially alter this finding. Secondary outcomes such as mean hours worked and mean monthly earnings favoured Supported Employment.
3.2 Clinical outcomes and costs
Data were inconclusive, but suggested no major differences between Supported Employment and Pre‐vocational Training.
3.3 Sub‐analysis
Data suggested that Individual Placement and Support was an effective form of Supported Employment, but were insufficient to say whether it was more effective than other less carefully specified forms of Supported Employment.
Only one trial (Chandler‐LongBeach) compared Supported Employment to standard community care. Although this trial suggested that Supported Employment was superior to standard community care, its findings are difficult to interpret as the intervention group received Assertive Community Treatment in addition to Supported Employment.
Comparison 1 PRE‐VOCATIONAL TRAINING versus STANDARD HOSPITAL CARE, Outcome 1 Not in competitive employment (at 8 months).
Comparison 1 PRE‐VOCATIONAL TRAINING versus STANDARD HOSPITAL CARE, Outcome 2 Not in any form of employment (at 8 months).
Comparison 1 PRE‐VOCATIONAL TRAINING versus STANDARD HOSPITAL CARE, Outcome 3 Not participating in program (excluding employed).
Comparison 1 PRE‐VOCATIONAL TRAINING versus STANDARD HOSPITAL CARE, Outcome 4 Not discharged from hospital (at 8 months).
Comparison 2 PRE‐VOCATIONAL TRAINING (ALL APPROACHES) versus STANDARD COMMUNITY CARE, Outcome 1 Not in competitive employment.
Comparison 2 PRE‐VOCATIONAL TRAINING (ALL APPROACHES) versus STANDARD COMMUNITY CARE, Outcome 2 Not in any form of employment.
Comparison 2 PRE‐VOCATIONAL TRAINING (ALL APPROACHES) versus STANDARD COMMUNITY CARE, Outcome 3 Not participating in program (excluding employed).
Comparison 2 PRE‐VOCATIONAL TRAINING (ALL APPROACHES) versus STANDARD COMMUNITY CARE, Outcome 4 Admitted to hospital (by 1 year).
Comparison 3 SUB‐ANALYSIS 1: CLUBHOUSE APPROACH (TYPE OF PVT) versus STANDARD COMMUNITY CARE, Outcome 1 Not in competitive employment (at 24 months).
Comparison 3 SUB‐ANALYSIS 1: CLUBHOUSE APPROACH (TYPE OF PVT) versus STANDARD COMMUNITY CARE, Outcome 2 Not in any form of employment.
Comparison 3 SUB‐ANALYSIS 1: CLUBHOUSE APPROACH (TYPE OF PVT) versus STANDARD COMMUNITY CARE, Outcome 3 Admitted to hospital in first year of study.
Comparison 4 MODIFICATION 1. PRE‐VOCATIONAL TRAINING + PAYMENT versus PRE‐VOCATIONAL TRAINING ALONE, Outcome 1 Not in any form of employment (at 6 months).
Comparison 4 MODIFICATION 1. PRE‐VOCATIONAL TRAINING + PAYMENT versus PRE‐VOCATIONAL TRAINING ALONE, Outcome 2 Not participating in program.
Comparison 4 MODIFICATION 1. PRE‐VOCATIONAL TRAINING + PAYMENT versus PRE‐VOCATIONAL TRAINING ALONE, Outcome 3 Admitted to hospital during first year of study.
Comparison 5 MODIFICATION 2. PRE‐VOCATIONAL TRAINING + PSYCHOLOGICAL INTERVENTIONS versus PRE‐VOCATIONAL TRAINING ALONE, Outcome 1 Not in competitive employment.
Comparison 5 MODIFICATION 2. PRE‐VOCATIONAL TRAINING + PSYCHOLOGICAL INTERVENTIONS versus PRE‐VOCATIONAL TRAINING ALONE, Outcome 2 Not in any form of employment.
Comparison 5 MODIFICATION 2. PRE‐VOCATIONAL TRAINING + PSYCHOLOGICAL INTERVENTIONS versus PRE‐VOCATIONAL TRAINING ALONE, Outcome 3 Not in any form or employment or training or education at end of study.
Comparison 5 MODIFICATION 2. PRE‐VOCATIONAL TRAINING + PSYCHOLOGICAL INTERVENTIONS versus PRE‐VOCATIONAL TRAINING ALONE, Outcome 4 Not participating in program.
Comparison 6 MODIFICATION 3. ACCELERATED ENTRY TO TRANSITIONAL EMPLOYMENT (TE ‐ TYPE OF PVT) versus GRADUAL ENTRY TO TE, Outcome 1 Not in competitive employment.
Comparison 6 MODIFICATION 3. ACCELERATED ENTRY TO TRANSITIONAL EMPLOYMENT (TE ‐ TYPE OF PVT) versus GRADUAL ENTRY TO TE, Outcome 2 Not in any form of employment (at 15 months).
Comparison 6 MODIFICATION 3. ACCELERATED ENTRY TO TRANSITIONAL EMPLOYMENT (TE ‐ TYPE OF PVT) versus GRADUAL ENTRY TO TE, Outcome 3 Numbers not participating in program.
Comparison 6 MODIFICATION 3. ACCELERATED ENTRY TO TRANSITIONAL EMPLOYMENT (TE ‐ TYPE OF PVT) versus GRADUAL ENTRY TO TE, Outcome 4 Number readmitted to hospital (at about 15 months).
Comparison 7 SUPPORTED EMPLOYMENT versus STANDARD COMMUNITY CARE, Outcome 1 Not in competitive employment.
Comparison 7 SUPPORTED EMPLOYMENT versus STANDARD COMMUNITY CARE, Outcome 2 Not in any form of employment.
Comparison 7 SUPPORTED EMPLOYMENT versus STANDARD COMMUNITY CARE, Outcome 3 Numbers not participating in program.
Comparison 7 SUPPORTED EMPLOYMENT versus STANDARD COMMUNITY CARE, Outcome 4 Numbers admitted to hospital during study.
Comparison 8 SUPPORTED EMPLOYMENT (ALL APPROACHES) versus PRE‐VOCATIONAL TRAINING, Outcome 1 Not in competitive employment.
Comparison 8 SUPPORTED EMPLOYMENT (ALL APPROACHES) versus PRE‐VOCATIONAL TRAINING, Outcome 2 Not in any form of employment.
Comparison 8 SUPPORTED EMPLOYMENT (ALL APPROACHES) versus PRE‐VOCATIONAL TRAINING, Outcome 3 Numbers not participating in program.
Comparison 9 SUB‐ANALYSIS 2: INDIVIDUAL PLACEMENT & SUPPORT (TYPE OF SUPPORTED EMPLOYMENT) versus PRE‐VOCATIONAL TRAINING, Outcome 1 Not in competitive employment.
Comparison 9 SUB‐ANALYSIS 2: INDIVIDUAL PLACEMENT & SUPPORT (TYPE OF SUPPORTED EMPLOYMENT) versus PRE‐VOCATIONAL TRAINING, Outcome 2 Numbers not participating in program.
| Study | Intervention | Mean monthly hrs | t (or F) | p |
| Drake‐NH | IP | 33.7 | 3.7 | <0.001 |
| PVT | 11.4 | |||
| Drake‐Wash | IP | 17.9 | 4.4 | <0.001 |
| PVT | 1.5 | |||
| Gervey | IP | 69 | 3.7 | 0.03 |
| PVT | 9.9 |
| Study | Intervention | Mean earnings | t or F | p |
| Bond‐Indiana | SE | 127.1 | 2.55 | <0.05 |
| PVT | 71.7 | |||
| McFarlane‐New York | SE | 41.9 | 2.35 | 0.019 |
| PVT | 11.8 | |||
| Drake‐NH1 | SE | 188.5 | 3.34 | <0.001 |
| PVT | 59.9 | |||
| Drake‐Wash | SE | 111.1 | 4.29 | NS |
| PVT | 111.4 |
| Study | Group | Program costs | Other health costs | Overall costs |
| Bond‐Indiana | Immediate Placement | $251.6 | $263.0 | $514.6 |
| Control | $132.0 | $586.5 | $718.5 | |
| Drake‐NH1 | Immediate Placement | $313.1 | $801.6 | $1114.7 |
| Control | $307.3 | $928.5 | $1235.8 |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Not in competitive employment (at 8 months) Show forest plot | 1 | 50 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.79 [0.63, 1.00] |
| 2 Not in any form of employment (at 8 months) Show forest plot | 1 | 50 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.42 [0.26, 0.68] |
| 3 Not participating in program (excluding employed) Show forest plot | 2 | 78 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.33 [0.01, 7.55] |
| 4 Not discharged from hospital (at 8 months) Show forest plot | 1 | 50 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.95 [0.76, 1.19] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Not in competitive employment Show forest plot | 2 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 1.1 at 18 months | 1 | 28 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.18 [0.87, 1.61] |
| 1.2 at 24 months | 1 | 215 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.95 [0.77, 1.17] |
| 2 Not in any form of employment Show forest plot | 3 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 2.1 at 3 months | 1 | 352 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.05 [0.89, 1.24] |
| 2.2 at 6 months | 1 | 285 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.95 [0.81, 1.12] |
| 2.3 at 9 months | 1 | 132 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.0 [0.76, 1.32] |
| 2.4 at 12 months | 1 | 215 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.95 [0.77, 1.17] |
| 2.5 at 18 months | 1 | 152 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.76 [0.57, 1.02] |
| 3 Not participating in program (excluding employed) Show forest plot | 2 | 284 | Risk Ratio (M‐H, Random, 95% CI) | 0.95 [0.52, 1.72] |
| 4 Admitted to hospital (by 1 year) Show forest plot | 3 | 887 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.79 [0.65, 0.95] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Not in competitive employment (at 24 months) Show forest plot | 1 | 215 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.95 [0.77, 1.17] |
| 2 Not in any form of employment Show forest plot | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 2.1 at 3 months | 1 | 352 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.05 [0.89, 1.24] |
| 2.2 at 6 months | 1 | 285 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.95 [0.81, 1.12] |
| 2.3 at 12 months | 1 | 215 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.95 [0.77, 1.17] |
| 3 Admitted to hospital in first year of study Show forest plot | 1 | 215 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.69 [0.49, 0.96] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Not in any form of employment (at 6 months) Show forest plot | 1 | 150 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.40 [0.28, 0.57] |
| 2 Not participating in program Show forest plot | 1 | 150 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.53 [0.39, 0.71] |
| 3 Admitted to hospital during first year of study Show forest plot | 1 | 150 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.55 [0.31, 0.96] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Not in competitive employment Show forest plot | 2 | 142 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.86 [0.77, 0.95] |
| 1.1 at 6 months | 1 | 20 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.56 [0.29, 1.07] |
| 1.2 at 9 months | 1 | 122 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.90 [0.83, 0.99] |
| 2 Not in any form of employment Show forest plot | 1 | 122 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.89 [0.81, 0.97] |
| 3 Not in any form or employment or training or education at end of study Show forest plot | 1 | 122 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.63 [0.52, 0.77] |
| 4 Not participating in program Show forest plot | 2 | 142 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.85 [0.33, 2.18] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Not in competitive employment Show forest plot | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 1.1 at 9 months | 1 | 131 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.00 [0.90, 1.10] |
| 1.2 at 15 months | 1 | 131 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.88 [0.78, 1.00] |
| 2 Not in any form of employment (at 15 months) Show forest plot | 1 | 131 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.96 [0.69, 1.33] |
| 3 Numbers not participating in program Show forest plot | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 3.1 at 4 months | 1 | 131 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.77 [0.98, 3.21] |
| 3.2 at 9 months | 1 | 131 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.20 [0.74, 1.92] |
| 4 Number readmitted to hospital (at about 15 months) Show forest plot | 1 | 131 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.05 [0.68, 1.62] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Not in competitive employment Show forest plot | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 1.3 at 12 months | 1 | 256 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.01 [0.93, 1.09] |
| 1.5 at 24 months | 1 | 256 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.92 [0.85, 0.99] |
| 1.6 at 36 months | 1 | 256 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.88 [0.82, 0.96] |
| 2 Not in any form of employment Show forest plot | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 3 Numbers not participating in program Show forest plot | 1 | 256 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.74 [0.55, 1.01] |
| 4 Numbers admitted to hospital during study Show forest plot | 1 | 256 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.83 [0.63, 1.10] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Not in competitive employment Show forest plot | 5 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 1.1 at about 4 months | 3 | 364 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.73 [0.66, 0.81] |
| 1.2 at 6 months | 3 | 364 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.74 [0.67, 0.82] |
| 1.3 at 9 months | 3 | 364 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.67 [0.60, 0.76] |
| 1.4 at 12 months | 5 | 484 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.76 [0.69, 0.84] |
| 1.5 at 15 months | 3 | 364 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.82 [0.73, 0.91] |
| 1.6 at 18 months | 3 | 364 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.78 [0.71, 0.87] |
| 1.7 at 24 months | 2 | 155 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.90 [0.81, 1.00] |
| 2 Not in any form of employment Show forest plot | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 2.1 at 6 months | 1 | 69 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.05 [0.62, 1.78] |
| 2.2 at 9 months | 1 | 69 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.61 [0.35, 1.08] |
| 2.3 at 12 months | 1 | 69 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.67 [0.40, 1.12] |
| 2.4 at 15 months | 1 | 69 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.93 [0.53, 1.61] |
| 2.5 at 18 months | 1 | 69 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.81 [0.50, 1.33] |
| 3 Numbers not participating in program Show forest plot | 4 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 3.1 at 6 months | 1 | 86 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.95 [0.58, 1.54] |
| 3.2 at 12 months | 2 | 295 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.67 [0.48, 0.96] |
| 3.3 at 18 months | 1 | 69 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.37 [0.10, 1.32] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Not in competitive employment Show forest plot | 2 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 1.1 at 4 months | 2 | 295 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.70 [0.62, 0.78] |
| 1.2 at 6 months | 2 | 295 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.71 [0.63, 0.80] |
| 1.3 at 9 months | 2 | 295 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.66 [0.57, 0.75] |
| 1.4 at 12 months | 2 | 295 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.79 [0.70, 0.89] |
| 1.5 at 15 months | 2 | 295 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.83 [0.74, 0.93] |
| 1.6 at 18 months | 2 | 295 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.79 [0.70, 0.89] |
| 2 Numbers not participating in program Show forest plot | 2 | 295 | Risk Ratio (M‐H, Random, 95% CI) | 0.52 [0.15, 1.85] |
