Scolaris Content Display Scolaris Content Display

PRISMA flow diagram
Figuras y tablas -
Figure 1

PRISMA flow diagram

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.
Figuras y tablas -
Figure 2

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
Figuras y tablas -
Figure 3

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Funnel plot of comparison: 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, outcome: 1.1 Mean improvement in symptom scores: OQ‐45 PROMS.
Figuras y tablas -
Figure 4

Funnel plot of comparison: 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, outcome: 1.1 Mean improvement in symptom scores: OQ‐45 PROMS.

Funnel plot of comparison: 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, outcome: 1.2 Mean improvement in symptom scores: OQ‐45 or ORS PROMs.
Figuras y tablas -
Figure 5

Funnel plot of comparison: 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, outcome: 1.2 Mean improvement in symptom scores: OQ‐45 or ORS PROMs.

Comparison 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, Outcome 1 Mean improvement in symptom scores: OQ‐45 PROMS.
Figuras y tablas -
Analysis 1.1

Comparison 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, Outcome 1 Mean improvement in symptom scores: OQ‐45 PROMS.

Comparison 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, Outcome 2 Mean improvement in symptom scores: OQ‐45 or ORS PROMs.
Figuras y tablas -
Analysis 1.2

Comparison 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, Outcome 2 Mean improvement in symptom scores: OQ‐45 or ORS PROMs.

Comparison 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, Outcome 3 Number of treatment sessions received: all participants.
Figuras y tablas -
Analysis 1.3

Comparison 1 Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback, Outcome 3 Number of treatment sessions received: all participants.

Comparison 2 Subgroup analysis: Setting, Outcome 1 Mean improvement in symptom scores by setting.
Figuras y tablas -
Analysis 2.1

Comparison 2 Subgroup analysis: Setting, Outcome 1 Mean improvement in symptom scores by setting.

Comparison 3 Subgroup analysis: Whether participants were given a formal diagnosis or not, Outcome 1 Mean improvement in symptom scores by whether participants were given a formal diagnosis or not.
Figuras y tablas -
Analysis 3.1

Comparison 3 Subgroup analysis: Whether participants were given a formal diagnosis or not, Outcome 1 Mean improvement in symptom scores by whether participants were given a formal diagnosis or not.

Comparison 4 Subgroup analysis: Feeback given to clinician, participant or both, Outcome 1 Mean improvement in symptom scores: feedback given to clinician, participant or both.
Figuras y tablas -
Analysis 4.1

Comparison 4 Subgroup analysis: Feeback given to clinician, participant or both, Outcome 1 Mean improvement in symptom scores: feedback given to clinician, participant or both.

Comparison 5 Subgroup analysis: Whether feedback included treatment instructions or an algorithm, Outcome 1 Mean improvement in symptom scores by whether feedback included treatment instructions or an algorithm.
Figuras y tablas -
Analysis 5.1

Comparison 5 Subgroup analysis: Whether feedback included treatment instructions or an algorithm, Outcome 1 Mean improvement in symptom scores by whether feedback included treatment instructions or an algorithm.

Comparison 6 Subgroup analysis: studies involving Michael Lambert versus studies not involving him, Outcome 1 Mean improvement in symptom scores by whether studies involved Michael Lambert.
Figuras y tablas -
Analysis 6.1

Comparison 6 Subgroup analysis: studies involving Michael Lambert versus studies not involving him, Outcome 1 Mean improvement in symptom scores by whether studies involved Michael Lambert.

Comparison 7 Post hoc analyses ‐ 'on track' and 'not on track' participants, Outcome 1 Mean improvement in symptom scores: 'not on track' participants only.
Figuras y tablas -
Analysis 7.1

Comparison 7 Post hoc analyses ‐ 'on track' and 'not on track' participants, Outcome 1 Mean improvement in symptom scores: 'not on track' participants only.

Comparison 7 Post hoc analyses ‐ 'on track' and 'not on track' participants, Outcome 2 Number of treatment sessions received by 'on track' and 'not on track' participants.
Figuras y tablas -
Analysis 7.2

Comparison 7 Post hoc analyses ‐ 'on track' and 'not on track' participants, Outcome 2 Number of treatment sessions received by 'on track' and 'not on track' participants.

Summary of findings for the main comparison. Treatment informed by feedback of patient reported outcome measures compared with treatment as usual

Feedback of PROM scores for routine monitoring of common mental health disorders

Patient or population: People with common mental health disorders1
Settings: Primary care, multidisciplinary mental health care, or psychological therapies

Intervention: Feedback of PROM scores to clinician, or both clinician and patient

Comparator: No feedback of PROM scores

Outcomes and length of follow‐up

Illustrative risk

Number of participants
(number of studies)

Quality of the evidence
(GRADE)

Comments

Assumed risk (range of means in no‐feedback groups)

Relative effect (95% CI) in feedback groups

Mean improvement in symptom scores
Outcome Questionnaire‐45 (OQ‐45) or Outcome Rating Scale (ORS)

Follow‐up: 1‐6 months2

Mean scores in no‐feedback groups ranged from 51.8 to 101.5 points for OQ‐45 and from 23.8 to 29.5 points for ORS. Standard deviations ranged from 17.8 to 28.6 points for OQ‐45 and from 7.1 to 9.6 points for ORS

Standard mean difference in symptom scores at end of study in feedback groups was 0.07 standard deviations lower
(0.16 lower to 0.01 higher)3,4

3696
(12 studies)

⊕⊕⊝⊝
low5,6

Neither study in the primary care setting used the OQ‐45 or ORS PROMs, and so could not be included in this meta‐analysis

Health‐related quality of life
Medical Outcomes Study Short Form (SF‐36)

Follow‐up: 1‐5 months2

Medical Outcomes Study (SF‐12) physical and mental subscales).

Scale from 0‐100

Follow‐up: 0‐1 year

Study results could not be combined in a meta‐analysis as data were not available in an appropriate format

Mathias 1994 reported no significant differences between feedback and control groups on all nine domains of the SF‐36

Scheidt 2012 reported no significant differences between feedback and no‐feedback groups in physical or mental sub‐scale scores

583
(1 study)

587

(1 study)

⊕⊕⊕⊝

moderate7

Adverse events
PHQ‐9 questionnaire8

Follow‐up: 6 months

Chang 2012 reported no immediate suicide risk across both feedback and no‐feedback groups combined. Number per group not given

642
(1 study)

⊕⊕⊕⊝

moderate7

Social functioning

Follow‐up: 0‐1 year2

Data for the social functioning subscale of the OQ‐45 were considered separately in Hansson 2013 and no difference was found

262

(1 study)

⊕⊕⊝⊝

low9

Costs

Not estimable

0

(0 studies)

No study assessed the impact of the intervention on direct or indirect costs

Changes in the management of CMHDs

Changes in drug therapy and referrals for specialist care

Follow‐up: 1‐6 months2

Study results could not be combined in a meta‐analysis as data were not available in an appropriate format

Chang 2012 and Mathias 1994 both reported no significant differences in changes in drug therapy between study arms

Mathias 1994 reported mental health referrals were significantly more likely in the feedback group (OR 1.73, 95% CI 1.11 to 2.70)

1215
(2 studies)

⊕⊕⊕⊝

moderate7

Changes in the management of CMHDs
Number of treatment sessions received

Follow‐up: 1‐6 months2

Mean in no‐feedback groups ranged from 3.7 to 33.5 treatment sessions

Mean difference in number of treatment sessions in feedback groups was 0.02 lower
(0.42 lower to 0.39 higher)

2608
(7 studies)

⊕⊕⊝⊝
low10

Post‐hoc analysis. Changes in medication and referrals for additional therapy were not assessed by any of these studies

CI: Confidence interval

GRADE Working Group grades of evidence
High quality: Further research is very unlikely to change our confidence in the estimate of effect.
Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.
Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.
Very low quality: We are very uncertain about the estimate.

1Studies were included if the majority of people diagnosed had CMHDs and no more than 10% had diagnoses of psychotic disorders, learning difficulties, dementia, substance misuse, or eating disorders

2Duration of therapy was variable in all studies and determined by the clinician or the patient, or both
3OQ‐45 range of scores 0‐180 (0 best, 180 worst). Three studies (Murphy 2012, Reese 2009a and Reese 2009b) used the ORS ‐ range of scores 0‐40 (0 worst, 40 best)
4 This is a difference in standard deviations. A standard deviation of 0‐0.2 represents no to small difference between groups (rule of thumb according to Cohen's interpretation of effect size)
5An expected effect size of 0.3 SD would require a minimum total sample size of 352 participants. An expected effect size of 0.1 SD would require 3142 participants

6Downgraded two levels due to risk of bias (all included studies were judged at high risk of bias in at least two domains, in particular blinding of participants and outcome assessment, and attrition), and indirectness (although symptom scores were compared between feedback and non‐feedback groups, wider social functioning and quality‐of‐life measurements were not assessed in nearly all studies)

7Downgraded one level due to risk of bias (judged at high risk of bias in at least two domains, in particular blinding of participants and outcome assessment, and attrition)

8Number of PHQ‐9 questionnaires which contained reports of self‐harming thoughts

9Downgraded two levels due to risk of bias and imprecision, as total participant numbers were less than 400

10Downgraded two levels due to risk of bias and for imprecision: estimate of effect includes no effect and incurs very wide confidence intervals

Figuras y tablas -
Summary of findings for the main comparison. Treatment informed by feedback of patient reported outcome measures compared with treatment as usual
Comparison 1. Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mean improvement in symptom scores: OQ‐45 PROMS Show forest plot

9

3438

Mean Difference (IV, Random, 95% CI)

‐1.14 [‐3.15, 0.86]

2 Mean improvement in symptom scores: OQ‐45 or ORS PROMs Show forest plot

12

3696

Std. Mean Difference (IV, Random, 95% CI)

‐0.07 [‐0.16, 0.01]

3 Number of treatment sessions received: all participants Show forest plot

7

2608

Mean Difference (IV, Random, 95% CI)

‐0.02 [‐0.42, 0.39]

Figuras y tablas -
Comparison 1. Difference in outcome feeding back OQ‐45 or ORS scores versus no feedback
Comparison 2. Subgroup analysis: Setting

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mean improvement in symptom scores by setting Show forest plot

12

3696

Std. Mean Difference (IV, Random, 95% CI)

‐0.07 [‐0.16, 0.01]

1.1 Multidisciplinary mental health care setting

7

1848

Std. Mean Difference (IV, Random, 95% CI)

‐0.05 [‐0.18, 0.07]

1.2 Psychological therapy setting

5

1848

Std. Mean Difference (IV, Random, 95% CI)

‐0.10 [‐0.23, 0.03]

Figuras y tablas -
Comparison 2. Subgroup analysis: Setting
Comparison 3. Subgroup analysis: Whether participants were given a formal diagnosis or not

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mean improvement in symptom scores by whether participants were given a formal diagnosis or not Show forest plot

12

3696

Std. Mean Difference (IV, Random, 95% CI)

‐0.07 [‐0.16, 0.01]

1.1 Mean improvement in symptom scores: participants given a formal diagnosis

3

1144

Std. Mean Difference (IV, Random, 95% CI)

‐0.01 [‐0.23, 0.21]

1.2 Mean difference in symptom scores: participants not given a formal diagnosis

9

2552

Std. Mean Difference (IV, Random, 95% CI)

‐0.08 [‐0.15, 0.00]

Figuras y tablas -
Comparison 3. Subgroup analysis: Whether participants were given a formal diagnosis or not
Comparison 4. Subgroup analysis: Feeback given to clinician, participant or both

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mean improvement in symptom scores: feedback given to clinician, participant or both Show forest plot

12

3696

Std. Mean Difference (IV, Random, 95% CI)

‐0.07 [‐0.16, 0.01]

1.1 Mean improvement in symptom scores: feedback given only to the clinician

2

140

Std. Mean Difference (IV, Random, 95% CI)

‐0.17 [‐0.63, 0.30]

1.2 Mean improvement in symptom scores: feedback given explicitly to both clinician and participant

6

862

Std. Mean Difference (IV, Random, 95% CI)

‐0.12 [‐0.30, 0.05]

1.3 Mean improvement in symptom scores: clinicians permitted or encouraged to share feedback with participant

6

2694

Std. Mean Difference (IV, Random, 95% CI)

‐0.05 [‐0.16, 0.06]

Figuras y tablas -
Comparison 4. Subgroup analysis: Feeback given to clinician, participant or both
Comparison 5. Subgroup analysis: Whether feedback included treatment instructions or an algorithm

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mean improvement in symptom scores by whether feedback included treatment instructions or an algorithm Show forest plot

12

3696

Std. Mean Difference (IV, Random, 95% CI)

‐0.07 [‐0.16, 0.01]

1.1 Mean improvement in symptom scores: treatment instructions or algorithm

2

1184

Std. Mean Difference (IV, Random, 95% CI)

‐0.03 [‐0.14, 0.09]

1.2 Mean improvement in symptom scores: no treatment instructions or algorithm

10

2512

Std. Mean Difference (IV, Random, 95% CI)

‐0.09 [‐0.20, 0.02]

Figuras y tablas -
Comparison 5. Subgroup analysis: Whether feedback included treatment instructions or an algorithm
Comparison 6. Subgroup analysis: studies involving Michael Lambert versus studies not involving him

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mean improvement in symptom scores by whether studies involved Michael Lambert Show forest plot

9

Std. Mean Difference (IV, Random, 95% CI)

Subtotals only

1.1 Mean improvement in symptom scores: studies involving Michael Lambert

5

2032

Std. Mean Difference (IV, Random, 95% CI)

‐0.06 [‐0.15, 0.03]

1.2 Mean improvement in symptom scores: studies not involving Michael Lambert

4

1406

Std. Mean Difference (IV, Random, 95% CI)

‐0.02 [‐0.19, 0.15]

Figuras y tablas -
Comparison 6. Subgroup analysis: studies involving Michael Lambert versus studies not involving him
Comparison 7. Post hoc analyses ‐ 'on track' and 'not on track' participants

Outcome or subgroup title

No. of studies

No. of participants

Statistical method

Effect size

1 Mean improvement in symptom scores: 'not on track' participants only Show forest plot

10

923

Std. Mean Difference (IV, Random, 95% CI)

‐0.22 [‐0.35, ‐0.09]

2 Number of treatment sessions received by 'on track' and 'not on track' participants Show forest plot

5

2114

Mean Difference (IV, Random, 95% CI)

0.06 [‐0.91, 1.02]

2.1 Number of treatment sessions received by "on track" participants only

4

1633

Mean Difference (IV, Random, 95% CI)

‐0.69 [‐1.10, ‐0.29]

2.2 Number of treatment sessions received by "not on track" participants only

5

481

Mean Difference (IV, Random, 95% CI)

0.73 [‐2.04, 3.50]

Figuras y tablas -
Comparison 7. Post hoc analyses ‐ 'on track' and 'not on track' participants
Table 1. Contact with investigators

Study ID

Response

Additional info

Outcome

Chang 2012

No

Failed to respond

Email sent to [email protected] and [email protected] on 22 January 2015 regarding randomisation process, CGI‐S and PGI‐S outcomes and criteria for diagnosis

De Jong 2012; De Jong 2014

Yes

Invited to become a co‐author

Replied immediately to an email sent on 17 June 2015. Provided details of data on OQ‐45 scores and further information on generation of random sequence and allocation concealment

All data extraction was done by TK and MEG as KdJ became a co‐author of the review

Haderlie 2012

Yes

Reply on 28 January 2015 in response to email sent on 22 January:

"We collected the data in a naturalistic setting at 2 clinics with clients who were already in treatment in some cases. We did collect outcome data over the course of the study period, but we do not have first and last measurements in all cases. I also do not have specific information regarding the clients (such as diagnoses). Therapists were aware which condition they were in as the independent variable was whether or not they received progress feedback. Clients did not know which condition they were in"

Hansson 2013

Yes

Emails sent to [email protected] on 22 January 2015, 29 July 2015 and 10 August 2015 enquiring about separate outcome data per diagnostic group. Reply on 30 August with requested information

Mathias 1994

No

Failed to contact

Study too old ‐ contact details of authors unobtainable

Probst 2013

No

Failed to respond

Email sent to [email protected]‐regensburg.de on 22 January 2015 regarding details of randomisation and blinding

Puschner 2009

Yes

Reply on 3 February 2015 in response to email sent 22 January 2015: query about breakdown of outcome data per diagnostic group, with tabulated data provided

Reese 2009a; Reese 2009b

Yes

Reply on 3 February 2015 in response to email sent 22 January 2015: to query about randomisation, allocation concealment and outcome blinding:

Randomisation:

"When a client was assigned to a therapist, the first client was randomised using a random number generator to either the feedback or TAU condition. The second client was then assigned to the other condition."

Allocation concealment:

"This was done by the person who assigned clients at the respective centres. This was done after enrolment into the study. Investigators and client participants could not foresee which condition a participant would be placed into."

Outcome blinding:

"The researchers did not know which condition participants were in until the time of analysis."

Query over discrepancy in data presented in main text and table:

"The table is correct and it should be 4.69. That is what was used in the analyses as well."

Further data regarding number of treatment sessions and standard deviations provided on request in December 2015

Further information provided on diagnoses of study participants on 3 May 2016, confirming that more than 90% of them would have had qualifying clinical diagnoses of anxiety or depressive disorders, or both

Trudeau 2001

Yes

Reply on 26 March 2015 in response to email sent on 26 March 2015 enquiring about the details of managed care in place in some of the study participants, and about outcome data. "Managed care consisted of session limits and utilization review". Further email exchanges from 31 March to 2 April to enquire about blinding details. Reply on 13 May 2016 to email sent 13 May 2016, enquiring whether study participants would have met our review inclusion/exclusion characteristics, confirming that they would have met them

Lambert 2001; Simon 2012;Probst 2013; Whipple 2003

No

Failed to respond

Email sent to [email protected] on 22 January 2015 as listed as corresponding author on all four studies. Enquiries about randomisation procedure and allocation concealment. No reply regarding any of the studies was made. Further email sent to [email protected] and Michael Lambert on 17 July 2015, no reply received

Hawkins 2004

Yes

Email sent to [email protected] on 18 November 2015, reply received: further data provided on OQ‐45 outcomes and number of treatment sessions on 7 December 2015

Figuras y tablas -
Table 1. Contact with investigators
Table 1. Contact with investigators

Study ID

Response

Additional info

Outcome

Chang 2012

No

Failed to respond

Email sent to [email protected] and [email protected] on 22 January 2015 regarding randomisation process, CGI‐S and PGI‐S outcomes and criteria for diagnosis

De Jong 2012; De Jong 2014

Yes

Invited to become a co‐author

Replied immediately to an email sent on 17 June 2015. Provided details of data on OQ‐45 scores and further information on generation of random sequence and allocation concealment

All data extraction was done by TK and MEG as KdJ became a co‐author of the review

Haderlie 2012

Yes

Reply on 28 January 2015 in response to email sent on 22 January:

"We collected the data in a naturalistic setting at 2 clinics with clients who were already in treatment in some cases. We did collect outcome data over the course of the study period, but we do not have first and last measurements in all cases. I also do not have specific information regarding the clients (such as diagnoses). Therapists were aware which condition they were in as the independent variable was whether or not they received progress feedback. Clients did not know which condition they were in"

Hansson 2013

Yes

Emails sent to [email protected] on 22 January 2015, 29 July 2015 and 10 August 2015 enquiring about separate outcome data per diagnostic group. Reply on 30 August with requested information

Mathias 1994

No

Failed to contact

Study too old ‐ contact details of authors unobtainable

Probst 2013

No

Failed to respond

Email sent to [email protected]‐regensburg.de on 22 January 2015 regarding details of randomisation and blinding

Puschner 2009

Yes

Reply on 3 February 2015 in response to email sent 22 January 2015: query about breakdown of outcome data per diagnostic group, with tabulated data provided

Reese 2009a; Reese 2009b

Yes

Reply on 3 February 2015 in response to email sent 22 January 2015: to query about randomisation, allocation concealment and outcome blinding:

Randomisation:

"When a client was assigned to a therapist, the first client was randomised using a random number generator to either the feedback or TAU condition. The second client was then assigned to the other condition."

Allocation concealment:

"This was done by the person who assigned clients at the respective centres. This was done after enrolment into the study. Investigators and client participants could not foresee which condition a participant would be placed into."

Outcome blinding:

"The researchers did not know which condition participants were in until the time of analysis."

Query over discrepancy in data presented in main text and table:

"The table is correct and it should be 4.69. That is what was used in the analyses as well."

Further data regarding number of treatment sessions and standard deviations provided on request in December 2015

Further information provided on diagnoses of study participants on 3 May 2016, confirming that more than 90% of them would have had qualifying clinical diagnoses of anxiety or depressive disorders, or both

Trudeau 2001

Yes

Reply on 26 March 2015 in response to email sent on 26 March 2015 enquiring about the details of managed care in place in some of the study participants, and about outcome data. "Managed care consisted of session limits and utilization review". Further email exchanges from 31 March to 2 April to enquire about blinding details. Reply on 13 May 2016 to email sent 13 May 2016, enquiring whether study participants would have met our review inclusion/exclusion characteristics, confirming that they would have met them

Lambert 2001; Simon 2012;Probst 2013; Whipple 2003

No

Failed to respond

Email sent to [email protected] on 22 January 2015 as listed as corresponding author on all four studies. Enquiries about randomisation procedure and allocation concealment. No reply regarding any of the studies was made. Further email sent to [email protected] and Michael Lambert on 17 July 2015, no reply received

Hawkins 2004

Yes

Email sent to [email protected] on 18 November 2015, reply received: further data provided on OQ‐45 outcomes and number of treatment sessions on 7 December 2015

Figuras y tablas -
Table 1. Contact with investigators