Governance arrangements for health systems in low-income countries: an overview of systematic reviews

  • Review
  • Overview

Authors

  • Cristian A Herrera,

    Corresponding author
    1. Pontificia Universidad Católica de Chile, Department of Public Health, School of Medicine, Santiago, Chile
    2. Pontificia Universidad Católica de Chile, Evidence Based Health Care Program, Santiago, Chile
    • Cristian A Herrera, Department of Public Health, School of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 434, Santiago, Chile. crherrer@uc.cl.

  • Simon Lewin,

    1. Norwegian Institute of Public Health, Oslo, Norway
    2. South African Medical Research Council, Health Systems Research Unit, Tygerberg, South Africa
  • Elizabeth Paulsen,

    1. Norwegian Institute of Public Health, Oslo, Norway
  • Agustín Ciapponi,

    1. Institute for Clinical Effectiveness and Health Policy (IECS-CONICET), Argentine Cochrane Centre, Buenos Aires, Capital Federal, Argentina
  • Newton Opiyo,

    1. Cochrane, Cochrane Editorial Unit, London, UK
  • Tomas Pantoja,

    1. Pontificia Universidad Católica de Chile, Evidence Based Health Care Program, Santiago, Chile
    2. Pontificia Universidad Católica de Chile, Department of Family Medicine, Faculty of Medicine, Santiago, Chile
  • Gabriel Rada,

    1. Pontificia Universidad Católica de Chile, Evidence Based Health Care Program, Santiago, Chile
    2. Pontificia Universidad Católica de Chile, Department of Internal Medicine and Evidence-Based Healthcare Program, Faculty of Medicine, Santiago, Chile
  • Charles S Wiysonge,

    1. South African Medical Research Council, Cochrane South Africa, Cape Town, Western Cape, South Africa
    2. Stellenbosch University, Centre for Evidence-based Health Care, Faculty of Medicine and Health Sciences, Cape Town, South Africa
  • Gabriel Bastías,

    1. Pontificia Universidad Católica de Chile, Department of Public Health, School of Medicine, Santiago, Chile
  • Sebastian Garcia Marti,

    1. Institute for Clinical Effectiveness and Health Policy, Buenos Aires, Capital Federal, Argentina
  • Charles I Okwundu,

    1. Stellenbosch University, Centre for Evidence-based Health Care, Faculty of Medicine and Health Sciences, Cape Town, South Africa
  • Blanca Peñaloza,

    1. Pontificia Universidad Católica de Chile, Evidence Based Health Care Program, Santiago, Chile
    2. Pontificia Universidad Católica de Chile, Department of Family Medicine, Faculty of Medicine, Santiago, Chile
  • Andrew D Oxman

    1. Norwegian Institute of Public Health, Oslo, Norway

Abstract

Background

Governance arrangements include changes in rules or processes that determine authority and accountability for health policies, organisations, commercial products and health professionals, as well as the involvement of stakeholders in decision-making. Changes in governance arrangements can affect health and related goals in numerous ways, generally through changes in authority, accountability, openness, participation and coherence. A broad overview of the findings of systematic reviews can help policymakers, their technical support staff and other stakeholders to identify strategies for addressing problems and improving the governance of their health systems.

Objectives

To provide an overview of the available evidence from up-to-date systematic reviews about the effects of governance arrangements for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on governance arrangements and informing refinements of the framework for governance arrangements outlined in the overview.

Methods

We searched Health Systems Evidence in November 2010 and PDQ Evidence up to 17 December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of governance arrangements on patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use (health expenditures, healthcare provider costs, out-of-pocket payments, cost-effectiveness), healthcare provider outcomes (such as sick leave), or social outcomes (such as poverty, employment) and that were published after April 2005. We excluded reviews with limitations that were important enough to compromise the reliability of the findings of the review. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence) and assessments of the relevance of findings to low-income countries.

Main results

We identified 7272 systematic reviews and included 21 of them in this overview (19 primary reviews and 2 supplementary reviews). We focus here on the results of the 19 primary reviews, one of which had important methodological limitations. The other 18 were reliable (with only minor limitations).

We grouped the governance arrangements addressed in the reviews into five categories: authority and accountability for health policies (three reviews); authority and accountability for organisations (two reviews); authority and accountability for commercial products (three reviews); authority and accountability for health professionals (seven reviews); and stakeholder involvement (four reviews).

Overall, we found desirable effects for the following interventions on at least one outcome, with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects.

Decision-making about what is covered by health insurance

- Placing restrictions on the medicines reimbursed by health insurance systems probably decreases the use of and spending on these medicines (moderate-certainty evidence).

Stakeholder participation in policy and organisational decisions

- Participatory learning and action groups for women probably improve newborn survival (moderate-certainty evidence).
- Consumer involvement in preparing patient information probably improves the quality of the information and patient knowledge (moderate-certainty evidence).

Disclosing performance information to patients and the public

- Disclosing performance data on hospital quality to the public probably encourages hospitals to implement quality improvement activities (moderate-certainty evidence).
- Disclosing performance data on individual healthcare providers to the public probably leads people to select providers that have better quality ratings (moderate-certainty evidence).

Authors' conclusions

Investigators have evaluated a wide range of governance arrangements that are relevant for low-income countries using sound systematic review methods. These strategies have been targeted at different levels in health systems, and studies have assessed a range of outcomes. Moderate-certainty evidence shows desirable effects (with no undesirable effects) for some interventions. However, there are important gaps in the availability of systematic reviews and primary studies for the all of the main categories of governance arrangements.

摘要

低收入国家对医疗系统的资金管理:系统综述之概述

研究背景

政府措施包括规则或过程的变化,包括决定卫生政策,组织,商业产品和卫生专业人员的权力和责任的规则或流程的变化,以及利益相关者参与决策。政府措施的变化会从多种方式影响健康及相关方面,一般是通过权力、责任、公开、参与、和相关性的变化。基于系统综述的概述可以帮助政策决策者、技术支持员工、和其他利益相关者,找到解决和改善卫生系统管理水平的最佳战略。

研究目的

对低收入国家资金管理政策对医疗系统的影响的系统综述进行概述。次要结局包括发现将来进行评估和系统综述时的需求和休闲事项,以及彰显这篇概述所呈现的财政安排框架需要改进的地方。

研究方法

我们检索了2010年11月的卫生系统证据,以及截止到2016年12月的PDQ证据。我们没有设定任何日期、语言、发表状态的限制。我们纳入了2005年4月以后出版的较规范的系统综述,这些系统综述评估了政府措施对病人康复结果(健康或者健康行为),医疗服务的质量或利用,资源使用(健康花费,医疗服务提供者的支出,自付额,花费的有效性),医疗服务提供者的结局(比如病假),或者社会性结局(比如贫穷,雇佣)。我们排除了存在足以降低结果可信度的局限之处的综述。两位概述的作者各自独立地筛选了综述,提取了数据,并运用GRADE评估了证据的可信度。我们对合格的综述进行了SUPPORT总结,包括主要信息,“结果总结”表格(使用GRADE评估证据确信度),以及评估这些结果在低收入国家的相关性。

主要结果

我们检索到了7272篇系统综述,并最终纳入21篇(19篇主要综述和2篇补充综述)。我们重点关注19篇主要综述的结果,其中一个存在重要的方法学局限性。其他18篇研究是可信的(只有很小的局限性)。

我们将综述中涉及的政府措施分为五类:对健康政策的权利和责任(三篇综述);对组织们的权利和责任(两篇综述);对商业产品的权利和责任(三篇综述);对健康专业人士的权利和责任(七篇综述);以及利益相关者的参与(四篇综述)。

总之,我们发现以下干预措施至少在一个结局指标上的效果是令人满意的,证据质量为中等或高等;而没有发现不良影响的中等或高等质量的证据。

关于医疗保险所涵盖内容的决策。

-医保系统着眼于药品报销制度的限制可能减少此类医疗的使用和开销。(有适度把握的证据)

利益相关者参与政策和组织决策中。

-妇女参与式学习和行动小组有可能提高新生儿的存活率(有适度把握的证据)。
-患者参与准备信息有可能提高信息质量和患者知识贮备。(有适度把握的证据)。

向病人和公众披露信息。

-向公众揭露医院质量的运营数据有可能鼓励医院实施质量提升行动(有适度把握的证据)。-向公众揭露个体卫生保健提供者的性能数据可能导致人们选择有更高等级品质的提供者(有适度把握的证据)。

作者结论

研究人员已经评估了那些采用系统综述方法研究低收入国家政府措施的研究。这些决策涉及不同等级的卫生系统,并且研究已经测定了一系列的结果。有适度把握的证据显示了一些干涉具有良性影响(没有不良的影响)。然而,相关系统综述和政府措施主要类别的原始研究之间存在差距。

Plain language summary

Effects of governance arrangements for health systems in low-income countries

What is the aim of this overview?

The aim of this Cochrane Overview is to provide a broad summary of what is known about the effects of different governance arrangements for health systems in low-income countries.

This overview is based on 19 relevant systematic reviews. These systematic reviews searched for studies that evaluated different types of governance arrangements. The reviews included a total of 172 studies.

This overview is one of a series of four Cochrane Overviews that evaluate health system arrangements.

Main results

What are the effects of different ways of organising authority and accountability for health policies?

Three reviews were included and the key findings are that:

- collaboration between local health agencies and other local government agencies may lead to little or no difference in physical health or quality of life (low-certainty evidence);

- placing restrictions on the medicines reimbursed by health insurance systems probably decreases the use of and spending on these medicines (moderate-certainty evidence);

- it is uncertain if fraud prevention, detection and response interventions reduce healthcare fraud and related spending (very low-certainty evidence).

What are the effects of different ways of organising authority and accountability for organisations?

Two reviews were included and the key findings are that:

- Contracting non-state, not-for-profit providers to deliver health services may increase access to and use of these services, improve people's health outcomes and reduce household spending on health (low-certainty evidence). No evidence was available on whether contracting out was more effective than using these funds in the state sector.

What are the effects of different ways of organising authority and accountability for commercial products such as medicines and technologies?

Three reviews were included and the key findings are that:

- systems in which the World Health Organization (WHO) certifies medicine manufacturers (prequalification) and medicines registration (in which medicine regulatory authorities assess medicine manufacturers to ensure they meet international standards) may decrease the proportion of medicines that are substandard or counterfeit (low-certainty evidence);

- establishing a maximum reimbursement for pharmacies dispensing similar medicines covered by insurance may increase the use of generic medicines and may reduce the use of brand-name medicines (low-certainty evidence), but it is uncertain whether this approach affects the overall amount spent on medicines (very low-certainty evidence);

- direct-to-consumer advertising increases people's requests for medicines and the numbers of prescriptions given (high-certainty evidence).

What are the effects of different ways of organising authority and accountability for healthcare providers?

Seven reviews were included and the key findings are that:

- training programmes for district health system managers may increase their knowledge of planning processes and their monitoring and evaluation skills (low-certainty evidence);

- reducing immigration restrictions in high-income countries probably increases the migration of nurses from low- and middle-income to these countries (moderate-certainty evidence);

- it is uncertain whether inspection by an external body of healthcare organisation adherence to quality standards improves adherence, quality of care or health-acquired infection rates in hospitals (very low-certainty evidence).

What are the effects of different ways of organising stakeholder involvement in governing health services?

Four reviews were included and the key findings are that:

- participatory learning and action groups for women probably improve newborn survival (moderate-certainty evidence) and may improve maternal survival (low-certainty evidence);

- disclosing performance data on health insurance scheme quality to the public may lead people to select health plans that have better quality ratings or to avoid those with worse ratings and may lead to slight improvements in clinical outcomes for health insurance schemes (low-certainty evidence);

- disclosing performance data on hospital quality to the public may lead to little or no difference in people's selection of hospitals (low-certainty evidence), probably encourages hospitals to implement quality improvement activities (moderate-certainty evidence) and may lead to slight improvements in hospital clinical outcomes (low-certainty evidence);

- disclosing performance on individual healthcare providers to the public probably leads people to select providers that have better quality ratings (moderate-certainty evidence).

No studies evaluated the effects of stakeholder participation in policy and organisational decisions.

How up-to-date is this overview?

The overview authors searched for systematic reviews that had been published up to 17 December 2016.

Резюме на простом языке

Влияние механизмов управления в системах здравоохранения в странах с низким уровнем дохода

Какова цель этого обзора обзоров?

Цель этого Кокрейновского Обзора обзоров - представить широкое обобщение того, что известно о влиянии различных механизмов управления на системы здравоохранения в странах с низким уровнем дохода.

Этот обзор обзоров основан на 19 соответствующих систематических обзорах. В этих систематических обзорах проведен поиск исследований, в которых оценивали различные типы механизмов управления. Всего в эти обзоры было включено 172 исследования.

Это один из серии четырех Кокрейновских Обзоров обзоров, в которых оценивают механизмы управления системой здравоохранения.

Основные результаты

Как влияют различные методы организации управления и отчетности на политику здравоохранения?

Было включено три обзора, основными результатами которых являются:

- сотрудничество между местными органами здравоохранения и другими местными правительственными агентствами может привести к малым различиям или не оказывает никакого влияния на физическое здоровье или качество жизни (доказательства низкой определенности);

- введение ограничений на лекарства, оплачиваемых системами медицинского страхования, вероятно, уменьшает использование и расходы на эти лекарства (доказательства умеренной определенности)

- не ясно, сокращается ли число случаев мошенничества и связанных с этим расходов в здравоохранении в результате мер по предотвращению, выявлению и реагированию на мошенничество (доказательства очень низкой определенности).

Каково влияние различных методов организации управления и отчетности в организациях?

Были включены два обзора, основными результатами которых являются:

- Негосударственные, некоммерческие поставщики услуг здравоохранения, принимая на себя контрактные обязательства, могут увеличить доступность и использование этих услуг, улучшить показатели здоровья людей (исходы в отношении здоровья) и сократить семейные расходы на здоровье (доказательства низкой определенности). Не было доказательств того, является ли заключение сторонних контрактов более эффективным, чем использование этих средств в государственном секторе.

Каково влияние различных способов организации управления и отчетности на коммерческие продукты, такие как лекарства и технологии?

Было включено три обзора, основными результатами которых являются:

- системы, в которых Всемирная организация здравоохранения (ВОЗ) сертифицирует производителей лекарств (преквалификация) и регистрацию лекарственных средств (в которых органы регулирования лекарств оценивают производителей лекарств, чтобы гарантировать, что они соответствуют международным стандартам) могут уменьшить долю некачественных или контрфактных лекарств (доказательства низкой определенности);

- установление максимального уровня возмещения для аптек, отпускающих аналогичные лекарства, покрываемые страхованием, может увеличить использование генерических лекарств и уменьшить использование брендовых лекарств (доказательства низкой определенности), однако не ясно, влияет ли этот подход на общую сумму, потраченную на лекарства (доказательства очень низкой определенности);

- прямая реклама, направленная на потребителя, увеличивает запросы людей на лекарства и число выписанных рецептов (доказательства высокой определенности).

Каково влияние различных способов организации управления и отчетности на провайдеров медицинской помощи?

Было включено семь обзоров, основными результатами которых являются:

- учебные программы для руководителей районных систем здравоохранения могут повысить их знания в области процессов планирования и навыки мониторинга и оценки (доказательства низкой определенности);

- снятие ограничений на иммиграцию в странах с высоким уровнем доходов, вероятно, увеличивает миграцию медсестер из стран с низким и средним уровнем доходов в эти страны (доказательства умеренной определенности);

- не ясно, приводят ли проверки внешними органами соблюдения стандартов качества организацией здравоохранения к улучшению приверженности стандартам качества, повышению качества медицинской помощи или снижению уровня внутрибольничных инфекций (доказательства очень низкой определенности).

Каково влияние различных способов организации участия заинтересованных сторон в управлении здравоохранением?

Было включено четыре обзора, основными результатами которых являются:

- интерактивное обучение и группы помощи женщинам, вероятно, улучшают выживаемость новорожденных (доказательства умеренной определенности) и могут улучшить выживание матерей (доказательства низкой определенности);

- общественное обнародование данных о показателях качества медицинского страхования может привести к тому, что люди будут выбирать программы страхования, с более высокими оценками качества, или избегать те, у которых оценки ниже, а также может привести к незначительному улучшению клинических исходов в медицинском страховании (доказательства низкой определенности) ;

- общественное обнародование данных о качестве помощи в больницах может привести к незначительной разнице или не оказывает влияния на выбор больниц людьми (доказательства низкой определенности); вероятно, способствует осуществлению мероприятий по улучшению качества больниц (доказательства умеренной определенности) и может привести к незначительному улучшению клинических исходов в больницах (доказательства низкой определенности);

- общественное обнародование данных о работе / эффективности отдельных работников здравоохранения, вероятно, приводит к тому, что люди выбирают тех, у кого более высокие оценки качества (доказательства умеренной определенности).

Ни одно исследование не оценивало влияние участия заинтересованных сторон в политике здравоохранения и организационных решениях.

Насколько современен этот обзор обзоров?

Авторы этого Обзор обзоров провели поиск систематических обзоров, которые были опубликованы к 17 декабря 2016.

Заметки по переводу

Перевод: Курбатова Ольга Геннадьевна. Редактирование: Зиганшина Лилия Евгеньевна. Координация проекта по переводу на русский язык: Cochrane Russia - Кокрейн Россия (филиал Северного Кокрейновского Центра на базе Казанского федерального университета). По вопросам, связанным с этим переводом, пожалуйста, обращайтесь к нам по адресу: cochrane.russia.kpfu@gmail.com; cochranerussia@kpfu.ru

Ringkasan bahasa mudah

Kesan pengaturan tadbir urus sistem kesihatan di negara berpendapatan rendah

Apakah matlamat gambaran keseluruhan ini?

Tujuan ulasan Cochrane ini adalah untuk memberikan ringkasan yang luas tentang apa yang diketahui mengenai kesan pengaturan tadbir urus yang berbeza untuk sistem kesihatan di negara berpendapatan rendah.

Gambaran keseluruhan ini berdasarkan kepada 19 ulasan sistematik yang berkaitan. Ulasan sistematik ini mencari kajian-kajian yang menilai pelbagai jenis tadbir urus. Ulasan ini melibatkan sejumlah 172 kajian.

Gambaran keseluruhan ini adalah satu daripada empat siri gambaran keseluruhan Cochrane yang menilai susunan sistem kesihatan.

Keputusan utama

Apakah kesan cara yang berbeza untuk mengurus kuasa dan akauntabiliti untuk polisi kesihatan?

Tiga ulasan telah dimasukkan dan penemuan utama ialah:

- kerjasama antara agensi kesihatan tempatan dan agensi kerajaan tempatan yang lain boleh membawa kepada sedikit atau tiada perbezaan dalam kesihatan fizikal atau kualiti hidup (kepastian bukti yang rendah);

- meletakkan sekatan ke atas ubat-ubatan yang dibayar balik oleh sistem insurans kesihatan mungkin mengurangkan penggunaan dan perbelanjaan ubat-ubatan ini (kepastian bukti yang sederhana);

- Ia adalah tidak pasti jika intervensi pencegahan, pengesanan dan respon intervensi mengurangkan penipuan penjagaan kesihatan dan perbelanjaan yang berkaitan (kepastian bukti yang sangat rendah).

Apakah kesan cara yang berbeza untuk mengurus kuasa dan akauntabiliti untuk organisasi?

Dua ulasan dimasukkan dan penemuan utama adalah:

- Kontrak bukan kerajaan, penyedia bukan keuntungan untuk menyediakan perkhidmatan kesihatan boleh meningkatkan akses kepada dan menggunakan perkhidmatan ini, meningkatkan hasil kesihatan rakyat dan mengurangkan perbelanjaan isi rumah terhadap kesihatan (kepastian bukti yang rendah). Tiada bukti yang boleh didapati sama ada kontrak keluar adalah lebih berkesan daripada menggunakan dana ini di sektor kerajaan.

Apakah kesan cara yang berbeza untuk mengurus kuasa dan akauntabiliti untuk produk komersial seperti ubat dan teknologi?

Tiga ulasan telah dimasukkan dan penemuan utama ialah:

- sistem di mana Organisasi Kesihatan Sedunia (WHO) memperakui pengeluar ubat (prakualifikasi) dan pendaftaran ubat (di mana pihak berkuasa pengawalseliaan menilai pengeluar ubat untuk memastikan mereka memenuhi piawaian antarabangsa) boleh menurunkan kadar ubat-ubatan yang kurang bermutu atau palsu (kepastian bukti yang rendah);

- menubuhkan pembayaran balik maksimum untuk farmasi yang mengeluarkan ubat-ubatan yang serupa yang dilindungi oleh insurans boleh meningkatkan penggunaan ubat-ubatan generik dan boleh mengurangkan penggunaan ubat-ubatan jenama (kepastian bukti yang rendah), tetapi tidak pasti sama ada pendekatan ini mempengaruhi jumlah keseluruhan yang dibelanjakan ubat (kepastian bukti yang sangat rendah);

- Pengiklanan langsung kepada pengguna meningkatkan permintaan untuk ubat-ubatan dan bilangan preskripsi yang diberikan (kepastian bukti yang tinggi).

Apakah kesan cara yang berbeza untuk menguruskan kuasa dan akauntabiliti untuk penyedia penjagaan kesihatan?

Tujuh ulasan dimasukkan dan penemuan utama adalah:

- program latihan untuk pengurus sistem kesihatan daerah boleh meningkatkan pengetahuan mereka tentang proses perancangan dan kemahiran pemantauan dan penilaian mereka (kepastian bukti yang rendah);

- Mengurangkan sekatan imigresen di negara berpendapatan tinggi mungkin meningkatkan penghijrahan jururawat dari pendapatan rendah dan sederhana ke negara-negara ini (kepastian bukti yang sederhana);

- Ia adalah tidak pasti sama ada pemeriksaan oleh badan luar organisasi penjagaan kesihatan yang mematuhi piawaian kualiti meningkatkan kepatuhan, kualiti penjagaan atau kadar jangkitan kesihatan yang diperolehi di hospital (kepastian bukti yang sangat rendah).

Apakah kesan cara yang berbeza untuk menguruskan penyertaan pihak berkepentingan dalam mengurus perkhidmatan kesihatan?

Empat ulasan telah dimasukkan dan penemuan-penemuan utama adalah:

- kumpulan pembelajaran dan tindakan partisipatif bagi wanita mungkin meningkatkan kelangsungan hidup bayi baru (kepastian bukti yang sederhana) dan boleh meningkatkan kelangsungan hidup ibu (kepastian bukti yang rendah);

- mendedahkan data prestasi terhadap kualiti skim insurans kesihatan kepada orang ramai boleh menyebabkan orang ramai memilih pelan kesihatan yang mempunyai penilaian kualiti yang lebih baik atau untuk mengelakkan penarafan yang lebih teruk dan boleh membawa sedikit peningkatan dalam hasil klinikal untuk skim insurans kesihatan (kepastian bukti yang rendah) ;

- mendedahkan data prestasi terhadap kualiti hospital kepada orang awam mungkin membawa sedikit atau tiada perbezaan dalam pemilihan hospital orang (kepastian bukti yang rendah), mungkin menggalakkan hospital untuk melaksanakan aktiviti penambahbaikan berkualiti (kepastian bukti yang sederhana) dan boleh menyebabkan sedikit penambahbaikan hasil klinikal hospital (kepastian bukti yang rendah);

- mendedahkan prestasi pada penyedia penjagaan kesihatan individu kepada orang awam mungkin membawa orang untuk memilih pembekal yang mempunyai penilaian kualiti yang lebih baik (kepastian bukti yang sederhana).

Tiada kajian menilai kesan penyertaan pihak berkepentingan dalam dasar dan keputusan organisasi.

Adakah gambaran keseluruhan ini terkini?

Pengarang gambaran keseluruhan telah mencari ulasan sistematik yang telah diterbitkan sehingga 17 Disember 2016.

Catatan terjemahan

Diterjemahkan oleh Wong Chun Hoong (International Medical University). Disunting oleh Tuan Hairulnizam Tuan Kamauzaman (Universiti Sains Malaysia). Untuk sebarang pertanyaan berkaitan terjemahan ini sila hubungi Wong.ChunHoong@student.imu.edu.my.

概要

低收入国家对医疗系统资金管理的影响

这篇概述的目的是什么?

这篇Cochrane概述旨在全面总结关于已知的低收入国家中不同医疗系统管理措施带来的影响。

这篇概述基于19篇有关的系统综述。这些系统综述研究了关于不同类型国家管理措施测评的研究。此综述共纳入了172项研究。

此概述是评估不同医疗系统管理措施的四篇Cochrane概述系列中的一篇。

主要结果

对卫生政策不同的组织权力和问责制的影响是什么?

包含了三个综述和关键结果的是:

-在卫生机构和其他当地政府机构之间的合作可能对体质健康或生活品质的影响很小甚至无。(低确定性的证据)

-医保系统着眼于药品报销制度的限制可能减少此类医疗的使用和开销。(有适度把握的证据)

-预防、检测、和响应欺诈的干预措施对减少医疗欺诈和相关支出的影响是不确定的。(非常低确定性的证据)

对于卫生政策的不同组织权利和问责制的影响是什么?

包含了两个综述和关键结果的是:

-缩小非国有、非营利性提供者对健康服务的提供可能增加这些服务的使用,提高人们的保健疗效,以及减少家庭在健康方面的支出(低确定性的证据)。没有证据表明承包是否比使用这些国家部门的基金更有效。

对商业产品不同的组织权力和问责制的影响是什么?比如医疗和技术?

包含了三个综述和关键结果的是:

-由世界卫生组织(WHO)所证明的医疗制造业(资格预审)和药品注册(其中医疗法规机构对这些医疗制造者进行评估来确保他们达到了国际标准)系统可能降低不达标药品或伪品的比率(低确定性的证据)。

-建立一个最大的补偿机制,为了被保险覆盖的药房配药和相似的治疗,可能增加仿制药的使用,也可能减少知名药品的使用。(低确定性的证据)但这种方法是否影响医疗总开支是不确定的(非常低确定性的证据)。

-DTC营销模式的宣传增长了大众对医疗的需求和处方的数量(高确定性的证据)。

对于医疗保健提供者的不同组织权利和问责制的影响是什么?

包含了两个综述和关键结果的是:

-关于卫生系统地区管理人员的培训项目可能增长他们规划流程的知识,以及他们的监测和评价技能(低确定性的证据)。

-减少高收入国家的移民居住区有可能增加来自中低收入国家的护士移民数量。(有适度把握的证据)

-医疗机构组织对外部身体检查的坚持是否对质量等级的提升是不确定的,在医院中照顾的质量或者健康获得感染的等级(十分低确定性的证据)。

在管理健康服务中,对于利益相关者参与的不同组织权利和问责制的影响是什么?

包含了四个综述和关键结果的是:

-女性参与式学习和行动小组有可能提高新生儿的存活率(有适度把握的证据)和产妇生存率(低把握度的证据)。

-向公众揭露医保方案质量的数据可能导致人们选择有更高质量等级的健康计划,或者避免这些有很糟糕等级的计划,这可能导致医保方案临床效果的轻微提高(低确定性的证据)。

-向公众揭露医院质量的表现数据可能导致人们在医院的选择上很小或几乎无变化(低确定性的证据),也许会鼓励医院实施质量提升(有适度把握的证据),以及可能导致医院临床疗效的轻微提高(低确定性的证据)。

-对公众揭露个体卫生保健提供者的表现可能导致人们选择提供更好质量等级服务的提供者(有适度把握的证据)。

-没有研究评估了利益相关者在政策和组织决定上的参与所造成的影响。

此概述的时效性如何?

此概述的作者研究了2016年12月17日前已发表的系统综述。

翻译注解

译者:臧琰,审校:梁宁。北京中医药大学循证医学中心。2017年12月15日。

Streszczenie prostym językiem

Efekty zarządzania systemami opieki zdrowotnej w krajach o niskich dochodach

Jaki jest cel tego przeglądu?

Celem niniejszego przeglądu Cochrane jest zapewnienie pełnego podsumowania informacji na temat skutków wdrażania różnych strategii zarządzania systemami opieki zdrowotnej w krajach o niskich dochodach.

Do przeglądu włączono 19 istotnych przeglądów systematycznych. W przeglądach tych oceniano różne strategie zarządzania systemami opieki zdrowotnej. Łącznie włączono do nich 172 badania.

Niniejszy przegląd jest jednym z czterech przeglądów tego typu (Cochrane Overview), którego celem była ocena organizacji systemów opieki zdrowotnej.

Główne wyniki

Jakie są efekty stosowania różnych strategii organizacji zarządzania i odpowiedzialności za politykę zdrowotną?

Do analizy włączono trzy przeglądy, na podstawie których wyciągnięto następujące wnioski:

- współpraca między placówkami służby zdrowia na poziomie lokalnym i innymi organami samorządu terytorialnego może prowadzić do niewielkiej lub żadnej różnicy w zakresie poprawy zdrowia fizycznego oraz jakości życia pacjentów (dane naukowe o niskiej pewności co do ich wiarygodności);

- wprowadzanie maksymalnej kwoty refundacji leków przez systemy ubezpieczeń zdrowotnych prawdopodobnie wpływa na zmniejszenie spożycia leków oraz związanych z tym wydatków (dane naukowe o umiarkowanej pewności co do ich wiarygodności).

- nie ma pewności czy działania, których celem jest zapobieganie, wykrywanie i reagowanie na oszustwa w opiece zdrowotnej są skuteczne w ograniczeniu nadużyć i związanych z nimi wydatków (dane naukowe o bardzo niskiej pewności co do ich wiarygodności).

Jakie są efekty stosowania różnych strategii organizacji zarządzania i odpowiedzialności w organizacjach zajmujących się opieką zdrowotną?

Do analizy włączono dwa przeglądy, na podstawie których wyciągnięto następujące wnioski:

- Zawieranie umów na świadczenie usług zdrowotnych z niepublicznymi dostawcami opieki zdrowotnej lub organizacjami typu non-profit może poprawić dostęp do świadczeń i wyniki zdrowotne pacjentów oraz obniżyć wydatki gospodarstwa domowego na zdrowie (dane naukowe o niskiej pewności co do ich wiarygodności). Nie odnaleziono danych naukowych, które pozwoliłyby ocenić, czy zlecanie świadczeń zewnętrznym dostawcom było bardziej skuteczne niż wykorzystanie tych samych środków w sektorze państwowym.

Jakie są efekty stosowania różnych strategii organizacji zarządzania i odpowiedzialności w przypadku produktów komercyjnych, takich jak leki i wyroby medyczne?

Do analizy włączono trzy przeglądy, na podstawie których wyciągnięto następujące wnioski:

- systemy opieki zdrowotnej, w których Światowa Organizacja Zdrowia (WHO) zatwierdza rejestrację (wstępna kwalifikacja) producentów oraz leków (tj. w których organy regulacyjne prowadzą kontrolę producentów leków w celu zapewnienia spełnienia międzynarodowych standardów) mogą wpływać na zmniejszenie odsetka leków, które nie spełniają norm lub są podrabiane (dane naukowe o niskiej pewności co do ich wiarygodności);

- ustanowienie maksymalnej kwoty refundacji dla leków dostępnych w aptece w ramach ubezpieczenia zdrowotnego, może wpłynąć na zwiększenie zużycia leków generycznych oraz zmniejszenie zużycia leków oryginalnych (dane naukowe o niskiej pewności co do ich wiarygodności); odnalezione dowody nie pozwalają ocenić czy podejście to wpływa na całkowitą kwotę wydatków na leki (dane naukowe o bardzo niskiej pewności co do ich wiarygodności);

- reklama bezpośrednia, skierowana do pacjentów wpływa na zwiększenie liczby próśb pacjentów o przepisanie leku oraz zwiększenie liczby wydawanych recept (dane naukowe o dużej pewności co do ich wiarygodności).

Jakie są efekty stosowania różnych strategii organizacji zarządzania i odpowiedzialności wśród podmiotów świadczących opiekę zdrowotną?

Do analizy włączono siedem przeglądów, na podstawie których wyciągnięto następujące wnioski:

- szkolenia dla managerów opieki zdrowotnej mogą poszerzyć ich wiedzę dotyczącą planowania procesów oraz rozwinąć umiejętności z zakresu ich monitorowania i ewaluacji (dane naukowe o niskiej pewności co do ich wiarygodności);

- zmniejszenie ograniczeń imigracyjnych w krajach o wysokich dochodach prawdopodobnie zwiększa do nich migrację pielęgniarek z krajów o niskich i średnich dochodach (dane naukowe o umiarkowanej pewności co do ich wiarygodności);

- nie ma pewności czy kontrola prowadzona przez instytucje zewnętrzne, oceniająca przestrzeganie standardów jakości, poprawia stosowanie się do zaleceń (standardów), jakość świadczenia opieki zdrowotnej oraz wskaźniki zakażeń szpitalnych (dane naukowe o bardzo niskiej pewności co do ich wiarygodności).

Jakie są efekty stosowania różnych strategii organizacji zaangażowania uczestników systemu w zarządzanie opieką zdrowotną?

Do analizy włączono cztery przeglądy, na podstawie których wyciągnięto następujące wnioski:

- szkolenia bezpośrednie oraz grupy działania dla kobiet prawdopodobnie wpływają na poprawę przeżycia noworodków (dane naukowe o umiarkowanej pewności co do ich wiarygodności) i mogą wydłużać przeżycie wśród matek (dane naukowe o niskiej pewności co do ich wiarygodności);

- upublicznianie danych dotyczących jakości systemów ubezpieczenia zdrowotnego może prowadzić do wybierania przez pacjentów ubezpieczeń zdrowotnych posiadających wyższe oceny jakości oraz unikania ubezpieczeń charakteryzujących się gorszymi ocenami; rozwiązanie to może prowadzić do nieznacznej poprawy wyników zdrowotnych ubezpieczyciela (dane naukowe o niskiej pewności co do ich wiarygodności);

- upublicznianie danych dotyczących jakości opieki szpitalnej może w niewielkim lub żadnym stopniu wpływać na wybór szpitala przez pacjentów (dane naukowe o niskiej pewności co do ich wiarygodności); strategia ta prawdopodobnie zachęca szpitale do wdrażania działań mających na celu poprawę jakości świadczonych usług (dane naukowe o umiarkowanej pewności co do ich wiarygodności) i może prowadzić do nieznacznej poprawy wyników zdrowotnych w danym szpitalu (dane naukowe o niskiej pewności co do ich wiarygodności);

- upublicznianie danych dotyczących wydajności poszczególnych świadczeniodawców prawdopodobnie wpływa na wybór przez pacjenta świadczeniodawcy o wyższej ocenie jakości usług medycznych (dane naukowe o umiarkowanej pewności co do ich wiarygodności).

Nie odnaleziono badań, w których oceniano skutki udziału uczestników systemu opieki zdrowotnej w podejmowaniu decyzji dotyczących polityki zdrowotnej i organizacji służby zdrowia.

Jak aktualny jest ten przegląd ?

Autorzy przeglądu przeszukali przeglądy systematyczne opublikowane do 17 grudnia 2016 r.

Uwagi do tłumaczenia

Tłumaczenie: Aleksandra Pelczarska Redakcja: Karolina Moćko

Background

This is one of four overviews of systematic reviews of strategies for improving health systems in low-income countries (Ciapponi 2014; Pantoja 2014; Wiysonge 2014). The aim is to provide broad overviews of the evidence about the effects of delivery, financial and governance arrangements, and implementation strategies. This overview addresses governance arrangements.

We summarise the scope of each of the four overviews below.

  1. Delivery arrangements include changes in who receives care and when, who provides care, the working conditions of those who provide care, coordination of care amongst different providers, where care is provided, the use of information and communication technology to deliver care, and quality and safety systems (Ciapponi 2014).

  2. Financial arrangements include changes in how funds are collected, insurance schemes, how services are purchased, and the use of targeted financial incentives or disincentives (Wiysonge 2014).

  3. Governance arrangements include changes in rules or processes that determine authority and accountability for health policies, organisations, commercial products and health professionals, and the involvement of stakeholders in decision-making.

  4. Implementation strategies include interventions designed to bring about changes in healthcare organisations, the behaviour of healthcare professionals or the use of health services by healthcare recipients (Pantoja 2014).

The term 'governance' has been defined in several ways, as illustrated in Table 1. Although these definitions overlap, they may create confusion. We have defined governance here as rules or processes that affect the way in which powers are exercised, particularly with regard to authority, accountability, openness, participation, and coherence. Governance includes processes and institutions through which individuals and groups "articulate their interests, mediate their differences and exercise their legal rights and obligations" (Siddiqi 2009). Our focus accordingly is on the effects of governance arrangements to achieve health and related goals, such as efficiency, equity, human rights, responsiveness and fairness (Murray 2000). Attributes such as accountability, openness and participation can also be goals in and of themselves. For example, the World Health Organization (WHO)'s Declaration of Alma-Ata states that "The people have a right and duty to participate individually and collectively in the planning and implementation of their health care" (WHO 1978). Governance arrangements can potentially affect patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use, healthcare provider outcomes (such as sick leave) and social outcomes (such as poverty or employment) (EPOC 2017). Impacts on these outcomes can be intended and desirable, or unintended and undesirable. In addition, the effects of delivery arrangements on these outcomes can either reduce or increase inequities. Health systems in low-income countries differ from those in high-income countries in terms of the availability of resources and access to services. Thus, some problems in high-income countries are not relevant to low-income countries, such as governance arrangements that rely on expensive technologies that are not available in low-income countries. Similarly, some problems in low-income countries are not relevant to high-income countries, such as policies that regulate emigration of health workers. Our focus in this overview is specifically on governance arrangements in low-income countries, by which we mean countries that the World Bank classifies as low- or lower-middle-income (World Bank Group 2016). Because upper-middle-income countries often have a mixture of health systems with problems similar to both those in low-income countries and high-income countries, our focus is relevant to middle-income countries but excludes consideration of conditions that are not relevant in low-income countries and are relevant in middle-income countries.

Table 1. Definitions of governance and of stewardship
Governance: definitions
  • Governance is about oversight and guidance of the whole system. Governance and leadership involve ensuring strategic policy frameworks exist and are combined with effective oversight, coalition building, regulation, attention to system design and accountability. It is about the role of the government in health and its relation to other actors whose activities impact on health. This involves overseeing and guiding the whole health system, private as well as public, in order to protect the public interest. While ultimately it is the responsibility of government, this does not mean all leadership and governance functions have to be carried out by central ministries of health (WHO 2007).

  • Governance is defined as policy guidance to the whole health system, coordination between actors and regulation of different functions, levels and actors in the system, an optimal allocation of resources and accountability towards all stakeholders. Although many actors have an influence on governance, there is a central role for the state in ensuring equity, efficiency and sustainability of the health system (Van Olmen 2010).

  • The process of collective action that organises the interaction between actors, the dynamics of processes and the rules of the game (informal and formal), with which a society determines its behaviour and makes its decisions (Hufty 2006).

  • Governance is ultimately concerned with creating the conditions for ordered rule and collective action (Stoker 1998).

  • The traditions and institutions by which authority in a country is exercised. This considers the process by which governments are selected, monitored and replaced; the capacity of the government to effectively formulate and implement sound policies and the respect of citizens and the state of the institutions that govern economic and social interactions among them (World Bank Group 2013).

  • In broad terms, governance can be defined as the actions and means adopted by a society to promote collective action and deliver collective solutions in pursuit of common goals. Health governance concerns the actions and means adopted by a society to organise itself in the promotion and protection of the health of its population. The rules defining such organisation and its functioning can be formal or informal. Governance mechanisms can be situated at the local/subnational, national, regional, international or global level. Health governance can be public, private, or a combination of the two (Dogson 2002).

  • Simply put, governance is the association of citizens, experts, and elected representatives in the creation and implementation of policies. It is the combination of these three elements – citizens, experts and representatives – that distinguishes governance from politics and management, two concepts that are also used in societies and organisations to describe the way policies are created and implemented (Forest 1999).

  • Governance is not synonymous with government. Both refer to purposive behaviour, to goal-oriented activities, to systems of rule; but government suggests activities that are backed by formal authority, whereas governance refers to activities backed by shared goals that may or may not derive from legal and formally prescribed responsibilities and that do not necessarily rely on police powers to overcome defiance and attain compliance (Rosenau 1995).

  • The activity of governing relates to decisions that define expectations, grant power, or verify performance. It consists either of a separate process or of a specific part of management or leadership processes. Sometimes people set up a government to administer these processes and systems (Wikipedia 2011).

  • Governance is the combination of political, social, economic and institutional factors that affect the behavior of organisations and individuals and influence their performance (Savedoff 2011).

Stewardhip: definitions and features distinguishing it from governance

Stewardship is similar to the concept of public governance but, as envisaged by the WHO, is more specifically focused on the state's role in taking responsibility for the health and well-being of the population, and guiding the health system as a whole (Travis 2003). Stewardship has been described as one of the four basic functions of health system organisations (Murray 2000). The other three functions in this model are financing, provision, and resource generation. Definitions of stewardship include the following.

  • The term 'stewardship', as it relates to the state, has been defined in various related ways. The definitions reflect concerns similar to those underpinning the WHO World Health Report 2000 (WHO 2000), which views stewardship as "the effective trusteeship of national health". They all indicate stewardship to be a particular type of governance linked with agency theory and the concomitant role of the state as an agent for its citizens. The most basic approach defines stewardship as "the disinterested performance of a duty by government and/or its agents on behalf of a superior". The notion of stewardship can be viewed as an ethically informed or 'good' form of governance. Saltman 2000 defines governance as having very similar functions to stewardship.

  • Stewardship incorporates much of what is described as (public) governance. Stewardship differs from governance more in its style or approach to particular tasks than in its scope. More specifically, stewardship is 'good', 'ethical', 'inclusive' or 'proactive' governance (Murray 2000).

  • Stewardship is the function of a government responsible for the welfare of the population and concerned about the trust and legitimacy with which its activities are viewed by the citizenry (WHO 2000).

  • Stewardship goes beyond the conventional notion of regulation. It involves three key aspects: setting, implementing and monitoring the rules for the health system; assuring a level playing field for all actors in the system (particularly purchasers, providers and patients); and defining strategic directions for the health system as a whole. To deal with these aspects, stewardship can be subdivided into 6 sub-functions: overall system design, performance assessment, priority setting, intersectoral advocacy, regulation, and consumer protection (Murray 2000).

Description of the interventions

It is possible to categorise alternative governance arrangements in a number of ways. For example, Health Systems Evidence (Lavis 2015) uses the following categories: policy authority, organisational authority, commercial authority, professional authority, and consumer and stakeholder involvement. Frenk 2013 and Murray 2000, as noted in Table 1, have described six sub-functions of stewardship (a particular type of governance): overall system design, performance assessment, priority setting, intersectoral advocacy, regulation and consumer protection. Furthermore, WHO has identified three basic tasks of stewardship (WHO 2000): formulating health policy (defining the vision and direction), exerting influence (approaches to regulation), and collecting and using intelligence. The types of interventions that we include in this overview are listed in Table 2 using a structure derived from the taxonomy developed by Lavis 2015. We used this framework as our starting point because it is not limited to stewardship, and it is comprehensive and detailed. We adapted the framework in order to clarify the classification of interventions where this was ambiguous.

Table 2. Types of governance arrangements
Governance arrangementDefinition
Authority and accountability for health policies
Interagency collaborationCollaboration and partnerships for health and social development between the health sector and other different sectors
Centralisation and decentralisationPolicies to regulate the degree of which managerial responsibilities are transferred to regional or local authorities in contrast to having them at the central level
District managementPolicies that regulate the management of district health systems
Decision-making about what or who is covered by health insuranceProcesses for deciding what is reimbursed and who is covered by health insurance
Policies to reduce corruptionPolicies for reducing corruption in the health sector
Policies to manage absenteeismRegulations for managing absenteeism of health professionals
Requirements for monitoring or evaluationPolicies that regulate programme monitoring and evaluation
Authority and accountability for organisations
OwnershipPolicies that regulate who can own health service organisations
Stewardship of private health servicesPolicies that regulate health services provided by the private sector
InsurancePolicies that regulate the provision of insurance (e.g. who can provide insurance, mandatory open enrolment, coverage of essential drugs)
AccreditationProcesses for accrediting healthcare providers
Multi-institutional arrangementsPolicies for how multiple organisations work together
Authority and accountability for commercial products
RegistrationProcedures for registering or licensing commercial products (e.g. drugs)
Patents and profitsPolicies that regulate patents and profits
Pricing and purchasing policiesPolicies that determine the price that is paid or how commercial products are purchased
Marketing regulationsPolicies that regulate marketing of commercial products
Sales and dispensingPolicies that regulate the sale and dispensing of drugs or other healthcare products
Liability for commercial productsPolicies that regulate liability for commercial products
Authority and accountability for health professionals
Training and licensingPolicies that regulate training and licensure requirements for health professionals
Scope of practicePolicies that regulate what health professionals can do
Recruitment and retention strategiesPolicies that regulate where health professionals work (e.g. restrictions on where they can work or requirements to work in rural areas)
Emigration and immigration policiesPolicies that regulate emigration and immigration of health professionals
Dual practicePolicies that regulate dual practice, in which health workers hold two or more jobs, for example in both the public or private sectors
Quality of practicePolicies or systems for assuring quality of care
Professional competencePolicies or procedures for assuring professional competence
Policies to manage absenteeismPolicies for managing absenteeism of health professionals
Professional liabilityPolicies that regulate liability for health professionals
Stakeholder involvement
Stakeholder participation in policy and organisational decisionsPolicies and procedures for involving stakeholders in decision-making
Community mobilisationProcesses that enable people to organise themselves
Community monitoringMonitoring of health services by individuals or community organisations
Patient informationPolicies that regulate what information is provided to patients
Patients' rightsPolicies that regulate patients' rights, including access to care and information

How the intervention might work

Changes in governance arrangements can affect health and related goals in multiple ways. Generally, this is likely to occur through changes in authority, accountability, openness, participation, and coherence (promotion of mutually reinforcing policy actions). Table 3 presents examples of how changes in different types of governance arrangements might lead to better healthcare outcomes.

Table 3. Examples of how changes in governance arrangements might work
Governance arrangement Definition
Authority and accountability for health policies
Interagency collaborationPolicies to facilitate interagency collaboration, for instance, between local government and local health authorities in order to address social determinants of health, can contribute to improve health of the population.
Decentralisation and centralisationShifting authority closer to those who are affected might improve accountability, openness and participation, which might in turn lead to more appropriate priorities, more efficiency and less corruption, and in turn better health outcomes.
District managementRegulations that lead to improvements in the management of district health systems can improve access to and the quality of care, and in turn better health outcomes.
Decision-making about what or who is covered by health insuranceChanges in processes used to decide what is reimbursed or who is covered by health insurance might improve access to cost-effective interventions, and in turn lead to better health outcomes.
Policies to reduce corruptionRegulations that reduce corruption can increase the availability of resources for care, and in turn improve health outcomes.
Requirements for monitoring or evaluationPolicies that improve decisions about when and how healthcare programmes are monitored or evaluated can lead to better-informed decisions, and in turn better health outcomes.
Authority and accountability for organisations
OwnershipFor-profit health services might limit access for people who cannot afford to pay or divert funds from care to profits and taxes, which might result in poorer quality care and worse health outcomes.
Stewardship of private health servicesRegulations that increase the accountability of the private sector might improve the quality of care, and in turn lead to better health outcomes.
InsuranceChanges in regulations that determine who can provide insurance, who receives it, who pays for it, and who makes decisions about reimbursement might affect coverage and access to care, and in turn health outcomes.
AccreditationChanges in provider accreditation might improve the quality of care, and in turn health outcomes.
Multi-institutional arrangementsChanges in how donors and governments work together might result in more effective and efficient use of resources, and in turn lead to better health outcomes.
Authority and accountability for commercial products
RegistrationChanges in how drugs or other health technologies are licensed might improve safety, and in turn health outcomes,
Patents and profitsChanges in patent regulations might affect the development and availability of drugs or other health technologies, and in turn health outcomes.
Pricing and purchasing policiesRegulations that reduce the price that is paid or how drugs or services are purchased might improve access to care, and in turn health outcomes.
Marketing regulationsRegulations that limit inappropriate marketing of drugs, other technologies or services might reduce inappropriate use and increase the availability of resources for cost-effective care, and in turn improve health outcomes.
Sales and dispensingChanges in who can sell drugs or other healthcare products might improve access or improve quality, and in turn health outcomes.
Liability for commercial productsChanges in liability for drugs, other technologies or services might improve safety, and in turn health outcomes.
Authority and accountability for health professionals
Training and licensingRegulations that improve training or licensure of health professionals might improve the safety and quality of care, and in turn health outcomes.
Scope of practiceRegulations that determine what health professionals can do might improve access to care or safety, and in turn health outcomes.
Recruitment and retention strategiesRegulations that determine where health professionals can work might improve access to care, and in turn health outcomes.
Emigration and immigration policiesRegulations that determine emigration or immigration of health professionals might improve access to care, and in turn health outcomes.
Dual practiceRegulations that affect the extent of dual practice might improve access to care, and in turn health outcomes.
Quality of practicePolicies or systems for assuring quality of care might improve the quality of care, and in turn health outcomes.
Professional competencePolicies or procedures for assuring professional competence might improve the safety and quality of care, and in turn health outcomes.
Policies to manage absenteeismRegulations that reduce absenteeism can improve access to care, and in turn health outcomes.
Professional liabilityChanges in liability for health professionals might improve safety or remove impediments to evidence-based care, and in turn improve health outcomes.
Stakeholder involvement
Stakeholder participation in policy and organisational decisionsInvolving stakeholders in decision-making might improve the overall decision-making about how to use resources and organise care, and in turn lead to better health outcomes.
Community mobilisationProcesses that enable people to organise themselves might raise awareness, change behaviours and lead to improvements in access and utilisation of health services, and in turn improve health outcomes.
Community monitoringMonitoring of health services by individuals or community organisations might help to ensure quality, improve access to care, and reduce corruption, and in turn improve health outcomes.
Patient informationRegulations that improve the extent to which patients are well-informed might lead to better informed decisions, and in turn improve health outcomes.
Patients' rightsPolicies that regulate patients' rights, such as access to care and information, might improve access and utilisation of health services and improve the quality of health services, and in turn improve health outcomes.

Why it is important to do this overview

Our objective is to provide a broad overview of current evidence from systematic reviews evaluating the effects of alternative governance arrangements for health systems in low-income countries. We recognise that there is a paucity of research that has evaluated the effects of governance arrangements (Bennington 2010; Frenk 2013). Nonetheless, a broad overview of the findings of systematic reviews can help policymakers, their technical support staff and other stakeholders to identify strategies for addressing problems with the governance of their health systems. It can also help to identify needs and priorities for evaluations of governance arrangements, as well as priorities for systematic reviews of the effects of governance arrangements. The overview also helps to refine the framework outlined in Table 2 for considering alternative health system arrangements for allocating authority and ensuring accountability, openness, participation and coherence.

Our focus is specifically on low-income countries in this overview because there are structural differences in health systems and country contexts compared to middle- and high-income countries. These differences make it difficult to select, analyse and summarise the evidence for low-, middle- and high-income countries in a single overview. By focusing on low-income countries, we were able to exclude reviews that are not relevant to those countries and to consistently address the relevance of the evidence in included reviews for those countries. This makes the overview more helpful for people making decisions about governance arrangements in low-income countries.

Changes in health systems are complex. They may be difficult to evaluate, the applicability of the findings of evaluations from one setting to another may be uncertain, and synthesising the findings of evaluations may be difficult. However, the alternative to well-designed evaluations is poorly designed evaluations; the alternative to systematic reviews is non-systematic reviews; and the alternative to using the findings of systematic reviews to inform decisions is making decisions without the support of this rigorous evidence. Policymakers still need other types of information, including context specific information and judgments (e.g. judgments about the applicability of the findings of systematic reviews in a specific context) when making decisions about governance arrangements.

This overview can help people making decisions about governance arrangements by summarising the findings of available systematic reviews, including estimates of the effects of changes in governance arrangements and the certainty of those estimates, by identifying important uncertainties identified by those systematic reviews and by identifying where new or updated systematic reviews are needed. The overview can also help to inform judgments about the relevance of the available evidence in a specific context (Rosenbaum 2011).

Objectives

To provide an overview of the available evidence from up-to-date systematic reviews about the effects of governance arrangements for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on governance arrangements and informing refinements of the framework for governance arrangements outlined in the overview (Table 2).

Methods

We used the methods described below in all four overviews of health system arrangements and implementation strategies in low-income countries (Ciapponi 2014; Pantoja 2014; Wiysonge 2014).

Criteria for considering reviews for inclusion

We included systematic reviews that:

  • assessed the effects of governance arrangements (as defined in the Background);

  • had a Methods section with explicit selection criteria;

  • reported at least one of the following types of outcomes: patient outcomes (health and health behaviors), the quality or utilisation of healthcare services, resource use (health expenditures, healthcare provider costs, out-of-pocket payments, cost-effectiveness), healthcare provider outcomes (such as sick leave, burnout), or social outcomes (such as poverty, employment);

  • were relevant to low-income countries as classified by the World Bank (World Bank Group 2016);

  • were published after April 2005.

Judgments about relevance to low-income countries are sometimes difficult to make, and we are aware that evidence from high-income countries is not directly generalisable to low-income countries. We based our judgments on an assessment of the likelihood that the governance arrangements considered in a review address a problem that is important in low-income countries, would be feasible, and would be of interest to decision-makers in low-income countries, regardless of where the included studies took place. So, for example, we excluded arrangements that require technology that is not widely available in low-income countries. At least two of the overview authors made judgments about the relevance to low-income countries and discussed with the other authors whenever there was uncertainty. Reviews that only included studies from a single high-income country were not eligible due to concerns about the wider applicability of the findings of such reviews. However, we did consider reviews that only included studies from high-income countries if the interventions were relevant for low-income countries.

We excluded reviews published before April 2005 as these were highly unlikely to be up-to-date. We also excluded reviews that had methodological limitations that were important enough to compromise the reliability of the review findings (Appendix 1).

Search methods for identification of reviews

We searched Health Systems Evidence in November 2010 using the following filters.

  1. Health system topics = governance arrangements.

  2. Type of synthesis = systematic review or Cochrane Review.

  3. Type of question = effectiveness.

  4. Publication date range = 2000 to 2010.

We conducted subsequent searches using PDQ ('pretty darn quick')-Evidence, which was launched in 2012. We searched PDQ up to 17 December 2016, using the filter 'Systematic Reviews' with no other restrictions. We updated that search, excluding records that were entered into PDQ-Evidence prior to the date of the last previous search.

PDQ-Evidence is a database of evidence for decisions about health systems, which is derived from the Epistemonikos database of systematic reviews (Rada 2013). It includes systematic reviews, overviews of reviews (including evidence-based policy briefs) and studies included in systematic reviews. Epistemonikos and PDQ-Evidence incorporate searches from the following databases with no language or publication status restrictions.

  1. Cochrane Database of Systematic Reviews (CDSR).

  2. PubMed.

  3. Embase.

  4. Database of Abstracts of Reviews of Effectiveness (DARE).

  5. Health Technology Assessment Database.

  6. CINAHL.

  7. LILACS.

  8. PsycINFO.

  9. Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) Evidence Library.

  10. 3ie Systematic Reviews and Policy Briefs.

  11. World Health Organization (WHO) Database.

  12. Campbell Library.

  13. Supporting the Use of Research Evidence (SURE) Guides for Preparing and Using Evidence-Based Policy Briefs.

  14. European Observatory on Health Systems and Policies.

  15. UK Department for International Development (DFID).

  16. National Institute for Health and Care Excellence (NICE) public health guidelines and systematic reviews.

  17. Guide to Community Preventive Services.

  18. Canadian Agency for Drugs and Technologies in Health (CADTH) Rx for Change.

  19. McMaster Plus KT+.

  20. McMaster Health Forum Evidence Briefs.

Appendix 2 presents the detailed search strategies for PubMed, LILACS, Embase, CINAHL and PsycINFO. We screened all records in the other databases. PDQ staff and volunteers update these searches weekly for Pubmed and monthly for the other databases, screening records continually, and adding new reviews to the database daily.

In addition, we screened all of the Cochrane Effective Practice and Organisation of Care (EPOC) Group reviews in Archie (i.e. Cochrane's central server for managing documents) and the reference lists of relevant policy briefs and overviews of reviews.

Data collection and analysis

Selection of reviews

Two of the overview authors (CH and SL) independently screened the titles and abstracts found in PDQ-Evidence to identify reviews that appeared to meet the inclusion criteria. Two other authors (AO and SL) screened all of the titles and abstracts that we could not confidently include or exclude after the first screening to identify any additional eligible reviews. One of the overview authors screened the reference lists (CH).

One of the overview authors applied the selection criteria to the full text of potentially eligible reviews and assessed the reliability of reviews that met all of the other selection criteria (CH) (Appendix 1). Two other authors (AO or SL) independently checked these judgments.

Data extraction and management

We summarised each included review using the approach developed by the SUPPORT collaboration (Rosenbaum 2011). We used standardised data extraction forms to extract data on the background of the review: interventions, participants, settings and outcomes; key findings; and considerations of applicability, equity, economic considerations, and monitoring and evaluation. We assessed the certainty of the evidence for the main comparisons using the GRADE approach (Guyatt 2008; Schünemann 2011a; Schünemann 2011b; EPOC 2016).

Each completed SUPPORT Summary underwent peer-review and was published on the SUPPORT Summaries website, where we provide details about how we prepared the summaries and how we assessed the applicability of the findings, impacts on equity, economic considerations, and the need for monitoring and evaluation. We describe the rationale for the criteria that we used for these assessments in the SUPPORT Tools for evidence-informed health policymaking (Fretheim 2009; Lavis 2009; Oxman 2009a; Oxman 2009b). As noted there, "a local applicability assessment must be done by individuals with a very good understanding of on-the-ground realities and constraints, health system arrangements, and the baseline conditions in the specific setting" (Lavis 2009). In this overview we have made broad assessments of the applicability of findings from studies in high-income countries to low-income countries using the criteria described in the SUPPORT summaries database, with input from people with relevant experience and expertise in low-income countries.

Assessment of methodological quality of included reviews

We assessed the reliability of systematic reviews that met our inclusion criteria using criteria developed by the SUPPORT and SURE collaborations (Appendix 2; SUPPORT 2009, SURE 2011). Based on these criteria, we categorised each review as having:

  • only minor limitations;

  • limitations that are important enough that it would be worthwhile to search for another systematic review and to interpret the results of this review cautiously, if no better review is available;

  • limitations that are important enough to compromise the reliability of the review and prompt its exclusion from the overview.

Data synthesis

We describe the methods used to prepare a SUPPORT Summary of each review in detail on the SUPPORT Summaries website. Briefly, for each included systematic review, we prepared a table summarising what the review authors searched for and what they found (Appendix 3), we prepared 'Summary of findings' tables for each main comparison, and we assessed the relevance of the findings for low-income countries. The SUPPORT Summaries include key messages, important background information, a summary of the findings of the review and structured assessments of the relevance of the review for low-income countries. We subjected the SUPPORT Summaries to review by the lead author of each review, at least one content area expert, people with practical experience in low-income settings, and a Cochrane EPOC Group editor (AO or SL). The authors of the SUPPORT Summaries responded to each comment and made appropriate revisions, and the summaries underwent copy-editing. The editor determined whether the comments had been adequately addressed and whether the summary was ready for publication on the SUPPORT Summary website.

We organised the review by modifying the taxonomy for health systems arrangements used by Health Systems Evidence (Lavis 2015), adjusting this framework iteratively to ensure that we appropriately categorised all of the included reviews and that we included and logically organised all relevant health system governance arrangements. We prepared a table listing the included reviews as well as the types of governance arrangements for which we were not able to identify a reliable, up-to-date review (Table 4). We also prepared a table of excluded reviews (Table 5), describing reviews that addressed a question for which another (more up-to-date or reliable) review was available, reviews that were published before April 2005 (for which a SUPPORT Summary was available), reviews with results that we did not consider transferable to low-income countries, and reviews with limitations that were important enough to compromise the reliability of the review findings.

Table 4. Included reviews
Governance arrangement Included reviews
Authority and accountability for health policies
Interagency collaborationCollaboration between local health and local government agencies for health improvement (Hayes 2012)
Decentralisation and centralisationNo eligible systematic review found
District managementNo eligible systematic review found
Decision-making about what or who is covered by health insurance
Policies that regulate what drugs are reimbursedNo eligible systematic review found
Policies that regulate what services are reimbursedNo eligible systematic review found
Restrictions on drug reimbursementPharmaceutical policies: effects of restrictions on reimbursement (Green 2010)
Restrictions on reimbursement for health insuranceNo eligible systematic review found
Strategies for expanding health insurance coverageNo eligible systematic review found
Policies to reduce corruptionNo evidence of the effect of the interventions to combat health care fraud and abuse: a systematic review of literature (Rashidian 2012)
Policies to manage absenteeismNo eligible systematic review found
Requirements for monitoring or evaluationNo eligible systematic review found
Authority and accountability for organisations
OwnershipNo eligible systematic review found
Stewardship of private health servicesNo eligible systematic review found
Contracting outThe impact of contracting out on health outcomes and use of health services in low and middle income countries (Lagarde 2009)
AccreditationNo eligible systematic review found
Regulation of insurance provision
Provision of drug insuranceNo eligible systematic review found
Provision of health insuranceNo eligible systematic review found
Multi-institutional arrangements
Policies that regulate interactions between donors and governmentsNo eligible systematic review found
Social franchisingThe effect of social franchising on access to and quality of health services in low- and middle-income countries (Koehlmoos 2009)
Governance arrangements for coordinating care across multiple providersNo eligible systematic review found
MergersNo eligible systematic review found
Authority and accountability for commercial products
Registration
DrugsInterventions to combat or prevent drug counterfeiting: a systematic review (El-Jardali 2015)
Health technologyNo eligible systematic review found
Patents and profits
DrugsNo eligible systematic review found
Health technologyNo eligible systematic review found
Pricing and purchasing policies
DrugsPharmaceutical policies: effects of reference pricing, other pricing, and purchasing policies (Acosta 2014)
Health technology and servicesNo eligible systematic review found
Marketing regulations
DrugsBenefits and harms of direct to consumer advertising: a systematic review (Gilbody 2005)
Health technology and servicesNo eligible systematic review found
Sales and dispensing
DrugsNo eligible systematic review found
Health technologyNo eligible systematic review found
Liability for commercial productsNo eligible systematic review found
Authority and accountability for health professionals
Training and licensing
Pre-licensure educationEffects of changes in the pre-licensure education of health workers on health-worker supply (Pariyo 2009)
Training district health system managersInterventions for hiring, retaining and training district health system managers in low- and middle-income countries (Rockers 2013)
LicensureNo eligible systematic review found
Specialty certificationNo eligible systematic review found
Scope of practiceNo eligible systematic review found
Recruitment and retention strategiesInterventions for increasing the proportion of health professionals practising in underserved communities (Grobler 2015)
Recruitment and retention strategiesInterventions for hiring, retaining and training district health system managers in low- and middle-income countries (Rockers 2013)
Movement of health workers between public and private organisationsFinancial interventions and movement restrictions for managing the movement of health workers between public and private organizations in low- and middle-income countries (Rutebemberwa 2014)
Emigration and immigration policiesInterventions for controlling emigration of health professionals from low- and middle-income countries (Peñaloza 2011)
Dual practiceInterventions to manage dual practice among health workers (Kiwanuka 2011)
Authority and accountability for quality of practice
Authority and accountability for quality of outpatient careExternal inspection versus external standards for improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes (Flodgren 2011)
Authority and accountability for quality assurance of hospital (inpatient) careExternal inspection versus external standards for improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes (Flodgren 2011)
Professional competenceNo eligible systematic review found
Professional liabilityNo eligible systematic review found
Stakeholder involvement
Stakeholder participation in policy and organisational decisionsMethods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material (Nilsen 2010)
Community mobilisationWomen's groups practicing participatory learning and action to improve maternal and newborn health in low-resource settings: a systematic review and meta-analysis (Prost 2013)
What do community-based dengue control programmes achieve? A systematic review of published evaluations (Heintze 2007)
Community monitoringNo eligible systematic review found
Patient information
Drug informationNo eligible systematic review found
Public disclosure of performance dataSystematic review: the evidence that publishing patient care performance data improves quality of care (Fung 2008)
Patients’ rightsNo eligible systematic review found
Table 5. Excluded reviews
Review ID Excluded reviews Reasons for exclusion
Bärnighausen 2009Financial incentives for return of service in underserved areas: a systematic reviewAddressed by Grobler 2015
Berendes 2011Quality of private and public ambulatory health care in low and middle income countries: systematic review of comparative studiesAddressed by upcoming Herrera 2013
Boote 2002Consumer involvement in health research: a review and research agendaMore than 10 years out of date
Comondore 2009Quality of care in for-profit and not-for-profit nursing homes: systematic review and meta-analysisNot transferable to low-income countries
Crawford 2002Systematic review of involving patients in the planning and development of health careAddressed by Nilsen 2010
Devereaux 2002aA systematic review and meta-analysis of studies comparing mortality rates of private for-profit and private not-for-profit hospitals.More than 10 years out of date
Devereaux 2002bComparison of mortality between private for-profit and private not-for-profit hemodialysis centersMore than 10 years out of date
Devereaux 2004Payments for care at private for-profit and private not-for-profit hospitals: a systematic review and meta-analysisNot transferable to low-income countries
Ekman 2004Community-based health insurance in low-income countries: a systematic review of the evidenceAddressed by Meng 2010
Faber 2009Public reporting in health care: how do consumers use quality-of-care information? A systematic reviewAddressed by Fung 2008
Faden 2011Active pharmaceutical management strategies of health insurance systems to improve cost-effective use of medicines in low- and middle-income countries: a systematic review of current evidence.Major limitations
Greenfield 2008Health sector accreditation research: a systematic reviewMajor limitations
Greenfield 2012The standard of healthcare accreditation standards: a review of empirical research underpinning their development and impactMajor limitations
Griffiths 2007Effectiveness of intermediate care in nursing-led in-patient unitsNot transferable to low-income countries
Henderson 2010Provision of a surgeon’s performance data for people considering elective surgeryAddressed by Fung 2008
Jia 2014Strategies for expanding health insurance coverage in vulnerable populationsScope of the Implementation overview
Lagarde 2006Evidence from systematic reviews to inform decision making regarding financing mechanisms that improve access to health services for poor people. A policy brief prepared for the International Dialogue on Evidence-Informed Action to Achieve Health Goals in Developing Countries IDEAHealthAddressed by Lagarde 2009
Lee 2009Linking families and facilities for care at birth: what works to avert intrapartum-related deaths?Major limitations
Lehmann 2008Staffing remote rural areas in middle- and low-income countries: a literature review of attraction and retentionAddressed by Grobler 2015
Liu 2008The effectiveness of contracting-out primary health care services in developing countries: a review of the evidenceAddressed by Lagarde 2009
Loevinsohn 2004Contracting for the delivery of community health services: a review of global experienceAddressed by Lagarde 2009
Marshall 2000The public release of performance data: what do we expect to gain? A review of the evidenceMore than 10 years out of date
Meng 2010Expanding health insurance coverage in vulnerable groups: a systematic review of optionsAddressed by Jia 2014
Molyneux 2012Community accountability at peripheral health facilities: a review of the empirical literature and development of a conceptual frameworkMajor limitations
Montagu 2011Private versus public strategies for health service provision for improving health outcomes in resource-limited settingsMajor limitations
Morgan 2009Comparison of tiered formularies and reference pricing policies: a systematic reviewAddressed by Acosta 2014
Ossai 2012Rural retention of human resources for healthAddressed by Grobler 2015
Patouillard 2007Can working with the private for-profit sector improve utilization of quality health services by the poor? A systematic review of the literatureMajor limitations
Patterson 2010Systematic review of the links between human resource management practices and performanceMajor limitations
Peters 2004Strategies for engaging the private sector in sexual and reproductive health: how effective are they?More than 10 years out of date
Phillips 2010Can clinical governance deliver quality improvement in Australian general practice and primary care? A systematic review of the evidenceAddressed by Flodgren 2011
Preston 2010Community participation in rural primary health care: intervention or approach?Addressed by Nilsen 2010
Puig-Junoy 2007Impact of pharmaceutical prior authorisation policies: a systematic review of the literatureAddressed by Green 2010
Ranji 2007Effects of rapid response systems on clinical outcomes: systematic review and meta-analysisScope of the Delivery overview
Schadewaldt 2011Nurse-led clinics as an effective service for cardiac patients: results from a systematic reviewMajor limitations
Shah 2011Can interventions improve health services from informal private providers in low and middle-income countries? A comprehensive review of the literatureMajor limitations
Sharp 2002Specialty board certification and clinical outcomes: the missing linkMore than 10 years out of date
Shen 2007Hospital ownership and financial performance: a quantitative research reviewNot transferable to low-income countries
Socha 2011Physician dual practice: a review of literatureAddressed by Kiwanuka 2011
Steinman 2006Improving antibiotic selection: a systematic review and quantitative analysis of quality improvement strategiesAddressed by New Reference
Tait 2004Clinical governance in primary care: a literature reviewAddressed by Phillips 2010
Wafula 2012Examining characteristics, knowledge and regulatory practices of specialised drug shops in Sub-Saharan Africa: a systematic review of the literatureNot a review of effects of interventions
Waters 2003Working with the private sector for child healthMore than 10 years out of date
Willis-Shattuck 2008Motivation and retention of health workers in developing countries: a systematic reviewNot a review of effects of interventions
Wilson 2009A critical review of interventions to redress the inequitable distribution of healthcare professionals to rural and remote areasAddressed by Grobler 2015

We described the characteristics of the included reviews in a table that included the date of the last search, any important limitations, what the review authors searched for and what they found (Appendix 3). We summarised our detailed assessments of the reliability of the included reviews in a separate table (Table 6) showing whether individual reviews met each criterion in Appendix 2.

Table 6. Reliability of included reviews
  1. a Identification, selection and critical appraisal of studies - details of assessment criteria

    1. Selection criteria: were the criteria used for deciding which studies to include in the review reported? (+ yes; ? can't tell/partially; − no)

    2. Search: was the search for evidence reasonably comprehensive? (+ yes; ? can't tell/partially; − no)

    3. Up-to-date: is the review reasonably up-to-date? (+ yes; ? can't tell/partially; − no)

    4. Study selection: was bias in the selection of articles avoided? (+ yes; ? can't tell/partially; − no)

    5. Risk of bias: did the authors use appropriate criteria to assess the risk for bias in analysing the studies that are included? (+ yes; ? can't tell/partially; − no)

    6. Overall: how would you rate the methods used to identify, include and critically appraise studies? (+ only minor limitations, − important limitations)

    b Analysis - details of assessment criteria

    1. Study characteristics: were the characteristics and results of the included studies reliably reported? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)

    2. Analytic methods: were the methods used by the review authors to analyse the findings of the included studies reported? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)

    3. Heterogeneity: did the review describe the extent of heterogeneity? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)

    4. Appropriate synthesis: were the findings of the relevant studies combined (or not combined) appropriately relative to the primary question the review addresses and the available data? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)

    5. Exploratory factors: did the review examine the extent to which specific factors might explain differences in the results of the included studies? (+ yes; ? can't tell/partially; − no, NA: not applicable; e.g. no studies or data)

    6. Overall: how would you rate the methods used to analyse the findings relative to the primary question addressed in the review? (+ only minor limitations, − important limitations)

    c Overall - details of assessment criteria

    1. Other considerations: are there any other aspects of the review not mentioned before which lead you to question the results?

    2. Reliability of the review: based on the above assessments of the methods how would you rate the reliability of the review? (+ only minor limitations, − important limitations)

ReviewA. Identification, selection and critical appraisal of studiesaB. AnalysisbC. Overallc
1. Selection criteria2. Search3. Up-to-date4. Study selection5. Risk of bias6. Overall1. Study characteristics2. Analytic methods3. Heterogeneity4. Appropriate synthesis5. Exploratory factors6. Overall1. Other considerations2. Reliability of the review
Acosta 2014++++++++++++No+
El-Jardali 2015++++++++++++No+
Flodgren 2011++++++++NA+NA+No+
Fung 2008+?++++++++++No+
Gilbody 2005+++++++++++No+
Green 2010++++++++++++No+
Grobler 2015++++++++++++No+
Hayes 2012++++++++++++No+
Heintze 2007+?+?++++++?+No+
Kiwanuka 2011++++++NANANANANANANo+
Koehlmoos 2009++++++NANANANANANANo+
Lagarde 2009++++++++++++No+
Nilsen 2010++++++++++++No+
Pariyo 2009++++++++++++No+
Peñaloza 2011++++++++++++No+
Prost 2013+++?++++++++No 
Rashidian 2012?+?+?+?+NA+No
Rockers 2013+?++++++++NA+No+
Rutebemberwa 2014+?++++NANANANANANANo+

We based our structured synthesis of the findings of our overview on two tables (Table 7; Table 8). We summarised the main findings of each review in a table that included the key messages from each SUPPORT Summary (Table 7). In a second table (Table 8), we reported the direction of the results and the certainty of the evidence for each of the following type of outcomes: health and other patient outcomes; access, coverage or utilisation; quality of care; resource use; social outcomes; impacts on equity; healthcare provider outcomes; adverse effects (not captured by undesirable effects on any of the preceding types of outcomes); and any other important outcomes (that did not fit into any of the preceding types of outcomes) (EPOC 2016). We categorised the direction of results as: a desirable effect, little or no effect, an uncertain effect (very low-certainty evidence), no included studies, an undesirable effect, not reported (i.e. not specified as a type of outcome that was considered by the review authors), or not relevant (i.e. no plausible mechanism by which the type of health system arrangement could affect the type of outcomes).

Table 7. Key messages of included reviews
Governance arrangement Key messages
Authority and accountability for health policies

Interagency collaboration

Hayes 2012

➡ Local interagency collaborative interventions may lead to little or no difference in physical health and quality of life compared with standard care.

➡ It is uncertain whether local interagency collaborative interventions decrease mortality or mental health symptoms.

➡ This review did not include any evidence from low-income countries.

Decision-making about what or who is covered by health insurance

- Restrictions on drug reimbursement

Green 2010

➡ Restrictions on reimbursement in health insurance systems with substantial coverage for medicines probably decreases targeted drug use and expenditures on targeted drugs or drug classes.

➡ The effects of restrictions on reimbursement vary by drug and drug class, and by how the restrictions are implemented and enforced.

➡ The impacts of restrictions on health outcomes and health service utilisation are uncertain.

➡ All the studies were done in high-income countries and participants were mainly senior citizens or low-income adult populations whose medications were being paid for in whole or part through publicly funded drug benefit plans.

➡ There are no studies on the effect of reimbursement restrictions on equity.

Policies to reduce corruption

Rashidian 2012

➡ It is uncertain if prevention, detection or response interventions reduce healthcare fraud and abuse and related expenditures.

➡ None of the included studies took place in a low-income country.

Authority and accountability for organisations

Contracting out

Lagarde 2009

➡ Contracting out services to non-state not-for-profit providers may increase access to and utilisation of health services.

➡ Patient outcomes may be improved and household health expenditures reduced by contracting out.

➡ None of the included studies presented evidence on whether contracting out was more effective than making a similar investment in the public sector. We are therefore uncertain of the effects of investing in contracting out compared to an equivalent investment in public sector health services.

Multi-institutional arrangements

- Social franchising

Koehlmoos 2009

➡ We found no evidence regarding the effects of social franchising on access to or the quality of health services in low- and middle-income countries. We are therefore uncertain of the effects of social franchising.

➡ There is a need for well-designed experimental studies that are informed by the theoretical and empirical literature.

Authority and accountability for commercial products

Registration

- Drugs

El-Jardali 2015

➡ Certain regulatory measures, specifically drug registration, may decrease the prevalence of counterfeit and substandard drugs. It is uncertain whether licensing of drug outlets reduces the prevalence of counterfeit drugs or the failure rates of drugs undergoing quality testing.

➡ WHO prequalification of drugs may lead to a reduction in the failure rates of drugs undergoing quality testing.

➡ Multifaceted interventions (including a mix of regulations, training of inspectors, public-private collaborations and legal actions against counterfeiters) may be effective in decreasing the prevalence of counterfeit and substandard drugs.

➡ All studies identified took place in low- and middle-income countries.

➡ The transferability of the findings may be influenced by a country's existing pharmaceutical supply chain and infrastructure, the availability of routine data on drug quality, qualified and skilled personnel, and financial resources.

Pricing and purchasing policies

- Drugs

Acosta 2014

➡ Reference pricing may reduce insurers' cumulative drug expenditures by shifting drug use from cost-share drugs to reference drugs.

➡ Index pricing may increase the use of generic drugs, reduce the use of brand-name drugs, slightly reduce the price of generic drugs, and have little or no effect on the price of brand-name drugs.

➡ It is uncertain whether maximum pricing affects drug expenditures.

➡ The effects of these policies on healthcare utilisation or health outcomes is uncertain.

➡ None of the included studies took place in a low-income country.

➡ The effects of other pharmaceutical pricing and purchasing policies are uncertain.

Marketing regulations

- Drugs

Gilbody 2005

➡ Direct-to-consumer advertising increases patient demand for advertised medicines and the number of related prescriptions by doctors.

➡ We found no studies that reported on the impact of direct-to-consumer advertising on health outcomes. We are therefore uncertain of their effects.

➡ In light of the lack of evidence of the benefits, potential harms, and costs of direct-to-consumer advertising:

- the value of policies that allow for the increased use of direct to consumer advertising is uncertain at best; and

- rigorous monitoring and evaluation are warranted if such policies are implemented.

Authority and accountability for health professionals

Training and licensing

- Pre-licensure education

Pariyo 2009

➡ There is little evidence of the effects of interventions to increase the capacity of health professional training institutions, reduce student dropout rates or increase the number of students recruited from other countries into health professional training institutions.

➡ Academic advising programmes for minority groups may:

- increase the number of minority students enrolled in health sciences;

- slightly increase retention through to graduation;

- decrease differences in retention levels through to graduation between minority and non-minority students in the health sciences.

➡ We found no studies of the effects of other pre-licensure measures to increase health worker supply.

Training and licensing

- Training district health system managers

Rockers 2013

➡ Private contracting ("contracting in") of district health managers compared to direct employment by the Ministry of Health may improve access and utilisation of healthcare. It is uncertain whether contracting in improves health outcomes.

➡ Intermittent training programmes may increase knowledge of planning processes and monitoring and evaluation skills of district managers.

➡ The effects of other interventions are uncertain, including changes in how district managers are hired, strategies for retaining district managers such as making the positions more attractive, and other training programmes such as in-service workshops with onsite support.

Recruitment and retention strategies

Grobler 2015

➡ It is uncertain whether any of the following types of interventions to recruit or retain health professionals increase the number of health professionals practising in in underserved areas

- Educational interventions (e.g. student selection criteria, undergraduate and postgraduate teaching curricula, exposure to rural and urban underserved areas)

- Financial interventions (e.g. undergraduate and postgraduate bursaries or scholarships linked to future practice location, rural allowances, increased public sector salaries)

- Regulatory strategies (e.g. compulsory community service, relaxing work regulations imposed on foreign medical graduates who are willing to work in rural or urban underserved areas)

- Personal and professional support strategies (e.g. providing adequate professional support and attending to the needs of the practitioners family)

Recruitment and retention strategies Rockers 2013

➡ Private contracting ("contracting in") of district health managers compared to direct employment by the Ministry of Health may improve access and utilisation of healthcare. It is uncertain whether contracting in improves health outcomes.

➡ Intermittent training programmes may increase knowledge of planning processes and monitoring and evaluation skills of district managers.

➡ The effects of other interventions are uncertain, including changes in how district managers are hired, strategies for retaining district managers such as making the positions more attractive, and other training programmes such as in-service workshops with onsite support.

Movement of health workers between public and private organisations

Rutebemberwa 2014

➡ No rigorous studies have evaluated the effects of interventions to manage the movement of health workers between public and private organisations.

➡ There is a need for well-designed studies to evaluate the impact of interventions that attempt to regulate health worker movement between public and private organisations in low-income countries.

Emigration and immigration policies

Peñaloza 2011

➡ Lowering immigration restrictions in high-income countries probably increases the migration of nurses from low- and middle-income countries to high-income countries. The effectiveness of interventions implemented in low- and middle-income countries to decrease the emigration of health professionals is uncertain. No studies were found that evaluated such interventions.

➡ Low- and middle-income countries should monitor changes in high-income countrie immigration legislation, model the impact of proposed migration changes on their own retention of domestic health professionals, and lobby for immigration laws in high-income countries that consider the health system needs of source countries.

➡ Rigorous studies are needed of the effectiveness of interventions designed to decrease the emigration of health professionals, particularly the effectiveness of interventions in low- and middle-income countries.

Dual practice

Kiwanuka 2011

➡ No studies met the inclusion criteria for the review, as no rigorous studies have evaluated the effects of interventions to manage dual practice.

➡ There is a need for well-designed studies to evaluate the impact of interventions that attempt to regulate health worker dual practice in low-income countries.

Authority and accountability for quality of practice

- Authority and accountability for quality of outpatient care

- Authority and accountability for quality assurance of hospital (inpatient) care

Flodgren 2011

➡ It is uncertain whether external inspection results in improved compliance with accreditation standards, improved quality of care or decreased healthcare-acquired infection (i.e. methicillin-resistant Staphylococcus aureus) rates in hospitals.

➡ This review found no direct evidence on the effectiveness of external inspections of compliance with standard in ambulatory settings. We are therefore uncertain of the effects in this setting.

➡ This review found no direct evidence on the effectiveness of external inspections of compliance with standards in low-income countries.

Stakeholder involvement

Stakeholder participation in policy and organisational decisions

Nilsen 2010

➡ Consumer consultations in developing patient information probably:

- facilitate the development of material that is more relevant, readable and understandable to patients;

- improve patient knowledge;

- make little or no difference in decreasing the anxieties that patients may associate with clinical procedures.

➡ Consumer interviewers may lead to small differences in the results of satisfaction surveys compared to healthcare professional interviewers.

➡ It is uncertain whether telephone discussions compared with face-to-face meetings change consumer priorities for community health goals.

➡ Consumer consultation in the development of consent documents may have little or no impact on self-reported participant understanding of the trial described in the consent document, satisfaction with study participation, adherence to the protocol or refusal to participate

➡ There are good arguments for introducing consumer involvement in low-income countries. To accomplish this:

- strategies to overcome barriers such as low baseline levels of social participation, organisation and education should be explored;

- efforts to include consumers or families of disadvantaged groups should be considered in order to achieve inclusive representation;

- evaluations are needed of the effects of consumer involvement on healthcare decisions and how to achieve more effective consumer involvement.

Community mobilisation

Prost 2013

➡ Women's groups practising participatory learning and action probably improve newborn survival, may improve maternal survival, may slightly reduce stillbirths, and may be a cost-effective strategy in rural areas in low- and middle-income countries.

➡ The effectiveness of women's groups may depend on participation of a substantial proportion of pregnant women, adequate supervision and support, home visits, access to care, improving the quality of care, and adequate resources.

Community mobilisation

Heintze 2007

➡ Multi-component community-based dengue control programmes may reduce mosquito larval indices.

➡ Multi-component community-based dengue control programmes combined with chemical larvicides may reduce mosquito larval indices.

➡ Multi-component community-based dengue control programmes combined with fish and chemical larvicides may reduce mosquito larval indices.

➡ Multi-component community-based dengue control programmes combined with the use of crustaceans that eat mosquito larvae (Mesocyclops copepods) may reduce mosquito larval indices.

➡ It is uncertain whether multi-component community-based dengue control programmes combined with the use of crustaceans that eat mosquito larvae (Mesocyclops copepods) reduce dengue incidence.

➡ Most studies took place in low- and middle-income countries.

Patient information

- Public disclosure of performance data

Fung 2008

➡ Public disclosure of performance for health plans:

- may lead to patients selecting health plans that have better quality ratings;

- has uncertain effects on quality improvement activities;

- may slightly improve health outcomes.

➡ Public disclosure of performance for hospitals:

- may lead to little or no difference in patient selection of hospitals;

- probably stimulates quality improvement activities;

- may improve health outcomes.

➡ Public disclosure of performance for individual healthcare providers:

- probably leads to patients selecting providers that have better quality ratings;

- has uncertain effects on quality improvement activities;

- may improve health outcomes.

➡ All of the included studies took place in high-income countries.

- Public disclosure of performance may be difficult to implement in low-income countries because of limitations of the ability of health facilities and providers to produce accurate data, the capacity to disseminate the data, the ability of patients to interpret the data and, in some places, the lack of choice available in terms of facilities or providers.

Table 8. Intervention-outcome matrix for included reviews
  1. a✓: a desirable effect; ∅: little or no effect; ?: an uncertain effect; ✕: an undesirable effect; NS: no studies found by this review that reported this outcome; NR: outcome not reported by this review.
    bOther than adverse effects on any of the outcomes in the previous columns.
    cLocal interagency collaborative interventions may lead to little or no difference in physical health and may slightly improve functional level in patients with psychiatric disorders, compared with standard care. It is uncertain whether local interagency collaborative interventions decrease mortality and mental health symptoms.
    dLocal interagency collaborative interventions may lead to little or no difference in quality of life.
    eIt is uncertain whether pharmaceutical policies that restrict reimbursements change health outcomes.
    fRestrictions to pharmaceutical reimbursement probably decrease targeted drug use in the short and long term and reduce expenditures on target drug or drug class.
    gIt is uncertain if prevention, detection and response interventions reduce healthcare fraud and abuse and related expenditures.
    hPatient outcomes (auto-reporting of being sick in the past month, diarrhoea incidence) may be improved and household health expenditures reduced by contracting out.
    iContracting out services to non-state not-for-profit providers may increase access to and utilisation of health services.
    jMedicine registration and multifaceted interventions (including a mix of regulations, training of inspectors, public-private collaborations and legal actions against counterfeiters) may decrease the prevalence of counterfeit and substandard medicines; WHO prequalification of medicines may lead to a decrease in the failure rates of medicines undergoing quality testing.
    kReference pricing (a system in which a reference price is established within a country as the maximum level of reimbursement for a group of medicines) may reduce insurers' cumulative medicine expenditures; may increase the use of reference medicines; and may reduce the use of cost-share medicines.
    lIndex pricing (a maximum refundable price to pharmacies for medicines within an index group of therapeutically interchangeable medicines) may increase the use of generic medicines and reduce the use of brand-name medicines.
    mIndex pricing may slightly reduce the price of generic medicines and may have little or no effect on the price of brand-name medicines.
    nDirect-to-consumer advertising increases people's requests for advertised medicines as well as prescription volumes for advertised medicine. The direction of the effect depends on the medicine. For instance, for essential medicines this may be a desirable effect but for non-essential medicines this may be a harmful effect.
    oMinority academic advising programmes may increase the number of black health sciences students enrolled and slighlty increase retention to graduation.
    pManager training programmes may increase knowledge of planning processes and monitoring and evaluation skills.
    qIt is uncertain whether educational or financial interventions or regulatory or personal and professional support strategies to recruit or retain health professionals increase the number of health professionals practising in in underserved areas.
    rHiring district health managers to work within the Ministry of Health system through private contracts may improve access to health care and service use, but it is uncertain if this improves population health outcomes.
    sReducing immigration restrictions in high-income countries probably increases the migration of nurses from low- and middle-income countries to high-income countries.
    tIt is uncertain whether external inspection adherence to accreditation standards improves quality of care.
    uIt is uncertain whether telephone discussions compared with face-to-face meetings change consumer priorities for community health goals.
    vConsumer interviewers may slightly improve responses regarding patient satisfaction, compared to staff interviewers.
    wConsumer consultation in the development of consent documents may have little or no impact on self-reported participant understanding of the trial described in the consent document, satisfaction with study participation, adherence to the protocol or refusal to participate.
    xPatients probably experience little or no difference in their levels of worry or anxiety associated with procedures when they receive information material that has been developed following consumer consultation.
    yConsumer consultation in developing patient information material probably results in material that is more relevant, readable and understandable to patients, and probably improves the knowledge of patients who read the material.
    zWomen's groups practising participatory learning and action cycles may improve survival in mothers and may slightly reduce stillbirths. .
    aaWomen's groups practising participatory learning and action cycles probably improve survival in newborn babies.
    bbMulti-component community-based dengue control programmes may reduce mosquito larval indices, and such programmes combined with fish and chemical larvicides may reduce mosquito larval indices.
    ccMulti-component community-based dengue control programmes combined with the use of crustaceans that eat mosquito larvae may reduce mosquito larval indices.
    ddIt is uncertain whether multi-component community-based dengue control programmes combined with the use of crustaceans that eat mosquito larvae reduce dengue incidence.
    eePublic disclosure may lead to slight improvements in clinical outcomes for health plans.
    ffPublic disclosure may lead patients to select health plans with better quality ratings or to avoid those with worse ratings.
    ggMay lead to slight improvements in hospital clinical outcomes.
    hhMay lead to little or no difference in patient selection of hospitals.
    iiProbably stimulates hospitals to undertake quality improvement activities.
    jjPublic disclosure of performance data may improve clinical outcomes (risk-adjusted mortality rates for surgeons) among individual providers.
    kkPublic disclosure probably influences users of health care services to select providers with better quality ratings or to avoid those with worse ratings.

  Direction of effects and certainty of the evidencea
Governance arrangement Patient outcomes Access, coverage, utilisation Quality of care Resource use Social outcomes Impacts on equity Health care provider outcomes Adverse effects b Other
Authority and accountability for health policies

Interagency collaboration

Hayes 2012

? ⊕⊖⊖⊖

∅⊕⊕⊖⊖c

NRNRNR∅⊕⊕⊖⊖dNRNRNRNR

Decision-making about what is covered by health insurance – restrictions on drug reimbursement

Green 2010

? ⊕⊖⊖⊖e✔⊕⊕⊕⊖fNR✔⊕⊕⊕⊖fNRNRNRNRNR

Policies to reduce corruption – fraud detection and response actions

Rashidian 2012

NRNRNR?⊕⊖⊖⊖gNRNRNRNRNR
Authority and accountability for organisations

Contracting out – to non-state not-for–profit providers

Lagarde 2009

✔⊕⊕⊖⊖h✔⊕⊕⊖⊖iNRNRNRNRNRNRNR

Multi-institutional arrangements – social franchising

Koehlmoos 2009

NSNSNSNSNSNSNSNSNS
Authority and accountability for commercial products

Registration – drugs

El-Jardali 2015

NRNRNRNRNRNRNRNR✔⊕⊕⊖⊖j

Pricing and purchasing policies – medicines – reference pricing

Acosta 2014

NR✔⊕⊕⊖⊖kNR✔⊕⊕⊖⊖kNRNRNRNRNR

Pricing and purchasing policies – medicines – index pricing

Acosta 2014

NR✔⊕⊕⊖⊖lNR∅⊕⊕⊖⊖mNRNRNRNRNR

Marketing regulationsmedicines direct-to-consumer advertising

Gilbody 2005

NS✔✕⊕⊕⊕⊕nNRNSNRNRNRNRNR
Authority and accountability for health professionals

Training and licensing

pre-licensure education – minority academic advising programme

Pariyo 2009

NRNRNRNRNR✔⊕⊕⊖⊖oNRNRNR

Training and licensing – manager training programme versus no training

Rockers 2013

NRNRNRNRNRNRNRNR✔⊕⊕⊖⊖p

Recruitment and retention strategies – health professionals in underserved areas

Grobler 2015

NSNSNSNSNS?⊕⊖⊖⊖qNSNSNS

Recruitment and retention strategies – private versus public contracts of district health managers

Rockers 2013

?⊕⊖⊖⊖r✔⊕⊕⊖⊖rNRNRNRNRNRNRNR

Movement of health workers between public and private organisations

Rutebemberwa 2014

NSNSNSNSNSNSNSNSNS

Emigration and immigration policies – reducing immigration restrictions

Peñaloza 2011

NS✕⊕⊕⊕⊖sNSNSNRNRNRNRNR

Dual practice

Kiwanuka 2011

NRNSNSNSNSNSNSNRNR

Authority and accountability for quality of practice

authority and accountability for quality of outpatient care – external inspection

Flodgren 2011

NSNS? ⊕⊖⊖⊖tNRNRNRNSNRNR
Stakeholder involvement

Stakeholder participation in policy and organisational decisions – communication forums

Nilsen 2010

NSNSNSNSNSNSNSNS? ⊕⊖⊖⊖u

Stakeholder participation in policy and organisational decisions – consumer involvement in research

Nilsen 2010

NSNS✔⊕⊕⊖⊖vNSNSNSNSNS∅⊕⊕⊖⊖w

Stakeholder participation in policy and organisational decisions – consumer involvement in preparing patient information

Nilsen 2010

∅⊕⊕⊕⊖xNSNSNSNSNSNSNS✔⊕⊕⊕⊖y

Community mobilisation – women's groups practising participatory learning and action

Prost 2013

✔⊕⊕⊖⊖z

✔⊕⊕⊕⊖aa

NSNSNSNSNSNSNSNS

Community mobilisation – community-based dengue control

Heintze 2007

✔⊕⊕⊖⊖bb

✔⊕⊕⊖⊖cc

? ⊕⊖⊖⊖dd

NSNSNSNSNSNSNSNS

Patient information

public disclosure of performance data – health plans

Fung 2008

∅⊕⊕⊖⊖ee✔⊕⊕⊖⊖ffNSNSNSNSNSNSNS

Patient information

public disclosure of performance data – hospitals

Fung 2008

✔⊕⊕⊖⊖gg✔⊕⊕⊖⊖hh✔⊕⊕⊕⊖iiNSNSNSNSNSNS

Patient information

public disclosure of performance data –individual healthcare providers

Fung 2008

✔⊕⊕⊖⊖jj✔⊕⊕⊕⊖kkNSNSNSNSNSNSNS
The certainty of the evidence is an assessment of how good an indication the research provides of the likely effect; i.e. the likelihood that the effect will be substantially different from what the research found. By 'substantially different' we mean a large enough difference that it might affect a decision. These judgements are made using the GRADE system and the following definitions.
Ratings Definitions Implications

⊕⊕⊕⊕

High

This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different is low.This evidence provides a very good basis for making a decision about whether to implement the intervention. Impact evaluation and monitoring of the impact are unlikely to be needed if it is implemented.

⊕⊕⊕⊖

Moderate

This research provides a good indication of the likely effect. The likelihood that the effect will be substantially different is moderate.This evidence provides a good basis for making a decision about whether to implement the intervention. Monitoring of the impact is likely to be needed and impact evaluation may be warranted if it is implemented.

⊕⊕⊖⊖

Low

This research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high.This evidence provides some basis for making a decision about whether to implement the intervention. Impact evaluation is likely to be warranted if it is implemented.

⊕⊖⊖⊖

Very low

This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different is very high.This evidence does not provide a good basis for making a decision about whether to implement the intervention. Impact evaluation is very likely to be warranted if it is implemented.

We took into account other relevant considerations besides the findings of the included reviews when drawing conclusions about implications for practice (EPOC 2017). This includes considerations related to the applicability of the findings and likely impacts on equity. Our conclusions about implications for systematic reviews were based on types of governance arrangements for which we were unable to find a reliable, up-to-date review and on the limitations identified in the included reviews. This includes considerations related to the applicability of the findings and likely impacts on equity. Our conclusions about implications for future evaluations are based on the findings of the included reviews (EPOC 2017).

Results

We identified 7272 systematic reviews of health systems arrangements and implementation strategies. We excluded 6953 reviews from this overview following a review of titles and abstracts. We retrieved the full texts of 66 reviews for further detailed assessment, excluding 43 for the following reasons (Table 5): they had important methodological limitations (10 reviews), were out-of-date (7 reviews), focused on an area already covered by one of the included reviews (20 reviews), did not focus on the effects of interventions (2 reviews), or were of limited relevance to low-income countries (4 reviews) (Figure 1). We considered two other reviews for inclusion but, after discussion, agreed that they were part of the scope of another of the overviews (Jia 2014; Maharaj 2015).We considered Ketelaar 2011 and WHO 2010 to be supplementary in that they contributed information about interventions for which other reviews were the main source of information (because those reviews, Fung 2008 and Grobler 2015, were more reliable, included more studies, or were more up-to-date). Appendix 5 lists the reviews still awaiting classification.

Figure 1.

Review flow diagram.

Description of included reviews

We included 19 systematic reviews published between 2005 and 2015 in this overview (Table 4). Of these, 13 were Cochrane Reviews and 6 non-Cochrane reviews.

The reviews reported results from 172 studies and included the following study designs .

  • 28 randomised trials (16.3%).

  • 6 non-randomised trials (3.5%).

  • 15 controlled before-after studies (8.7%).

  • 62 interrupted time series studies (36.0%).

  • 1 repeated measures study (0.6%).

  • 56 observational study designs (32.6%).

  • 3 studies used more than one design (1.7%).

  • 1 before-after study, reanalysed as an interrupted time series study (0.6%).

The number of studies included in each review ranged from zero (Koehlmoos 2009; Kiwanuka 2011; Rutebemberwa 2014) to 45 (Fung 2008). The dates of the most recent searches in the reviews ranged from October 2004 in Gilbody 2005 to April 2014 in Grobler 2015.

Nine reviews did not include any studies from low- or middle-income countries (Gilbody 2005; Fung 2008; Pariyo 2009; Green 2010; Nilsen 2010; Hayes 2012; Rashidian 2012; Acosta 2014; Grobler 2015), and four reviews only included studies conducted in low- or middle-income countries (Lagarde 2009; Prost 2013; Rockers 2013; El-Jardali 2015). Overall, 74% of the studies from the included reviews took place in high-income countries. Study settings varied and included primary care; home, workplace and community settings; and outpatient and inpatient settings in hospitals and non-primary level health centres (Appendix 3). Health workers who participated in the studies included in the reviews included: physicians, nurses, pharmacists, psychologists, dentists, social workers and traditional healers. Recipients of care participating in studies included in the reviews included children, adults and pregnant mothers (Appendix 3). Outcomes examined by the reviews included: healthcare provider performance, patient outcomes, access to care, coverage, utilisation of health services, social outcomes, impacts on equity and adverse effects (Table 8).

We grouped the governance arrangements addressed in the reviews into five categories.

  • Authority and accountability for health policies: 3 reviews.

  • Authority and accountability for organisations: 2 reviews.

  • Authority and accountability for commercial products: 3 reviews.

  • Authority and accountability for health professionals: 7 reviews.

  • Stakeholder involvement: 4 reviews.

Methodological quality of included reviews

We present the methodological quality (reliability) of the included reviews in Table 6. One of the 19 included reviews, Rashidian 2012, had important methodological limitations, but we retained it in the overview because no better review was available. We judged the other 18 reviews to have only minor limitations.

We found a number of problems with respect to the identification, selection and critical appraisal of the included studies in reviews. Five reviews had some limitations in relation to the comprehensiveness of the search, and three reviews had some limitations in relation to study selection. We found few problems with respect to the analysis of the available evidence. Two reviews had limitations related to either the description of the extent of heterogeneity or the examination of factors that might explain differences in the results of included studies (Rashidian 2012 and Heintze 2007, respectively).

Effect of interventions

Table 7 summarises the key messages from the included reviews, and Table 8 presents the key findings of the different governance interventions considered by each of the included reviews as well as the certainty of this evidence by outcome. Table 9 summarises the effects and certainty of the evidence from the included reviews according to whether the interventions had desirable effects, little or no effect, undesirable effects, or uncertain effects. In the following text, we report the main findings of the included comparisons.

Table 9. Summary of effects of interventions and certainty of evidence
  1. aFor this intervention, the direction of the effect depends on the medicine. For instance, for essential medicines this may be a desirable effect (and is therefore listed as such above) but for non-essential medicines this may be a undesirable effect.

Interventions found to have desirable effects on at least one outcome with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects
Authority and accountability for health policies

Decision-making about what is covered by health insurance

  • Restrictions on drug reimbursement (Green 2010)

    • Outcomes improved: drug utilisation and drug expenditure

Authority and accountability for commercial products

Marketing regulations

  • Direct-to-consumer advertising of prescription-only medicines (Gilbody 2005)

    • Outcomes improved: people's requests for advertised medicines and the number of related prescriptions by doctorsa

Stakeholder participation in policy and organisational decisions

Community mobilisation

  • Women's groups practising participatory learning and action cycles (Prost 2013)

    • Outcomes improved: neonatal mortality

Patient information

  • Public disclosure of hospital performance data (Fung 2008)

    • Outcome improved: hospitals' quality improvement activities

  • Public disclosure of individual healthcare providers performance data (Fung 2008)

    • Outcome improved: users' selection of providers

  • Consumer involvement in preparing patient information (Nilsen 2010)

    • Outcomes improved: quality of the material and patient knowledge

Interventions for which the certainty of the evidence was low or very low (or no studies were found) for all outcomes examined
Authority and accountability for health policies
  • Interagency collaboration (Hayes 2012)

  • Policies to reduce corruption – fraud detection and response actions (Rashidian 2012)

Authority and accountability for organisations
Authority and accountability for commercial products
Authority and accountability for health professionals
  • Pre-licensure education – minority academic advising programme (Pariyo 2009)

  • Location of practice – recruitment and retention strategies for health professionals (Grobler 2015)

  • Movement of health workers between public and private organisations (Rutebemberwa 2014)

  • Training and licensing – manager training programmes (Rockers 2013)

  • Recruitment and retention strategies – private versus public contracts for district health managers (Rockers 2013)

  • Dual practice (Kiwanuka 2011)

  • Authority and accountability for quality of inpatient and outpatient care – external inspection (Flodgren 2011)

Stakeholder participation in policy and organisational decisions
  • Stakeholder participation in policy and organisational decisions – communication forums and consumer involvement in research (Nilsen 2010)

  • Community-based dengue control (Heintze 2007)

    • Outcome improved: mosquito larval indices

  • Public disclosure of performance data – health plans (Fung 2008)

Authority and accountability for health policies

Three reviews considered interventions related to authority and accountability for health policies (Green 2010; Hayes 2012; Rashidian 2012).

Interagency collaboration

Hayes 2012 examined the effects of interagency collaboration between local health and other local government agencies and services, comparing it with standard practice or no intervention. The review included 16 studies, all conducted in high-income countries. The findings suggested that it is uncertain whether local interagency collaborative interventions decrease mortality or mental health symptoms (very low-certainty evidence). The studies also suggest that these interventions may lead to little or no difference in physical health and quality of life but may slightly improve functional levels among people with psychiatric disorders, compared with standard ways of delivering services (low-certainty evidence).

Decision-making about what is covered by health insurance – restrictions on medicines reimbursement

Green 2010 included 29 studies in high-income countries and assessed the effects of placing restrictions on the medicines reimbursed by health insurance systems. The review found that restrictions on reimbursement probably decrease the use of the targeted medicines as well as expenditures on targeted medicines or medicine classes (moderate-certainty evidence). The impacts of such restrictions on health outcomes and health service utilisation were uncertain (very low-certainty evidence). Review authors could not assess the impacts of such restrictions on equity measures, as none of the included studies reported this outcome.

Policies to reduce corruption

Rashidian 2012 studied the effects of interventions to reduce healthcare fraud. It included four studies from high-income countries. The review found that it is uncertain if prevention, detection and response interventions reduce healthcare fraud and related expenditures (very low-certainty evidence).

Authority and accountability for organisations

Two reviews considered interventions related to authority and accountability for organisations (Koehlmoos 2009; Lagarde 2009). The review addressing the effects of social franchising, Koehlmoos 2009, did not identify any eligible studies, so we do not discuss it further below.

Contracting out

Lagarde 2009 examined the effects of contracting out (sometimes called sub-contracting) and included three studies conducted in middle-income countries. The review found that contracting out services to non-state, not-for-profit providers may increase access to and utilisation of health services (low-certainty evidence). In addition, patient outcomes may be improved and household health expenditures reduced (low-certainty evidence). None of the included studies presented evidence on whether contracting out was more effective than making a similar investment in the public sector. We are therefore uncertain of the effects of investing in contracting out compared to an equivalent investment in public sector health services.

Authority and accountability for commercial products

Three reviews considered interventions related to authority and accountability for commercial products (Gilbody 2005; Acosta 2014; El-Jardali 2015).

Registration of medicines

El-Jardali 2015 explored the effect of interventions for combating or preventing medicine counterfeiting (e.g. medicines with the wrong ingredients, without active ingredients, with insufficient active ingredients or with fake packaging). The review included 21 studies conducted in low- and middle-income countries and found that it is uncertain whether the licensing of drug or medicines outlets reduces the prevalence of counterfeit medicines or the failure rates of medicines undergoing quality testing (very low-certainty evidence). The review also found that medicine registration may decrease the prevalence of counterfeit and substandard medicines (low-certainty evidence) and that the prequalification of medicines by WHO (in which manufacturers receive WHO-approved certificates of good manufacturing practices) may lead to a decrease in the failure rates of medicines undergoing quality testing (low-certainty evidence). Finally, multifaceted interventions (that include a mix of regulations, training of inspectors, public-private collaborations and legal actions against counterfeiters) may be effective in decreasing the prevalence of counterfeit and substandard medicines (low-certainty evidence).

Pricing and purchasing policies for pharmaceuticals

Acosta 2014 evaluated the effects of reference pricing (a system that establishes a benchmark or reference price within a country as the maximum level of reimbursement for a group of drugs or medicines), maximum pricing (a fixed, maximum price that a medicine can have within a health system) and index pricing (maximum refundable price to pharmacies for medicines within an index group of therapeutically interchangeable medicines). The 18 included studies took place in high-income countries. Reference pricing may reduce insurers' cumulative medicine expenditures by shifting medicine use from cost-share medicines (more expensive medicines in the same group as the reference medicines, for which patients have to pay the difference between the reference price and the price of the medicine purchased) to reference medicines; and may increase the use of reference medicines and reduce the use of cost-share medicines (low-certainty evidence). Index pricing may increase the use of generic medicines and may reduce the use of brand-name medicines; may slightly reduce the price of generic medicines; and may have little or no effect on the price of brand-name medicines (low-certainty evidence). It is uncertain whether maximum pricing affects medicine expenditures (very low-certainty evidence). The effects of reference pricing, maximum pricing and index pricing on healthcare utilisation or health outcomes is uncertain, as the included studies did not assess these outcomes.

Marketing regulations

Gilbody 2005 explored the effects of direct-to-consumer advertising of prescription-only medicines. The review included four studies performed in high-income countries and found that direct-to-consumer advertising increases people's requests for advertised medicines as well as the number of related prescriptions by doctors (high-certainty evidence). The direction of the effect depends on the medicine. For instance, for essential medicines this may be a desirable effect but for non-essential medicines this may be a undesirable effect. The review did not identify any studies that evaluated the impact of direct-to-consumer advertising on health outcomes or the cost-effectiveness of such advertising.

Authority and accountability for health professionals

Seven reviews considered interventions related to authority and accountability for health professionals (Pariyo 2009; Flodgren 2011; Kiwanuka 2011; Peñaloza 2011; Rockers 2013; Rutebemberwa 2014; Grobler 2015). Kiwanuka 2011 examined the effects of interventions to improve the management of dual practice, in which healthcare providers hold more than one job, but did not identify any eligible studies. Likewise, Rutebemberwa 2014 assessed interventions to manage the movement of health workers between public and private organisations but did not include any studies. Therefore, we do not discuss either of these empty reviews below.

Training and licensing – pre-licensure education

Pariyo 2009 examined the effects of changes in pre-licensure education (the training of health professional students prior to their registration as professionals) on the supply of health workers. The review included two studies that addressed the effects of an academic advising programme for minority groups, in which training institutions in a high-income country provide additional support for minority group students. The review found that such programmes may increase the number of minority group health sciences students enrolled, slightly increase retention to graduation and decrease the difference in retention levels to graduation between a minority group and those in other population groups (low-certainty evidence). The review did not find any studies of the effects on the supply of health workers of other changes in pre-licensure education.

Rockers 2013 examined the effects of interventions to hire, retain and train district health systems managers and included two studies conducted in four middle-income countries. The review found that manager training programmes may increase knowledge of planning processes as well as managers' monitoring and evaluation skills, compared with no training (low-certainty evidence).

Recruitment and retention strategies

Grobler 2015 examined strategies for the recruitment and retention of health workers practising in underserved and rural areas. The review included one study from a high-income country (Taiwan), but it is uncertain whether educational or financial interventions, or regulatory, personal and professional support strategies to recruit or retain health professionals increase the number of health professionals practising in underserved areas, as the review did not identify any studies that evaluated such interventions.

Rockers 2013 examined the effects of interventions to hire, retain and train district health systems managers and included two studies conducted in four middle-income countries. The review found that hiring district health managers to work within the Ministry of Health system through private contracts ('contracting in') may improve access to health care (health facilities open 24 hours and supplies and equipment available) and may increase use of antenatal care and other publicly funded services, compared to hiring managers through public sector contracts (low-certainty evidence). However, it is uncertain whether this approach improves population health outcomes (very low-certainty evidence).

Emigration and immigration policies

Peñaloza 2011 examined the effects of interventions for controlling the emigration of health professionals from low- and middle-income countries. It included one study that evaluated the effect of a change to immigration legislation in the USA on the migration of nurses from the Philippines to the USA. It found that reducing immigration restrictions in high-income countries probably increases the migration of nurses from low- and middle-income to high-income countries (moderate-certainty evidence). The review did not identify any studies that evaluated the effectiveness of interventions implemented in low-income countries to decrease the emigration of health professionals.

Authority and accountability for quality of care

Flodgren 2011 examined the effects on healthcare organisation behaviour, healthcare professional behaviour and patient outcomes of external inspection systems to improve adherence to external quality standards in organisations delivering health care. The review included one study each from a middle- and a high-income country. The review found that it is uncertain whether external inspection of adherence to standards improves adherence and quality of care or decreases health-acquired infection rates in hospitals (very low-certainty evidence). This review did not find any studies of the effectiveness of external inspections of adherence to standards in ambulatory (outpatient) settings.

Stakeholder involvement

Four reviews considered interventions related to stakeholder involvement (Heintze 2007; Fung 2008; Nilsen 2010; Prost 2013).

Stakeholder participation in policy and organisational decisions

Nilsen 2010 examined the effects of interventions to involve consumers in developing healthcare policies and research, clinical practice guidelines and patient information material. The review included six randomised trials, all conducted in high-income countries. One of these studies evaluated consumer involvement in policy development and found that it is uncertain whether telephone discussions change consumer priorities for community health goals compared with face-to-face meetings (very low-certainty evidence). None of the other included studies assessed stakeholder participation in policy and organisational decisions, but rather assessed consumer involvement in developing patient information, delivering satisfaction with care interviews and developing informed consent forms for research.

Community mobilisation

Two reviews examined the effects of community mobilisation – strategies to empower people to organise themselves to address an issue of common concern, and to identify and employ available resources to change a given situation. Prost 2013 included seven cluster-randomised trials from low- and middle-income countries. The review found that women's groups practising participatory learning and action cycles may improve maternal survival and may slightly reduce stillbirths (low-certainty evidence), and these interventions probably improve survival in newborn babies (moderate-certainty evidence). Heintze 2007 included 11 studies of community-based interventions for dengue control: 9 from middle-income countries and 2 from high-income countries. The review found that community-based dengue control programmes that include some form of mobilisation may reduce mosquito larval indices (low-certainty evidence).

Patient information – public disclosure of performance data

Fung 2008 examined the effects of public disclosure of performance data on health plans (including health insurance schemes, health maintenance organisations, private health insurance, etc.) as well as on hospitals and healthcare professionals, and included 45 studies from high-income countries. The review found that public disclosure of performance data on health insurance scheme quality may lead people to select health plans with better quality ratings or to avoid those with worse ratings and may lead to slight improvements in clinical outcomes for health insurance schemes (low-certainty evidence). Public disclosure of performance data on hospital quality may lead to little or no difference in patient selection of hospitals (low-certainty evidence), probably stimulates hospitals to undertake quality improvement activities (moderate-certainty evidence), and may lead to slight improvements in hospital clinical outcomes (low-certainty evidence). Public disclosure of performance for individual healthcare providers probably leads to patients selecting providers that have better quality ratings (moderate-certainty evidence) and may improve clinical outcomes among individual providers (low-certainty evidence).

Discussion

Summary of main results

The evidence from the 19 included systematic reviews of governance arrangements for health systems in low-income countries covers a range of strategies (e.g. at policy, organisational, commercial, health professional and stakeholder levels), involving diverse settings (geographical, health system level) and populations (managers, health professionals, patients). Of the 24 outcomes for which an intervention had a desirable effect, 7 were supported by evidence of moderate certainty and 17 by evidence of low certainty. The one outcome on which an intervention had an undesirable effect was supported by evidence of moderate certainty. For eight outcomes reported in the included reviews, we assessed the effects as uncertain (very low-certainty evidence). We found high or moderate-certainty evidence that interventions in the areas of restrictions on medicine reimbursement, community mobilisation, public disclosure of provider's performance data and patient involvement in decision-making had desirable effects, with no undesirable effects.

Overall completeness and applicability of evidence

We identified reviews for 19 of 48 types of the governance arrangements. However, three of these reviews did not identify any eligible studies (Koehlmoos 2009; Kiwanuka 2011; Rutebemberwa 2014). We found only three reviews of strategies addressing authority and accountability for commercial products (Gilbody 2005; Acosta 2014; El-Jardali 2015). Table 8 summarises the outcomes examined in the individual reviews. Only two reviews in the overview reported on the impacts of governance interventions on equity (Pariyo 2009; Grobler 2015). Three reviews reported outcomes related to resource use (Green 2010; Rashidian 2012; Acosta 2014), with none addressing cost-effectiveness of the interventions. The sparse economic and equity data (in comparison to effectiveness data) limit assessment of the cost-effectiveness and equity impacts of the interventions examined.

We incorporated our judgments about the applicability of summarised evidence (particularly, indirectness in relation to settings, populations and outcomes) into the GRADE assessments of its certainty, and we reported these applicability judgments in each of the SUPPORT Summaries. In general, it is difficult to draw firm conclusions regarding the applicability of the overview findings to low-income countries. For many of the comparisons and outcomes, the evidence comes from studies conducted in high-income countries (mainly the USA, UK, Canada and Australia) with very different on-the-ground realities and health systems arrangements. These differences are particularly important in relation to interventions that require substantial resources for design and implementation or that may require advanced technology or specialised skills for delivery, for instance systems for reimbursement and reference pricing for medicines (Green 2010; Acosta 2014), for fraud detection and response actions (Rashidian 2012), and for public disclosure of performance data (Fung 2008). These differences may also affect the applicability of interventions that are complex and may require substantial changes to the organisation of care – for example, improved collaboration between local health and local government agencies (Hayes 2012). It is therefore uncertain whether similar effects are likely if the interventions assessed in these reviews are implemented in low-income countries.

Certainty of the evidence

Although some of the included reviews had methodological limitations, they were, for the most part, relatively well conducted (Table 6). The certainty of the evidence for the effect estimates for the interventions considered in these reviews ranged from very low to high (Table 8). Of the 39 outcomes considered by at least one study, the certainty of the evidence was high for 1 (3%), moderate for 8 (22%), low for 22 (56%) and very low for 8 (21%) (Table 10).

Potential biases in the overview process

Although our searches were relatively comprehensive, it is possible that we missed some relevant reviews. We also excluded reviews that were published before April 2005. It is possible that some of those reviews provide information that is still useful and that might supplement information provided by the included reviews. Although this cut-off was arbitrary, it is unlikely that we excluded a substantial amount of useful information. However, 6 of the 19 included reviews were published before 2010, and it is possible that more recent evidence has been published since then that would change the review conclusions. None of these considerations would likely bias the results of this overview, but they might limit its comprehensiveness.

Classifying the interventions in the included reviews was sometimes uncertain and required judgment. For example, Jia 2014 assessed strategies for expanding health insurance coverage in vulnerable populations, and we decided to include it in the implementation strategies overview (Pantoja 2014). Another review evaluated the effects of rapid response systems on clinical outcomes (Maharaj 2015), and we included that one in the delivery overview (Ciapponi 2014). On the other hand, Fung 2008 related to the public disclosure of information directed to patients, and we included it in this overview instead of the implementation strategies overview. Although these judgments and differences in approaches to characterising governance interventions are unlikely to have introduced bias into this overview, they might result in some confusion, since there is no universally agreed upon classification system for governance arrangements. Moreover, any system for categorising health system interventions is, to some extent, arbitrary. A unified taxonomy for classifying health system interventions could facilitate explicit and systematic synthesis and interpretation of the existing body of evidence on health systems interventions across studies.

Judgments about the relevance of some interventions to low-income countries (applicability, equity, economic considerations, and monitoring and evaluation) were sometimes difficult to make. While these judgments might have led to systematic errors, it seems unlikely. At least two overview authors made all of these judgments on the basis of the SUPPORT Summaries, which undergo peer review by the contact author of the summarised review and by individuals from low- and middle-income countries.

Our general approach towards including reviews of studies from high-income countries was inclusive rather than exclusive to enable readers to assess for themselves the relevance of the review findings. Similarly, our approach was to assume that findings are applicable to low-income countries unless we identified differences between the study settings and settings in low-income countries or factors that would likely modify the effects in low-income countries.

Agreements and disagreements with other studies or reviews

We identified three related overviews of reviews published in the last 10 years (Lewin 2008; Scott 2009; Brunton 2015). These overviews addressed a range of governance arrangements in diverse settings and populations. As with our overview, most of the studies included in those overviews were from high-income countries, and data on patient outcomes, equity, costs and cost-effectiveness were scarce. We describe the findings of the three overviews briefly below.

Brunton 2015 aimed to understand the components of community engagement and the contribution of active content to health and social outcomes. The overview included three reviews, which found that more extensive community engagement (i.e. where community members design, deliver and evaluate health interventions) was associated with improved behavioural outcomes. More extensive engagement across design, delivery and evaluation was noted in studies where community engagement processes included bidirectional communication, collective decision-making and intervention delivery training support to community members.

Lewin 2008 reviewed the effects of governance, financial and delivery arrangements, and implementation strategies that have the potential to improve the delivery of cost-effective interventions in primary health care in low- and middle-income countries. It included 21 systematic reviews, one of which addressed governance strategies for working with the private for-profit sector – including franchising, regulation and accreditation – to improve the use of quality health services by people in low-income settings (Patouillard 2007). We excluded this particular review in the present overview and did not identify any other eligible reviews that addressed governance strategies for working with the private for-profit sector. Lewin 2008 did not find any systematic reviews that addressed other questions about governance arrangements for primary health care, including decentralisation of decision-making, the regulation of training, or the control of corruption.

Scott 2009 included 23 reviews and assessed public scorecards and performance reports, external accreditation and clinical governance arrangements. Review authors found that studies have not adequately evaluated these interventions. These quality improvement strategies are heterogeneous, and methodological flaws in much of the evaluative literature limit the validity and generalisability of results. The authors assert that, based on current best available evidence, clinician/patient-driven quality improvement strategies appear to be more effective than manager/policymaker driven ones. Some of the included reviews would have been excluded from our overview as they are more than 10 years old; some are covered in the delivery and implementation overviews; and some reviews address interventions that we did not consider to be highly relevant to low-income countries.

Authors' conclusions

Well-conducted, systematic Cochrane Reviews and non-Cochrane reviews have evaluated a wide range of governance arrangements relevant to health systems in low-income countries. The interventions assessed have targeted different levels of the health system and report a range of outcomes. However, in all the main categories of our taxonomy of governance arrangements for health systems there are important evidence gaps where primary studies and/or rigorous reviews are needed.

Implications for practice

We found the following governance arrangements to be effective (moderate or high-certainty evidence of desirable effects on at least one outcome and no moderate or high-certainty evidence of undesirable effects).

  • Restrictions on medicine reimbursement for prescription medicines (Green 2010).

  • Public disclosure of hospitals' and individual healthcare providers' performance data (Fung 2008).

  • Consumer involvement in developing patient information materials (Nilsen 2010).

  • Women's groups practising participatory learning and action, in relation to newborn survival (Prost 2013).

The following governance arrangements have undesirable effects (moderate or high certainty evidence of at least one outcome with an undesirable effect, and no moderate or high certainty evidence of desirable effects).

  • Reducing immigration restrictions in high income countries for health workers from other settings (Peñaloza 2011).

The effects of the following governance arrangements are uncertain (low- or very-low certainty evidence (or no studies were found) for all outcomes examined).

  • Interagency collaborative interventions (Hayes 2012).

  • Prevention, detection, and response interventions to reduce healthcare fraud and abuse and related expenditures (Rashidian 2012).

  • Contracting out service delivery to non-state, not-for-profit providers (Lagarde 2009).

  • Social franchising within health services (Koehlmoos 2009).

  • Regulatory measures and multifaceted interventions to decrease the prevalence of counterfeit and substandard medicines, and WHO prequalification of medicines to reduce medicine quality testing failure rates (El-Jardali 2015).

  • Index pricing and reference pricing for prescription medicines (Acosta 2014).

  • Pre-licensure academic advising programmes for minority groups (Pariyo 2009).

  • Recruitment strategies for health professionals in underserved areas (Grobler 2015).

  • Movement of health workers between public and private organisations (Rutebemberwa 2014).

  • District manager training programmes, in relation to managers' knowledge of planning processes and monitoring and evaluation skills (Rockers 2013).

  • Private contracting ("contracting in") of district health managers compared to direct employment by the Ministry of Health (Rockers 2013).

  • Dual practice among health professionals (Kiwanuka 2011).

  • External inspection for adherence to accreditation standards in hospitals (Flodgren 2011).

  • Different communication forums (face-to-face, telephone discussions, mail surveys, etc.) for consumer involvement in healthcare policy (Nilsen 2010).

  • Community mobilisation for dengue control (Heintze 2007).

  • Public disclosure of data on the performance of health plans (Fung 2008).

Because the effects of these arrangements are uncertain, their health system impacts need to be monitored and evaluated if they are implemented.

Implications for research

Based on the included reviews, we have identified gaps in primary research because of uncertainty about the applicability of the evidence to low-income countries (Table 10) and low-certainty evidence or a lack of studies (Table 11). It is notable that in 9 out of the 19 included reviews, all of the studies took place in high-income countries, and in 15 of the 19 reviews there was at least one comparison where the certainty of the evidence on effects was low, or no studies were included. Further studies evaluating the effects of these interventions are needed, particularly in low-income countries.

Table 10. Priorities for primary research based on the applicability limitations to low-income countries of the governance arrangements identifieda
  1. aPriorities for primary research are based on the applicability limitations to low-income countries of the governance arrangement interventions identified by the included reviews. We did not search for additional primary studies.

Governance arrangement Applicability limitations
Findings Interpretation
Authority and accountability for health policies

Interagency collaboration

Hayes 2012

All studies included in this review took place in high income countries.The reality of local agencies in low-income countries is probably very different to that in high-income countries so results reported in this review should be applied with caution in low-income countries settings.

Decision-making about what is covered by health insurance

restrictions on drug reimbursement

Green 2010

All of the included studies took place in high-income countries. Thus there is uncertainty regarding the transferability of the results to low- and middle-income country settings.

Participants were mainly senior citizens or low-income adult populations in publicly subsidised or administered pharmaceutical benefit plans.

Only two of the studies included in this review reported health outcome data, precluding any conclusions about the impact of the policies on patient outcomes.

Applicability of these interventions to low-income country settings depends on there being:

– a regulatory framework;

– an administrative and managerial system which support the implementation of the policy;

–an insurance system with relatively broad medicines benefit;

– efficient, timely access to patient-specific information;

– availability of preferred products incentivised by the re-imbursement policy;

– product quality assessments and prescriber and patient trust in the quality of preferred products.

Policies to reduce corruption – fraud detection and response actions

Rashidian 2012

There is no study from low income-countries and only two from middle-income countries.

Low-income countries might be more prone and vulnerable to health care fraud and its consequences.

When assessing the transferability of these findings to low-income countries the following factors should be considered.

– The availability of human and technical resources to combat fraud.

– The acceptability and costs of the interventions.

Authority and accountability for organisations

Subcontracting to non-state not-for-profit providers

Lagarde 2009

All of the studies took place in low- and middle-income countries

In the three included studies, the contracts were carried out with non-governmental organisations (NGOs); no studies were found that evaluated contracts with private for-profit providers.

The studies provided very little description of the actual measures implemented by the contractor (management, organisation, salaries, and incentives) to achieve the goals established in the contract.

Differences in health systems; patient and physician attitudes to NGOs; and legal restrictions may limit applicability of the findings.

Subcontracting can be a potentially effective strategy in particular settings but it may be difficult for governments to re-deploy public funds to private providers when available funds are already committed to public services.

Factors that need to be considered to asses whether the intervention effects are likely to be transferable include:

– the availability of not-for-profit organisations to carry out the contracts;

– the capacity within the public sector for set up and monitor the contracts.

Multi-institutional arrangements

–social franchising

Koehlmoos 2009

The review did not find any studies conducted in low- and middle-income countries that met its inclusion criteria.Although social franchising is currently used and advocated in low- and middle-income countries, no rigorous evaluations of its impacts (both positive and negative) are available.
Authority and accountability for commercial products

Registration

drugs

El-Jardali 2015

The studies were all undertaken in low- and middle-income countries.

The results suggest that drug registration, WHO prequalification of drugs, and multi-faceted interventions may be effective in reducing the prevalence of counterfeit drugs.

The findings are applicable to low- and middle- income settings. However, a country's existing pharmaceutical supply chain and infrastructure, availability of routine data on quality of drugs, qualified and skilled personnel, and financial resources may facilitate the transferability of the findings.

While registration may be effective, it should probably encompass both domestic manufacturers and importers and be complemented with routine postmarketing surveillance to sustain the quality of drugs circulating in the market.

Countries that rely heavily on imported drugs may consider opting for drugs that are WHO-prequalified. However, even among WHO-prequalified products, the quality may vary depending on the country of export.

The success of multifaceted interventions requires collaborations with drug regulatory bodies, skilled human resources, and technical capacity for routine drug inspections.

Reference pricing

Acosta 2014

All of the 18 studies included were in high-income countries.

The effectiveness of reference pricing policy in low-income countries may depend on factors such as:

– health systems structure and settings as copayments, reimbursment and cost share;

– access to prices data sources;

– availability of adequate incentives for healthcare providers, patients, physicians, pharmacists and pharmaceutical companies to comply with the reference pricing policy;

– significant price differences between the drugs in the intervention group before reference pricing is introduced;

– clear information for managers, clinicians and patients;

– availability and access to drugs in the reference group;

– a regulatory framework that allows generic substitution or prescribing by international non-proprietary name (INN);

– appropriate exemptions (exemptions that are too limited could lead to higher co-payments for appropriate use of more expensive drugs and incentives to use a less effective drug. Exemptions that are too broad could reduce savings by not shifting drug use towards appropriate use of less expensive drugs.).

Marketing regulations – Drugs direct-to-consumer advertising

Gilbody 2005

The studies, all conducted in high-income countries, show that direct-to-consumer advertising alters prescribing behaviour and volume, but no studies examined the impact of such advertising on health outcomesGiven the absence of any evidence of improvement in health outcomes from direct-to-consumer advertising, its benefits are uncertain in any setting.
Authority and accountability for health professionals

Pre-licensure education

Pariyo 2009

All included studies took place in high-income countries.The challenges faced in health care worker education in high- and low-income countries are qualitatively and quantitatively different (e.g. the availability of funds, laws regarding equity and awareness of these, job prospects including remuneration, and curricula). Appropriate interventions could be expected to have a comparatively higher impact in low-income countries, where alternatives and opportunities are generally more limited than in high-income countries. However, there is no evidence regarding the effects of such interventions.

Recruitment and retention strategies

Grobler 2015

Some observational studies, mostly from high-income countries, suggest that some interventions, such as selecting students from rural areas, exposing students to clinical rotations in rural areas, or financial incentive programmes might increase the number of health professionals in underserved areas. However, the certainty of this evidence is very low.Economic and cultural differences, differences between health system structures, and differences in state and educational institutional capacity to regulate and manage various types of interventions may limit the applicability of findings from high- to low-income countries.

Training/recruitment and retention strategies

Rockers 2013

The two included studies took place in low and middle income countries.

Tested in a low income country, there is uncertainty about the impact of having private contracts (contract-in districts) compared to public contracts of district health managers.

The capacity and strength of the government to oversee and supervise districts with private contracts could be an important issue to consider when it comes to assure the attainment of public regulations and goals.

The level of power decentralisation in the districts might change the impact of policies related with health managers. The higher the degree of decentralisation, the higher the impact they might have.

Movement of health workers between public and private organisations

Rutebemberwa 2014

No studies met the inclusion criteria for the review.Health worker availability remains one of the key barriers to strengthening health systems in low-income countries. Effective interventions to manage the movement of health professionals could help to address this and need to be evaluated rigorously.

Emigration and immigration policies – reducing immigration restrictions

Peñaloza 2011

The available evidence is based on an intervention made in a high-income country.

Policies in high-income countries may have an effect on the number of health workers migrating from low- and middle-income countries.

Low- and middle-income countries have little direct influence on high-income country policies, including immigration policies. However, low- and middle-income countries may attempt to influence such policies by means of diplomacy, lobbying, or public relations before they are enacted.

Dual practice

Kiwanuka 2011

No studies met the inclusion criteria for the review.Dual practice may be more of a problem in low-income countries, due to low wages in the public sector, and interventions to manage it may have different effects, e.g. the risk of health professionals migrating is likely to be greater in low-income countries compared to high-income countries.

Authority and accountability for quality of practice

authority and accountability for quality of outpatient care – external inspection

Flodgren 2011

Neither of the two studies included in this review took place in a low-income country: one was done in South Africa and the other in England.

Both studies assessed the effect of external inspection of compliance of different standards on quality of hospital services.

According to the findings in this review, it is uncertain whether external inspection contributes or not to improve quality of health services in hospital setting.

External inspection of compliance standards may have varying acceptability and impact across different healthcare and cultural settings; may involve different components from training to organisational restructuring; and may impact in different ways on consumer and provider satisfaction across different settings

Although quality of care is an objective of care in all health systems, it is not possible to be confident about the applicability of the reported interventions to low income countries and to settings other than hospital care

Stakeholder involvement

Stakeholder participation in policy and organisational decisions – consumer involvement in preparing patient information

Nilsen 2010

All the studies took place in high-income countries.

Some interventions used technologies such as telephones and email.

Baseline levels of consumers involvement were not reported.

Strategies to overcome barriers such as low baseline levels of social participation and education should be explored when considering consumer involvement in low-income countries. Training and support may be essential.

The attitudes and the perspectives of health professionals and policymakers regarding consumer involvement should also be considered.

As the availability of communication technologies may be a problem, face-to-face involvement may be most appropriate.

Community mobilisation – women's groups practising participatory learning and action

Prost 2013

All 7 studies took place in low-and middle-income countries, including Bangladesh, Malawi, India and Nepal.

The use of women's groups practicing participatory learning and action probably decreases newborn mortality and may reduce maternal mortality in rural areas in low-income countries. However, its effectiveness may depend on participation of a substantial proportion of pregnant women. It might also depend on adequate supervision and support, home visits, access to care, improving the quality of care, and adequate resources.

The intervention might be less effective in urban areas if there is less community cohesion and interaction among women included in women's groups, and higher baseline use of health services.

Community mobilisation – community-based dengue control

Heintze 2007

10 out of 11 studies included in the systematic review took place in low- and middle-income countries.These findings are applicable to low-income countries; however, the availability acceptability and costs of the interventions should be considered.

Patient information

public disclosure of performance data

Fung 2008

The studies, all conducted in high-income countries, provided mixed evidence for using the public disclosure of performance data to improve the quality of care.

There is no evidence to date that the public disclosure of performance data affects the quality of care. Even if public disclosure were effective in improving quality of care in high-income countries, the results would not be directly transferable to low-income country settings because of differences in health infrastructure, the ability of health facilities and providers to produce accurate data, the capacity to disseminate the data, and the ability of consumers to interpret the data.

There is a need for high-quality studies of public disclosure of performance data in high-, middle- and low-income countries.

Table 11. Priorities for primary research based on insufficient evidence for important outcomesa,b
  1. ACU: access, coverage and utilisation outcomes; PO: patient outcomes; QoC: quality of care outcomes; RU: resource use outcomes.
    aWe have included here only priorities for research on the effects of governance arrangements based on the included reviews for each category of the health systems taxonomy. Since we did not search for primary studies we cannot discard primary evidence outside this review-based approach.

Governance arrangement Included review No studies Very low certainty of evidence Low certainty of evidence
Authority and accountability for health policies
Interagency collaboration Hayes 2012PO, ACU, QoC, RUPOPO
Decision-making about what is covered by health insurance – Restrictions on drug reimbursement Green 2010QoCPO
Policies to reduce corruption – fraud detection and response actions Rashidian 2012PO, ACU, QoCRU
Authority and accountability for organisations
Subcontracting to non-state not-for-profit providers Lagarde 2009QoC, RUPO, ACU

Multi-institutional arrangements

Social franchising

Koehlmoos 2009PO, ACU, QoC, RU
Authority and accountability for commercial products
Registration – drugs El-Jardali 2015PO, ACU, QoC, RU
Reference pricing – reference and index price Acosta 2014PO, QoCACU, RU
Marketing regulations – drugs direct to consumer advertising Gilbody 2005PO, QoC, RU
Authority and accountability for health professionals

Training and licensing

Pre-licensure education – minority academic advising programme

Pariyo 2009PO, ACU, QoC, RU
Recruitment and retention strategies Grobler 2015 PO, ACU, QoC, RU
Training and licensing/recruitment and retention strategies Rockers 2013QoC, RUPO, ACU
Movement of health workers between public and private organisations Rutebemberwa 2014PO, ACU, QoC, RU
Emigration and immigration policies – reducing immigration restrictions Peñaloza 2011PO, QoC, RU
Dual practice Kiwanuka 2011PO, ACU, QoC, RU

Authority and accountability for quality of practice

Authority and accountability for quality of outpatient care – external inspection

Flodgren 2011PO, ACU, RUQoC
Stakeholder involvement
Stakeholder participation in policy and organisational decisions - consumer involvement in preparing patient information Nilsen 2010PO, ACU, RUQoC
Community mobilisation – women's groups practising participatory learning and action Prost 2013ACU, QoC, RU
Community mobilisation – community-based dengue control Heintze 2007ACU, QoC, RU

Patient information

Public disclosure of performance data

Fung 2008QoC, RUPO, ACU

The included reviews rarely reported social outcomes, resource use, impacts on equity or adverse (undesirable or unintended) effects (Table 8). Systematic reviews and updates of reviews should include all outcomes that are relevant to decision-makers and those groups affected by governance arrangements. In addition, there is a wide range of interventions for which we did not find a reliable up-to-date systematic review (Table 12), including the effects of governance arrangements affecting what or who is covered by health insurance; policies to manage absenteeism; requirements for monitoring or evaluation; organisational policies for accrediting healthcare providers; regulation of insurance provision; multi-institutional arrangements for coordinating care; regulation of registration, patents, profits and liability for commercial products; regulation of professional competence and liability; and regulation of patients' rights.

Table 12. Priorities for new systematic reviews on governance arrangements in low-income countries
Governance arrangement What we found
Authority and accountability for health policies
Decentralised versus centralised authority for health servicesNo reviews identified
Policies that regulate what drugs are reimbursedNo reviews identified
Policies that regulate what services are reimbursedNo reviews identified
Restrictions on reimbursement for health insuranceNo reviews identified
Strategies for expanding health insurance coverageNo reviews identified
Policies to manage absenteeismNo reviews identified
Requirements for monitoring or evaluationNo reviews identified
Authority and accountability for organisations
OwnershipReview in progress (Herrera 2013)
Stewardship of private health servicesNo reviews identified
AccreditationNo reviews identified
Provision of drug insuranceReview in progress (Pantoja 2015)
Provision of health insuranceNo reviews identified
Policies that regulate interactions between donors and governmentsNo reviews identified
Governance arrangements for coordinating care across multiple providersNo reviews identified
MergersNo reviews identified
Authority and accountability for commercial products
Registration of health technologyNo reviews identified
Patents and profits of drugsNo reviews identified
Patents and profits of health technologyNo reviews identified
Pricing and purchasing policies of health technology and servicesNo reviews identified
Marketing regulations for health technology and servicesNo reviews identified
Sales and dispensing policies for drugsReview in progress (Peñaloza 2015)
Liability for commercial productsNo reviews identified
Authority and accountability for health professionals
Licensure of health professionalsNo reviews identified
Specialty certificationNo reviews identified
Scope of practiceNo reviews identified
Authority and accountability for quality assurance of hospital careNo reviews identified
Professional competenceNo reviews identified
Professional liabilityNo reviews identified
Stakeholder involvement
Community monitoringNo reviews identified
Patient information about drugsNo reviews identified
Patients' rightsNo reviews identified

Acknowledgements

We would like to thank the following editors and peer referees who provided comments to improve the overview: Sasha Shepperd (editor), Kaelan Moat, Rhona Mijumbi-Deve and to Meggan Harris for copy-editing the overview.

We would also like to acknowledge the following colleagues who helped to produce the SUPPORT Summaries upon which this overview is based: Racha Fadlallah, Fadi El-Jardali, Elie Akl, Taryn Young, Peter Steinmann, Primus Che Chi and Yasser Sami Amer.

Additionally, we thank Susan Munabi-Babigumira, Atle Fretheim, Simon Goudie and Hanna Bergman for editing some of the SUPPORT Summaries as well as the review authors and others who provided feedback on them.

Charles S Wiysonge's work is supported by the South African Medical Research Council and the National Research Foundation of South Africa (Grant Numbers: 106035 and 108571).

The Norwegian Satellite of the Effective Practice and Organisation of Care (EPOC) Group receives funding from the Norwegian Agency for Development Co-operation (Norad), via the Norwegian Institute of Public Health to support review authors in the production of their reviews.

This overview is a product of the Effective Health Care Research Consortium, which provided funding to make this overview open access. The Consortium is funded by UK aid from the UK Government for the benefit of developing countries (Grant: 5242). The views expressed in this overview do not necessarily reflect UK government policy.

Appendices

Appendix 1. PubMed and LILACS search strategies

PubMed

From 2000 to present. Update: weekly

#1. MEDLINE[Title/Abstract]

#2. (systematic[Title/Abstract] AND review[Title/Abstract])

#3. meta analysis[Publication Type]

#4. #1 OR #2 OR #3 (Methods filter for systematic reviews – Clinical Queries – Max Specificity)

#5. overview[Title] AND (reviews[Title] OR systematic[Title]

#6. meta-review[Title]

#7. review of reviews[Title]

#8. review[Title] AND systematic reviews[Title]

#9. umbrella[Title] AND (review[Title] OR reviews[Title] OR systematic[Title])

#10. policy[Title] AND (brief[Title] OR evidence[Title])

#11. #5 OR #6 OR #7 OR #8 OR #9 OR #10 (Methods filter for overviews)

#12. #4 OR #11 (Methods filter for systematic reviews and for overviews)

LILACS

From 2000 to present. Update: monthly

(TW:"revision sistematica" OR TW:"revisao sistematica" OR TW:"systematic review" OR MH:"review literature as topic" OR MH:"meta-analysis as topic" OR PT:"meta-analysis")

OR

(PT:revision AND (TW:metaanal$ OR TW:"meta-analysis" OR TW:"metaanalise" OR TW:"meta-analisis" OR TI:overview$ OR TW:"estudio sistematico" OR TW:"systematic study" OR TW:"estudo sistematico" OR TI:review OR TI:revisao OR TI:revision OR TI:systematic OR TI:sistematico))

OR

((TW:overview OR TW:"estudio sistematico" OR TW:"systematic study" OR TW:"estudo sistematico") AND (TI:review OR TI:revisao OR TI:revision OR TI:systematic OR TI:sistematico))

CINAHL (EBSCO)

From 2000 to present. Update: monthly

((TI meta analys* or AB meta analys*) or (TI systematic review or AB systematic review))

PsycINFO (EBSCO)

From 2000 to present. Update: monthly

meta-analysis OR search*

EMBASE (Ovid)

From 2000 to present. Update: monthly

meta-analysis.tw. OR systematic review.tw

Appendix 2. SUPPORT Summaries checklist for making judgments about how much confidence to place in a systematic review

Review:
Assessed by:
Date:
Section A: Methods used to identify, include and critically appraise studies

A.1 Were the criteria used for deciding which studies to include in the review reported?

Did the authors specify:

_ Types of studies

_ Participants

_ Intervention(s)

_ Outcome(s)

Coding guide - check the answers above

YES: All four should be yes

_ Yes

_ Can't tell/partially

_ No

Comments (note important limitations or uncertainty)

A.2 Was the search for evidence reasonably comprehensive?

Were the following done:

_ Language bias avoided (no restriction of inclusion based on language)

_ No restriction of inclusion based on publication status

_ Relevant databases searched (including Medline + Cochrane Library)

_ Reference lists in included articles checked

_ Authors/experts contacted

Coding guide - check the answers above:

YES: All five should be yes

PARTIALLY: Relevant databases and reference lists are both ticked off

_ Yes

_ Can't tell/partially

_ No

Comments (note important limitations or uncertainty)

A.3 Is the review reasonably up-to-date?

Were the searches done recently enough that more recent research is unlikely to be found or to change the results of the review?

Coding guide – consider how many years since the last search (e.g. if more than 10 years the review is unlikely to be up-to-date) and whether there is ongoing research

_ Yes

_ Can't tell/not sure

_ No

Comments (note important limitations or uncertainty)

A.4 Was bias in the selection of articles avoided?

Did the authors specify:

_ Explicit selection criteria

_ Independent screening of full text by at least 2 reviewers

_ List of included studies provided

_ List of excluded studies provided

Coding guide - check the above

YES: All four should be yes

_ Yes

_ Can't tell/partially

_ No

Comments (note important limitations or uncertainty)

A.5 Did the authors use appropriate criteria to assess the risk for bias in analysing the studies that are included? ( See Appendix for an example of criteria - Assessing Risk of Bias Criteria for EPOC Reviews)

_ The criteria used for assessing the risk of bias were reported

_ A table or summary of the assessment of each included study for each criterion was reported

_ Sensible criteria were used that focus on the risk of bias (and not other qualities of the studies, such as precision or applicability)

Coding guide - check the above

YES: All four should be yes

_ Yes

_ Can't tell/partially

_ No

Comments (note important limitations or uncertainty)

A.6 Overall – how would you rate the methods used to identify, include and critically appraise studies?

Summary assessment score A relates to the 5 questions above.

If the "No"or "Partial"option is used for any of the questions above, the review is likely to have important limitations.

Examples of major limitations might include not reporting explicit selection criteria, not providing a list of included studies or not assessing the risk of bias in included studies.

_ Major limitations (limitations that are important enough that the results of the review are not reliable and they should not be used in the policy brief)

_ Important limitations (limitations that are important enough that it would be worthwhile to search for another systematic review and to interpret the results of this review cautiously, if a better review cannot be found)

_ Reliable (only minor limitations)

Comments (note any major limitations or important limitations).
Section B: Methods used to analyse the findings

B.1 Were the characteristics and results of the included studies reliably reported?

Was there:

_ Independent data extraction by at least 2 reviewers

_ A table or summary of the characteristics of the participants, interventions and outcomes for the included studies

_ A table or summary of the results of the included studies.

Coding guide - check the answers above

YES: All three should be yes

_ Yes

_ Partially

_ No

_ Not applicable (e.g. no included studies)

Comments (note important limitations or uncertainty)
B.2 Were the methods used by the review authors to analyse the findings of the included studies reported?

_ Yes

_ Partially

_ No

_ Not applicable (e.g. no studies or no data)

Comments (note important limitations or uncertainty)

B.3 Did the review describe the extent of heterogeneity?

_ Did the review ensure that included studies were similar enough that it made sense to combine them, sensibly divide the included studies into homogeneous groups, or sensibly conclude that it did not make sense to combine or group the included studies?

_ Did the review discuss the extent to which there were important differences in the results of the included studies?

_ If a meta-analysis was done, was the I2, chi square test for heterogeneity or other appropriate statistic reported?

_ Yes

_ Can't tell/partially

_ No

_ Not applicable (e.g. no studies or no data)

Comments (note important limitations or uncertainty)

B.4 Were the findings of the relevant studies combined (or not combined) appropriately relative to the primary question the review addresses and the available data?

How was the data analysis done?

_ Descriptive only

_ Vote counting based on direction of effect

_ Vote counting based on statistical significance

_ Description of range of effect sizes

_ Meta-analysis

_ Meta-regression

_ Other: specify

_ Not applicable (e.g. no studies or no data)

How were the studies weighted in the analysis?

_ Equal weights (this is what is done when vote counting is used)

_ By quality or study design (this is rarely done)

_ Inverse variance (this is what is typically done in a meta-analysis)

_ Number of participants

_ Other, specify:

_ Not clear

_ Not applicable (e.g. no studies or no data)

Did the review address unit of analysis errors?

_ Yes - took clustering into account in the analysis (e.g. used intra-cluster correlation coefficient)

_ No, but acknowledged problem of unit of analysis errors

_ No mention of issue

_ Not applicable - no clustered trials or studies included

Coding guide - check the answers above

If narrative OR vote counting (where quantitative analyses would have been possible) OR inappropriate table, graph or meta-analyses OR unit of analyses errors not addressed (and should have been) the answer is likely NO.

If appropriate table, graph or meta-analysis AND appropriate weights AND the extent of heterogeneity was taken into account, the answer is likely YES.

If no studies/no data: NOT APPLICABLE

If unsure: CAN'T TELL/PARTIALLY

_ Yes

_ Can't tell/partially

_ No

_ Not applicable (e.g. no studies or no data)

Comments (note important limitations or uncertainty)

B.5 Did the review examine the extent to which specific factors might explain differences in the results of the included studies?

_ Were factors that the review authors considered as likely explanatory factors clearly described?

_ Was a sensible method used to explore the extent to which key factors explained heterogeneity?

_ Descriptive/textual

_ Graphical

_ Meta-regression

_ Other

_ Yes

_ Can't tell/partially

_ No

_ Not applicable (e.g. too few studies, no important differences in the results of the included studies, or the included studies were so dissimilar that it would not make sense to explore heterogeneity of the results)

Comments (note important limitations or uncertainty)

B.6 Overall - how would you rate the methods used to analyse the findings relative to the primary question addressed in the review?

Summary assessment score B relates to the 5 questions in this section, regarding the analysis.

If the "No” or "Partial” option is used for any of the 5 preceding questions, the review is likely to have important limitations.

Examples of major limitations might include not reporting critical characteristics of the included studies or not reporting the results of the included studies.

_ Major limitations (limitations that are important enough that the results of the review are not reliable and they should not be used in the policy brief)

_ Important limitations (limitations that are important enough that it would be worthwhile to search for another systematic review and to interpret the results of this review cautiously, if a better review cannot be found)

_ Reliable (only minor limitations)

Use comments to specify if relevant, to flag uncertainty or need for discussion
Section C: Overall assessment of the reliability of the review
C.1 Are there any other aspects of the review not mentioned before which lead you to question the results?

_ Additional methodological concerns

_ Robustness

_ Interpretation

_ Conflicts of interest (of the review authors or for included studies)

_ Other

_ No other quality issues identified

C.2 Based on the above assessments of the methods how would you rate the reliability of the review?

_ Major limitations (exclude); briefly (and politely) state the reasons for excluding the review by completing the following sentence: This review was not included in this policy brief for the following reasons:

Comments (briefly summarise any key messages or useful information that can be drawn from the review for policy makers or managers):

_ Important limitations ; briefly (and politely) state the most important limitations by editing the following sentence, if needed, and specifying what the important limitations are: This review has important limitations.

_ Reliable ; briefly note any comments that should be noted regarding the reliability of this review by editing the following sentence, if needed: This is a good quality systematic review with only minor limitations.

Appendix 3. Characteristics of included reviews

Authority and accountability for health policies
Interagency collaboration
Hayes 2012
Review objective: to evaluate the effects of interagency collaboration between local health and local government agencies on health outcomes in any population or age group.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials , non-randomised trials, controlled before-after studies and interrupted time series studies that assess any interventions of interagency collaboration and partnership and local government agenciesThis review included 16 studies: 7 randomised trials(7 studies), 4 non-randomised trials(4 studies), 4controlled before-after studies(4 studies) and 1 interrupted time series study. 11 studies were included in the meta-analysis. 7 studies reported on interventions to improve the care or treatment of patients and 9 studies about health education, health promotion or disease prevention
ParticipantsAll population types and all age groups were includedStudies were delivered through community and primary care services (8 studies), in schools (5 studies), and in the wider community (3 studies).
SettingsAny local or national settingStudies took place in the UK (7 studies), Denmark (1 study), Sweden (1 study), Norway and Sweden (1 study), the Netherlands (1 study), the USA (2 studies), Canada (1 study), Israel (1 study), and Australia (1 study).
OutcomesMortality, morbidity and behavioural changeA variety of outcomes were reported, including behavioural changes, morbidity and healthcare process
Date of most recent search: December 2011
Limitations: This is a well-conducted systematic review with only minor limitations.
Decision-making about what is covered by health insurance
Green 2010
Review objective: to determine the effects of a pharmaceutical policy restricting the reimbursement of selected medications on medicine use, healthcare utilisation, health outcomes and costs (expenditures).
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised and non-randomised trials, interrupted time series studies including repeated measures studies, and controlled before-after studies assessing prescribing policies – introduction of restriction to reimbursement, relaxation of previously instituted restrictions to reimbursement, or exemption from restrictive policies for targeted cost-effective medicines24 studies evaluating restrictions to reimbursement policies. Most interventions were prior authorization. 5 studies evaluated policies of releasing or relaxing past restrictions to reimbursement. All of the studies were interrupted time series.
ParticipantsHealthcare consumers and providers within a large jurisdiction or system of care (regional, national or international)Participants were predominantly the beneficiaries of publicly subsidised or administered pharmaceutical insurance plans – most often senior citizens aged 65 years or over and low-income adult populations.
SettingsAll settingsHealth insurance systems with substantial coverage of medicines in the USA (14 studies), Canada (11 studies), Norway (2 studies) and Denmark (2 studies)
Outcomes Primary outcomes: medicine use (prescribed, dispensed or actually used), healthcare utilisation, health outcomes, costs (expenditures). Secondary outcomes: changes in equity of access to medicines, changes in access to medically necessary medicines by disadvantaged groups, changes in the distribution of financial burdenMedicine use and medicine expenditures (24 studies), health outcome data (2 studies), healthcare utilisation (9 studies)
Date of most recent search: MEDLINE (2005 to January 2009) and other databases (2005 to October 2008)
Limitations: This is a well-conducted systematic review with only minor limitations; however, the most recent searches were in January 2009.
Policies to reduce corruption
Rashidian 2012
Review objective: to assess the effectiveness of interventions to combat healthcare fraud and abuse.
Types of What the review authors searched for What the review authors found
Study designs and interventionsInterventional studies with or without a concurrent control group assessing any intervention to combat healthcare fraud (including prevention, detection, and response interventions)4 studies were included: 3 assessing detection actions and 2 response actions. The study designs were: longitudinal with concurrent control group (1 study), data mining (2 studies) and before-after study (1 study).
ParticipantsProviders, patients or insured people, insurers (third party payers)Taiwan's National Health Insurance, Medicare and Medicaid (in USA)
SettingsPublic and private health sectorsTaiwan (2 studies) and the USA (2 studies)
OutcomesPrevention, detection, and response related outcomesDetection of fraudulent claims, amount of anti-fraud expenditure, occurrence of healthcare fraud and abuse, fraudulent activities in diagnostic laboratories
Date of most recent search: December 2010
Limitations: This is a well-conducted systematic review with only minor limitations.
Authority and accountability for organisations
Contracting out
Lagarde 2009
Review objective: to assess the effects of contracting out healthcare services in health services utilisation, equity of access, health expenditure and health outcomes.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, controlled before-after studies and interrupted time series studies of contracting out of healthcare services via a formal contractual relationship between government and non-state providers1 controlled before-after study, 1 interrupted time series study, and 1 cluster randomised trial
ParticipantsPopulations that would potentially access health services (users and non-users) as well as health facilities in low- and middle-income countries

- Bolivia: a neighbourhood in the capital city of la Paz

- Pakistan: the population of the rural district of Rahimyar Khan

- Cambodia: 6 districts of the country (2 contracted out and 4 run by the government). It also evaluated a non-reported number of districts contracted in

SettingsNot limited to any level of healthcare delivery2 studies (Pakistan, Cambodia) evaluated a contracting out motivated by weaknesses or absence of public system. Both took place in mostly rural areas. 1 study (Bolivia) included a programme based in an urban setting consisting of a network of 8 health centres and 1 hospital
OutcomesObjective measures of health services utilisation, access to care, healthcare expenditure, health outcomes or changes in equityHealth services utilisation and access to care (3 studies), health expenditure (1 study) and health outcomes (1 study). No studies were found that measured changes in equity of access
Date of most recent search: April 2006
Limitations: This is a well-conducted systematic review with only minor limitations, but the last search for studies was in 2006.
Multi-institutional arrangements
Koehlmoos 2009
Review objective: to assess the effects of the social franchising of health service delivery on access to and the quality of services and health outcomes in low- and middle-income countries.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, non-randomised trials, interrupted time series studies, and controlled before-after studies reporting on social franchises delivering health services, driven by seeking social benefitsNo studies meeting the inclusion criteria were identified.
ParticipantsAll levels of healthcare delivery, all types of patients and healthcare providers
SettingsLow- and middle-income countries
OutcomesHealthcare access, quality of care, health outcomes, adverse effects, equitable access or utilization, cost/service, patient satisfaction
Date of most recent search: October 2007 to March 2008
Limitations: This is a well-conducted systematic review with only minor limitations, but the last search for studies was done in 2008.
Authority and accountability for commercial products
Registration - medicines
El-Jardali 2015
Review objective: to assess the evidence on the effectiveness of interventions implemented to combat or prevent medicine counterfeiting, particularly in low- and middle-income countries.
Types of What the review authors searched for What the review authors found
Study designs and interventions

Randomised trials; non-randomised studies (e.g. cohort studies, retrospective studies, cross-sectional studies, before-after studies); and non-comparative studies

Any intervention at the health system level to combat or prevent medicine counterfeiting. The review excluded studies that focused on internet/online medicine counterfeiting, analytical techniques and medication errors. Studies that also considered substandard medicines were included only when they did not differentiate between substandard and counterfeit medicines, or where it was unclear if the poor quality medicine was counterfeit or substandard

Designs: 21 studies with 25 comparisons: cross-sectional (17 studies); before-after (5 studies); retrospective (1 study); non-comparative (1 study); randomised trial (1 study)

Interventions: medicine registration (5 comparisons); WHO prequalification of medicines (3 studies); licensing of drug or medicine outlets (8 studies); multi-faceted interventions (6 studies); deployment of handheld spectrometers at the point of sale (1 study); a public awareness campaign (1 study); an international model of collaboration (1 study)

Participants"Counterfeit/spurious/falsely-labeled/falsified/medicines", as defined by WHO as medicines with the wrong ingredients, without active ingredients, with insufficient active ingredients or with fake packagingMost of the studies did not distinguish between counterfeit and substandard medicines
SettingsAny settingStudies from low- and middle-income countries
OutcomesChanges in failure rates of tested medicines; changes in the prevalence of counterfeit medicines; changes in quality of medicine; changes in consumer behaviour; seizures of counterfeit medicines; and closures of illegal outlets/warehouses

Changes in failure rates of medicines (19 comparisons); changes in prevalence of counterfeit medicines (4 studies); changes in purchasing behaviour of consumers (1 study); confiscation of counterfeit medicines (2 studies); closure of illegal outlet(2 studies)

Some studies reported more than one outcome.

Date of most recent search: April 2014
Limitations: This was a well-conducted systematic review with only minor limitations. However, the included studies used largely observational designs.
Pricing and purchasing policies
Acosta 2014
Review objective: to determine the effects of pharmaceutical pricing and purchasing policies on medicine use, healthcare utilisation, health outcomes and costs (expenditures).
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, non-randomised trials, controlled repeated measures studies, interrupted time series studies and controlled before-after studies of pharmaceutical pricing and purchasing policies18 studies were included. Some used more than one design: 14 interrupted time series, 1 interrupted time series/controlled before-after/controlled repeated measures, 1 controlled repeated measures/repeated measures and 2 controlled before-after/repeated measures studies. 17 studies evaluated reference pricing, one of which also assessed maximum prices, and 1 study evaluated index pricing.
ParticipantsHealthcare users and providersIn 8 Canadian studies, the patients were Pharmacare beneficiaries in British Columbia: senior citizens aged 65 years and older. The other studies included all beneficiaries of national medicine insurance plans, including vulnerable groups of people from all ages. 1 German and 1 Spanish study did not provide information about the participants.
SettingsLarge jurisdictions or systems of care. Jurisdictions could be regional, national or international. Studies within organisations, such as health maintenance organisations, were included if the organisation was multi-sited and served a large population.Canada (8 studies), USA (2 studies), Spain (2 studies), Germany (2 studies), Norway (2 studies), Australia (1 study) and Sweden (1 study)
OutcomesMedicine use, healthcare utilisation, health outcomes, costs (expenditures), including medicine costs and prices, other healthcare costs and administration costsMedicine use (10 studies), third party (insurance) medicine expenditures (9 studies), medicine prices (4 studies), medicine expenditures savings (5 studies), and patient costs
Date of most recent search: December 2012
Limitations: This is well-conducted systematic review with only minor limitations.
Marketing regulations
Gilbody 2005
Review objective: to examine the benefits and harms of direct-to-consumer advertising of prescription-only medicines
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, controlled clinical trials, controlled before-after studies, interrupted time series studies, and cross-sectional studies with a control group3 interrupted time series studies and 1 comparative cross-sectional survey were found
ParticipantsNot pre-specifiedPatients and physicians in primary care
SettingsNot pre-specifiedUSA (2 studies), USA and Canada (1 study), Netherlands (1 study)
OutcomesHealth-seeking behaviours of patients at the point of access to care; requests for prescription-only medicines; patient-doctor communication and satisfaction with care; prescribing patterns; costsRequests for prescription only medicines (4 studies); prescription volume (4 studies); patient-doctor communication and satisfaction with care (1 study)
Date of most recent search: October 2004
Limitations: This is a well-conducted systematic review with only minor limitations
Authority and accountability for health professionals
Training and licensing
Pariyo 2009
Review objective: to assess the effect of changes in the pre-licensure education of health professionals on health-worker supply.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, controlled before-after studies and interrupted time series studies of interventions that could increase the capacity of health professional training institutions; reduce the loss of students (and increase the likelihood that students will graduate); or increase the recruitment of students from other countries into health professional training institutions2 controlled before-after studies of minority academic advising programmes consisting of academic, personal, financial and vocational advice, skills building, mentorships, supplementary training and annual evaluations
ParticipantsHealth professional students prior to licensure2 studies among minority groups and general health professional students
SettingsNo restrictions2 studies from the USA
OutcomesIncreased numbers of health workers ultimately available for recruitment into the health workforce, improved population-to-health professional ratios2 studies of the numbers of health workers ultimately available for recruitment into the health workforce
Date of most recent search: February 2008
Limitations: This is a well-conducted systematic review with only minor limitations
Training and licensing
Rockers 2013
Review objective: to assess the effectiveness of interventions to hire, retain and train district health systems managers in low- and middle-income countries.
Types of What the review authors searched for What the review authors found
Study designs and interventions

Randomised trials, quasi-randomised trials, controlled before-after studies, interrupted time series studies

Interventions related to hiring, retaining and training managers

One randomised trial: district managers were hired through private contracts to work within the Ministry of Health system.

One controlled before-after study: 18-month manager training programme.

ParticipantsDistrict health systems managers in low- and middle-income countriesDistrict health systems managers
SettingsDistricts in low- and middle-income countriesCambodia (1 study); Mexico, Colombia, El Salvador (1 study)
OutcomesHealth systems: population health outcomes; access; utilization; quality; efficiency; equity. Operational: job-posting vacancy rates, skillsHealth facility staffing and supervision, maternal and child health service use (e.g. immunisation, antenatal care), and population health outcomes (e.g. diarrhea incidence). Managers' competencies
Date of most recent search: December 2011
Limitations: This is a well-conducted systematic review with only minor limitations.
Rectruitment and retention strategies
Grobler 2015
Review objective: to assess the effectiveness of interventions to increase the proportion of healthcare professionals working in rural and other underserved areas
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, non-randomised trials, controlled before-after studies and interrupted time series studies of any intervention to increase the recruitment or retention of health professionals in underserved areas1 interrupted time series study from Taiwan of the effects of National Health Insurance on the equality of distribution of healthcare professionals
ParticipantsQualified healthcare professionals of any cadre or specialtyPhysicians, doctors of Chinese medicine and dentists
SettingsAll settingsTaiwan
OutcomesRecruitment of health professionals: the proportion of health professionals who initially choose to work in rural or urban underserved communities as a result of being exposed to the intervention. Retention: the proportion of healthcare professionals who continue to work in rural or urban underserved communities as a consequence of the interventionEquality of geographic distribution of healthcare professionals measured using the Gini coefficient
Date of most recent search: April 2014
Limitations: This is a well-conducted systematic review with only minor limitations.
Rectruitment and retention strategies
Rockers 2013
See characteristics above under 'Training and licensing'
Movement of health workers between public and private organisations
Rutebemberwa 2014
Review objective: to assess the effects of financial incentives and movement restriction interventions to manage the movement of health workers between public and private organisations in low- and middle-income countries.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials and non-randomised trials; controlled before-after studies; controlled interrupted time series and interrupted time series studies without controlsNo studies were found eligible for inclusion in the review. 9 surveys, 1 review of government reports, 1 study of speeches in the national assembly, and 1 policy analysis paper were found.
ParticipantsAll health professionalsNo studies were found eligible for inclusion in the review.
SettingsAny public or private sector organisationsNo studies were found eligible for inclusion in the review.
Outcomes

Change in the numbers or proportion of health workers entering or leaving the public or private sectors;

duration of stay in a particular sector

No studies were found eligible for inclusion in the review.
Date of most recent search: November 2012
Limitations: This is a well-conducted systematic review with only minor limitations.
Emigration and immigration policies
Peñaloza 2011
Review objective: to assess the effects of policy interventions to control the emigration of health professionals from low- and middle-income countries to high-income countries
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, non-randomised trials, controlled before-after studies, or interrupted studies of any interventions in source or recipient countries (or both) as well as international agreements that could have an impact on the outcomes1 interrupted time series study of the effects of a modification to USA immigration laws (The American Act of October, 1965, which decreased barriers to emigration from countries outside the Americas to the USA)
ParticipantsHealth professional nationals of a low- and middle-income country whose graduate training was in a low- and middle-income countryNurses
SettingsNot restrictedUSA and the Philippines
OutcomesProportion (or other measure of change in number) of health professionals that emigrate from a low- and middle-income country to a high-income countrieAnnual number of nurses migrating from the Philippines to the USA
Date of most recent search: March 2011
Limitations: This is a well-conducted systematic review with only minor limitations.
Dual practice
Kiwanuka 2011
Review objective: to assess the effects of interventions implemented to manage dual practice.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, non-randomised trials, controlled before-after studies, interrupted time series studiesNo studies were found eligible for inclusion in the review
ParticipantsAll health professionalsNo studies were found eligible for inclusion in the review
SettingsNot specifiedNo studies were found eligible for inclusion in the review
OutcomesIncreased working hours by health workers in public facilities, reduced patient waiting times, reduced absenteeism, reduction in number of private sector licenses issued, reduction in private earning, reduced job satisfactionNo studies were found eligible for inclusion in the review
Date of most recent search: May 2011
Limitations: This is a well-conducted systematic review with minor limitations, but no studies were found that met the inclusion criteria.
Quality of practice
Flodgren 2011
Review objective: to evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, non-randomised trials, interrupted time series studies and controlled before-after studies evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes1 cluster-randomised trial conducted in South Africa and 1 before-after study reanalysed as an interrupted time series study, conducted in England. The study in South Africa assessed the effects of external inspection on compliance with hospital accreditation standards. The study conducted in England assessed the effects of the Healthcare Commissions Infection Inspection programme on compliance with standards related to healthcare-acquired infections.
ParticipantsHospitals, primary healthcare organisations and other community-based healthcare organisations containing health professionals20 public hospitals in Kwa Zulu province of South Africa, and all acute hospital trusts in England
SettingsAny health system1 study was conducted in South Africa and 1 in England
OutcomesMeasures of healthcare organisational change (e.g. organisational performance, waiting list times, inpatient hospital stay time); measures of healthcare professional behaviour (e.g. referral rate, prescribing rate); measure of patient outcomes (e.g. mortality and condition-specific measures)Outcomes assessed in 1 study were related to adherence to standards in: medical records, patient outcomes such as satisfaction and patient education, and outcomes related with health processes. The other study assessed the rate of hospital-acquired infections.
Date of most recent search: May 2011
Limitations: This is a well-conducted systematic review with only minor limitations.
Stakeholder involvement
Stakeholder participation in policy and organizational decisions
Nilsen 2010
Review objective: to assess the effects of consumer involvement and to compare different methods of involvement in developing healthcare policy and research, clinical practice guidelines, and patient information material.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials of ways to involve consumers and enable them to inform and participate in decisions about healthcare policy and research, clinical practice guidelines or patient information material6 randomised trials of involvement compared with no involvement in developing patient information, satisfaction interviews conducted by patients compared with staff, informed consent forms developed by consumers versus investigators, and methods of consulting consumers regarding priorities for improving community health
ParticipantsHealthcare consumers or professionals involved in decisions about healthcare at the population level, or evaluating the effects of consumer involvementInvolvement in research (3 studies), developing patient information (2 studies) and healthcare policy (1 study)
SettingsNo specific settingsCanada (2 studies), USA (2 studies), Norway (1 study) and the UK (1 study)
OutcomesParticipation or response rates of consumers; consumer views elicited; consumer influence on decisions, healthcare outcomes or resource utilisation; consumer or professional satisfaction with the involvement process or resulting products; impact on participating consumers; costsLevels of patient satisfaction with different health services, self-reported participant understanding, satisfaction with study participation, adherence to the protocol and refusal to participate; knowledge and anxiety with a specific medical procedure; impact on prioritising health concerns and determinants
Date of most recent search: October 2009
Limitations: This is a well-conducted systematic review with only minor limitations
Community mobilisation
Prost 2013
Review objective: to assess the impact of women's groups practising participatory learning and action cycles on birth outcomes in low- and middle-income countries.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials of participatory women's groups in low- and middle-income countries7 cluster-randomised trials of participatory women's groups in low- and middle-income countries
ParticipantsWomen's groups in which most of the participants are of reproductive age (15–49 years)7 studies that included a total of 111 women's groups and 119,428 births
SettingsLow- and middle-income countriesRural areas in Bangladesh (2 studies), India (2 studies), Malawi (2 studies), and Nepal (1 study)
OutcomesMaternal mortality, neonatal mortality and stillbirthsMaternal mortality (7 studies), neonatal mortality (7 studies), and stillbirths (7 studies)
Date of most recent search: October 2012
Limitations: This is a well-conducted systematic review with only minor limitations.
Community mobilisation
Heintze 2007
Review objective: to assess the effectiveness of community-based interventions in reducing vector populations for dengue control.
Types of What the review authors searched for What the review authors found
Study designs and interventionsRandomised trials, non-randomised trials, controlled before-after studies and interrupted time series studies of community-based interventions aimed at reducing vector populations for dengue control.

11 included studies: 2 randomised trials, 6 controlled before-after studies and 3 interrupted time series studies assessing community-based dengue control interventions alone (5 studies); combined with chemical larvicides (2 studies); combined with fish and chemical larvicides (2 studies); and combined with larvae-eating crustaceans (Mesocyclops copepods) (2 studies)

Studies used educational materials (7 studies); educational meetings such as workshops (9 studies); and educational outreach visits (8 studies). Studies described the involvement of local opinion leaders (6 studies) and national institutions (5 studies), or the use of mass media (5 studies).

ParticipantsCommunity people and professionals serving the community.Household inhabitants (mostly housewives), the elderly, children, health committees, healthcare personnel, government officers, teachers and community organisations
SettingsCommunity5 studies took place in the Americas: Honduras (3 studies), Mexico (1 study), and Cuba (1 study). 6 studies were carried out in Asia: Vietnam (2 studies), Thailand (1 study), Taiwan (1 study), French Polynesia (1 study), Fiji Islands (1 study).
OutcomesIncidence of dengue disease or infestation of the community with Aedes mosquitoesClassical entomological/larval indices such as the House Index (HI), the Container Index (CI) and the Breteau Index (BI) – all measures of larvae infestation in the home or in water containers; seroconversion or incidence of dengue disease
Date of most recent search: March 2005
Limitations: This is a well-conducted systematic review with only minor limitations.
Patient information
Fung 2008
Review objective: to synthesise the evidence for using public disclosure of performance data to improve healthcare quality.
Types of What the review authors searched for What the review authors found
Study designs and interventionsPeer-reviewed articles published between 1986 and 2006. Type of studies not pre-specified2 randomised trials, 2 non-randomised trials, 1 controlled before-after study, 9 interrupted time series studies, and 31 other observational studies
ParticipantsNot pre-specifiedHospitals, patients, and hospital staff (45 studies)
SettingsNot pre-specifiedUSA (43 studies), United Kingdom (1 study), Canada (1 study)
OutcomesSelection of health plans, hospitals, and individual providers, quality improvement activity, clinical outcomes, unintended consequencesSelection of health plans (8 studies), selection of hospitals (9 studies), selection of individual providers (7 studies), quality improvement activity (11 studies), clinical outcomes (11 studies), unintended consequences (13 studies)
Date of most recent search: March 2006
Limitations: Only peer-reviewed, English-language articles were included.

Appendix 4. Supplementary and additional related reviews

Recruitment and retention strategies

Increasing access to health workers in remote and rural areas through improved retention (WHO 2010)

Public disclosure of performance data

Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations (Ketelaar 2011) (Supplementary review)

Appendix 5. Reviews awaiting classification

Likely included reviews

Bowman LR, Donegan S, McCall PJ. Is dengue vector control deficient in effectiveness or evidence?: Systematic review and meta-analysis. PLOS Neglected Tropical Diseases 2016;10(3):e0004551.

Abdel-Aleem H, El-Gibaly OMH, El-Gazzar AFS, Al-Attar GST. Mobile clinics for women's and children's health. Cochrane Database of Systematic Reviews. 2016;8:CD009677.

Akl EA, El-Jardali F, Bou Karroum L, El-Eid J, Brax H, Akik C, et al. Effectiveness of Mechanisms and Models of Coordination between Organizations, Agencies and Bodies Providing or Financing Health Services in Humanitarian Crises: A Systematic Review. PloS one. 2015;10(9):e0137159.

Algie CM, Mahar RK, Wasiak J, Batty L, Gruen RL, Mahar PD. Interventions for reducing wrong-site surgery and invasive clinical procedures. The Cochrane database of systematic reviews. 2015;3(3):CD009404.

Ambia J, Mandala J. A systematic review of interventions to improve prevention of mother-to-child HIV transmission service delivery and promote retention. Journal of the International AIDS Society. 2016;19(1):20309.

Barnard S, Kim C, Park MH, Ngo TD. Doctors or mid-level providers for abortion. The Cochrane database of systematic reviews. 2015;7(7):CD011242.

Basu S, Andrews J, Kishore S, Panjabi R, Stuckler D. Comparative performance of private and public healthcare systems in low- and middle-income countries: a systematic review. PLoS medicine. 2012;9(6):e1001244.

Blacklock C, Gonçalves Bradley DC, Mickan S, Willcox M, Roberts N, Bergström A, et al. Impact of Contextual Factors on the Effect of Interventions to Improve Health Worker Performance in Sub-Saharan Africa: Review of Randomised Clinical Trials. PloS one. 2016;11(1):e0145206.

Byrne A, Hodge A, Jimenez-Soto E, Morgan A. What works? Strategies to increase reproductive, maternal and child health in difficult to access mountainous locations: a systematic literature review. PloS one. 2014;9(2):e87683.

Coast E, Jones E, Lattof SR, Portela A. Effectiveness of interventions to provide culturally appropriate maternity care in increasing uptake of skilled maternity care: a systematic review. Health policy and planning. 2016;31(10):1479-91.

Cornish F, Priego-Hernandez J, Campbell C, Mburu G, McLean S. The impact of Community Mobilisation on HIV Prevention in Middle and Low Income Countries: A Systematic Review and Critique. AIDS and behavior. 2014;18(11):2110-34.

Dawson A, Tran NT, Westley E, Mangiaterra V, Festin M. Improving access to emergency contraception pills through strengthening service delivery and demand generation: a systematic review of current evidence in low and middle-income countries. PloS one. 2014;9(10):e109315.

de Jongh TE, Gurol-Urganci I, Allen E, Zhu NJ, Atun R. Integration of antenatal care services with health programmes in low- and middle-income countries: systematic review. Journal of global health. 2016;6(1):010403.

Dyer TA, Brocklehurst P, Glenny AM, Davies L, Tickle M, Issac A, et al. Dental auxiliaries for dental care traditionally provided by dentists. The Cochrane database of systematic reviews. 2014;8(8):CD010076.

Ehiri JE, Gunn JK, Center KE, Li Y, Rouhani M, Ezeanolue EE. Training and deployment of lay refugee/internally displaced persons to provide basic health services in camps: a systematic review. Global health action. 2014;7:23902.

Emdin CA, Chong NJ, Millson PE. Non-physician clinician provided HIV treatment results in equivalent outcomes as physician-provided care: a meta-analysis. Journal of the International AIDS Society. 2013;16(no pagination):18445.

Fernandez Turienzo C, Sandall J, Peacock JL. Models of antenatal care to reduce and prevent preterm birth: a systematic review and meta-analysis. BMJ open. 2016;6(1):e009044.

Feyissa GT, Lockwood C, Munn Z. The effectiveness of home-based HIV counseling and testing in reducing stigma and risky sexual behavior among adults and adolescents: a systematic review and meta-analysis. JBI Database of Systematic Reviews and Implementation Reports. 2015;13(6):318-72.

Fiander M, McGowan J, Grad R, Pluye P, Hannes K, Labrecque M, et al. Interventions to increase the use of electronic health information by healthcare practitioners to improve clinical practice and patient outcomes. The Cochrane database of systematic reviews. 2015;3(3):CD004749.

Flodgren G, Rachas A, Farmer AJ, Inzitari M, Shepperd S. Interactive telemedicine: effects on professional practice and health care outcomes. The Cochrane database of systematic reviews. 2015;9(9):CD002098.

Gaitonde R, Oxman AD, Okebukola PO, Rada G. Interventions to reduce corruption in the health sector. Cochrane Database of Systematic Reviews. 2016;8:CD008856.

George AS, Branchini C, Portela A. Do Interventions that Promote Awareness of Rights Increase Use of Maternity Care Services? A Systematic Review. PloS one. 2015;10(10):e0138116.

Ghada Abou El S, Therese D, Hatem AM. Planned home versus hospital care for preterm prelabour rupture of the membranes (PPROM) prior to 37 weeks' gestation. Cochrane Database of Systematic Reviews. 2014;4(4):CD008053.

Giedion U, Alfonso EA, Diaz Y. The Impact of Universal Coverage Schemes in the Developing World: A Review of the Existing Evidence. Universal Health Coverage (UNICO) studies series; no. 25. Washington D.C.: The Worldbank. 2013.

Handford CD, Tynan AM, Agha A, Rzeznikiewiz D, Glazier RH. Organization of care for persons with HIV-infection: a systematic review. AIDS care. 2016:1-10.

Health Quality Ontario. Interventions to Improve Access to Primary Care for People Who Are Homeless: A Systematic Review. Ontario health technology assessment series. 2016;16(9):1-50.

Hensen B, Taoka S, Lewis JJ, Weiss HA, Hargreaves J. Systematic review of strategies to increase men's HIV-testing in sub-Saharan Africa. AIDS (London, England). 2014;28(14):2133-45.

Hernández AV, Pasupuleti V, Benites-Zapata V, Velásquez-Hurtado E, Loyola-Romaní J, Rodríguez-Calviño Y, et al. [Systematic review of the efectiveness of community-based interventions to decrease neonatal mortality]. Revista peruana de medicina experimental y salud pública. 2015;32(3):532-45.

Hesselink G, Berben S, Beune T, Schoonhoven L. Improving the governance of patient safety in emergency care: a systematic review of interventions. BMJ open. 2016;6(1):e009837.

Hopkins U, Itty AS, Nazario H, Pinon M, Slyer J, Singleton J. The effectiveness of delegation interventions by the registered nurse to the unlicensed assistive personnel and their impact on quality of care, patient satisfaction, and RN staff satisfaction: a systematic review. JBI Library of Systematic Reviews. 2012;10(15):895-934.

Hoyler M, Hagander L, Gillies R, Riviello R, Chu K, Bergström S, et al. Surgical care by non-surgeons in low-income and middle-income countries: a systematic review. Lancet (London, England). 2015;385 Suppl 2:S42.

Joshi R, Alim M, Kengne AP, Jan S, Maulik PK, Peiris D, et al. Task shifting for non-communicable disease management in low and middle income countries--a systematic review. PloS one. 2014;9(8):e103754.

Kien C, Reichenpfader U, Nußbaumer B, Rohleder S, Punz P, Christof C, et al. [Comparative effectiveness and safety of screening and counselling interventions conducted by non-physicians and physicians: A systematic review]. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen. 2015;109(1):18-27.

Kilpatrick K, Kaasalainen S, Donald F, Reid K, Carter N, Bryant-Lukosius D, et al. The effectiveness and cost-effectiveness of clinical nurse specialists in outpatient roles: a systematic review. Journal of evaluation in clinical practice. 2014;20(6):1106-23.

Kim K, Choi JS, Choi E, Nieman CL, Joo JH, Lin FR, et al. Effects of Community-Based Health Worker Interventions to Improve Chronic Disease Management and Care Among Vulnerable Populations: A Systematic Review. American journal of public health. 2016;106(4):e1-e26.

Kredo T, Adeniyi FGB, Bateganya M, Pienaar ED. Task shifting from doctors to non-doctors for initiation and maintenance of antiretroviral therapy. Cochrane Database of Systematic Reviews. 2014;7(7):CD007331.

Lassi ZS, Musavi NB, Maliqi B, Mansoor N, de Francisco A, Toure K, et al. Systematic review on human resources for health interventions to improve maternal health outcomes: evidence from low- and middle-income countries. Human resources for health. 2016;14(1):10.

Lavender T, Richens Y, Milan SJ, Smyth RMD, Dowswell T. Telephone support for women during pregnancy and the first six weeks postpartum. Cochrane Database of Systematic Reviews. 2013;7(7):CD009338.

Lawrence D, Fedorowicz Z, van Zuuren EJ. Day care versus in-patient surgery for age-related cataract. The Cochrane database of systematic reviews. 2015;11(11):CD004242.

Liu G, Jack H, Piette A, Mangezi W, Machando D, Rwafa C, et al. Mental health training for health workers in Africa: a systematic review. The lancet Psychiatry. 2016;3(1):65-76.

MacPherson P, Munthali C, Ferguson J, Armstrong A, Kranzer K, Ferrand RA, et al. Service delivery interventions to improve adolescents' linkage, retention and adherence to antiretroviral therapy and HIV care. Tropical medicine & international health : TM & IH. 2015;20(8):1015-32.

Mbuagbaw L, Medley N, Darzi AJ, Richardson M, Habiba Garga K, Ongolo-Zogo P. Health system and community level interventions for improving antenatal care coverage and health outcomes. The Cochrane database of systematic reviews. 2015;12(12):CD010994.

Mdege ND, Chindove S. Bringing antiretroviral therapy (ART) closer to the end-user through mobile clinics and home-based ART: systematic review shows more evidence on the effectiveness and cost effectiveness is needed. The International journal of health planning and management. 2013;29(1):e31-47.

Moraros J, Lemstra M, Nwankwo C. Lean interventions in healthcare: do they actually work? A systematic literature review. International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua. 2016;28(2):150-65.

Nunan M, Duke T. Effectiveness of pharmacy interventions in improving availability of essential medicines at the primary healthcare level. Tropical medicine & international health : TM & IH. 2011;16(5):647-58.

Oluoch T, Santas X, Kwaro D, Were M, Biondich P, Bailey C, et al. The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: a systematic review. International journal of medical informatics. 2012;81(10):e83-92.

Palmer KS, Agoritsas T, Martin D, Scott T, Mulla SM, Miller AP, et al. Activity-based funding of hospitals and its impact on mortality, readmission, discharge destination, severity of illness, and volume of care: a systematic review and meta-analysis. PloS one. 2014;9(10):e109975.

Pega F, Liu SY, Walter S, Lhachimi SK. Unconditional cash transfers for assistance in humanitarian disasters: effect on use of health services and health outcomes in low- and middle-income countries. The Cochrane database of systematic reviews. 2015;9(9):CD011247.

Penazzato M, Davies MA, Apollo T, Negussie E, Ford N. Task shifting for the delivery of pediatric antiretroviral treatment: a systematic review. Journal of acquired immune deficiency syndromes (1999). 2014;65(4):414-22.

Pollaris G, Sabbe M. Reverse triage: more than just another method. European journal of emergency medicine : official journal of the European Society for Emergency Medicine. 2015;23(4):240-7.

Polus S, Lewin S, Glenton C, Lerberg PM, Rehfuess E, Gülmezoglu AM. Optimizing the delivery of contraceptives in low- and middle-income countries through task shifting: a systematic review of effectiveness and safety. Reproductive health. 2015;12(1):27.

Rashidian A, Omidvari AH, Vali Y, Sturm H, Oxman AD. Pharmaceutical policies: effects of financial incentives for prescribers. The Cochrane database of systematic reviews. 2015;8(8):CD006731.

Reichow B, Servili C, Yasamy MT, Barbui C, Saxena S. Non-specialist psychosocial interventions for children and adolescents with intellectual disability or lower-functioning autism spectrum disorders: a systematic review. PLoS medicine. 2013;10(12):e1001572.

Reisman J, Arlington L, Jensen L, Louis H, Suarez-Rebling D, Nelson BD. Newborn Resuscitation Training in Resource-Limited Settings: A Systematic Literature Review. Pediatrics. 2016;138(2):1-16.

Robyn PJ, Sauerborn R, Bärnighausen T. Provider payment in community-based health insurance schemes in developing countries: a systematic review. Health policy and planning. 2013;28(2):111-22.

Salam RA, Das JK, Lassi ZS, Bhutta ZA. Impact of community-based interventions for the prevention and control of malaria on intervention coverage and health outcomes for the prevention and control of malaria. Infectious diseases of poverty. 2014;3(1):25.

Schmidt E, Goldhaber-Fiebert SN, Ho LA, McDonald KM. Simulation exercises as a patient safety strategy: a systematic review. Annals of internal medicine. 2013;158(5 Pt 2):426-32.

Sharon RL, Amanda N, Andrew FS, Phil A. Physician anaesthetists versus non-physician providers of anaesthesia for surgical patients. Cochrane Database of Systematic Reviews. 2014;7(7):CD010357.

Sondaal SF, Browne JL, Amoakoh-Coleman M, Borgstein A, Miltenburg AS, Verwijs M, et al. Assessing the Effect of mHealth Interventions in Improving Maternal and Neonatal Care in Low- and Middle-Income Countries: A Systematic Review. PloS one. 2016;11(5):e0154664.

Spaan E, Mathijssen J, Tromp N, McBain F, ten Have A, Baltussen R. The impact of health insurance in Africa and Asia: a systematic review. Bulletin of the World Health Organization. 2012;90(9):685-92.

Sunguya BF, Poudel KC, Mlunde LB, Urassa DP, Yasuoka J, Jimba M. Nutrition training improves health workers' nutrition knowledge and competence to manage child undernutrition: a systematic review. Frontiers in public health. 2013;1:37.

Susan FM, Benjamin MH, Ramila B, Tim E, Debra B. Demand-side financing measures to increase maternal health service utilisation and improve health outcomes: a systematic review of evidence from low- and middle-income countries. JBI Library of Systematic Reviews. 2012;10(58):4165-567.

Thomas SM, Jeyaraman M, Hodge WG, Hutnik C, Costella J, Malvankar-Mehta MS. The effectiveness of teleglaucoma versus in-patient examination for glaucoma screening: a systematic review and meta-analysis. PloS one. 2014;9(12):e113779.

Tibingana-Ahimbisibwe B, Katabira C, Mpalampa L, Harrison RA. The effectiveness of adolescent-specific prenatal interventions on improving attendance and reducing harm during and after birth: a systematic review. International journal of adolescent medicine and health. 2016.

Till SR, Everetts D, Haas DM. Incentives for increasing prenatal care use by women in order to improve maternal and neonatal outcomes. Cochrane Database of Systematic Reviews. 2015;12(12):CD009916.

Watterson JL, Walsh J, Madeka I. Using mHealth to Improve Usage of Antenatal Care, Postnatal Care, and Immunization: A Systematic Review of the Literature. BioMed research international. 2015;2015(no pagination):153402.

Weeks G, George J, Maclure K, Stewart D. Non-medical prescribing versus medical prescribing for acute and chronic disease management in primary and secondary care. The Cochrane database of systematic reviews. 2016;11:CD011227.

Wiysonge CS, Abdullahi LH, Ndze VN, Hussey GD. Public stewardship of private for-profit healthcare providers in low- and middle-income countries. Cochrane Database of Systematic Reviews. 2016;8(8):CD009855.

Wong WC, Luk CW, Kidd MR. Is there a role for primary care clinicians in providing shared care in HIV treatment? A systematic literature review. Sexually transmitted infections. 2012;88(2):125-31.

Likely excluded reviews

Bassili A, Fitzpatrick C, Qadeer E, Fatima R, Floyd K, Jaramillo E. A systematic review of the effectiveness of hospital- and ambulatory-based management of multidrug-resistant tuberculosis. The American journal of tropical medicine and hygiene. 2013;89(2):271-80.

Bhageerathy R, Nair S, Bhaskaran U. A systematic review of community-based health insurance programs in South Asia. The International journal of health planning and management. 2016.

Blaya JA, Fraser HS, Holt B. E-health technologies show promise in developing countries. Health affairs (Project Hope). 2010;29(2):244-51.

Callese TE, Richards CT, Shaw P, Schuetz SJ, Paladino L, Issa N, et al. Trauma system development in low- and middle-income countries: a review. The Journal of surgical research. 2015;193(1):300-7.

Davy C, Bleasel J, Liu H, Tchan M, Ponniah S, Brown A. Effectiveness of chronic care models: opportunities for improving healthcare practice and health outcomes: a systematic review. BMC health services research. 2015;15(1):194.

Dawson AZ, Walker RJ, Campbell JA, Egede LE. Effective Strategies for Global Health Training Programs A Systematic Review of Training Outcomes in Low and Middle Income Countries. Global journal of health science. 2016;8(11):56719.

Higgs ES, Goldberg AB, Labrique AB, Cook SH, Schmid C, Cole CF, et al. Understanding the role of mHealth and other media interventions for behavior change to enhance child survival and development in low- and middle-income countries: an evidence review. Journal of health communication. 2014;19 Suppl 1:164-89.

Hubert GJ, Müller-Barna P, Audebert HJ. Recent advances in TeleStroke: a systematic review on applications in prehospital management and Stroke Unit treatment or TeleStroke networking in developing countries. International journal of stroke : official journal of the International Stroke Society. 2014;9(8):968-73.

Margaret Elizabeth K, Denis P, Peter CR, Wim Van L. The contribution of primary care to health and health systems in low- and middle-income countries: A critical review of major primary care initiatives. 2010.

Pannick S, Davis R, Ashrafian H, Byrne BE, Beveridge I, Athanasiou T, et al. Effects of Interdisciplinary Team Care Interventions on General Medical Wards: A Systematic Review. JAMA internal medicine. 2015;175(8):1288-98.

Schiavo R, May Leung M, Brown M. Communicating risk and promoting disease mitigation measures in epidemics and emerging disease settings. Pathogens and global health. 2014;108(2):76-94.

Zulfiqar AB, Zohra SL, Nadia M. Systematic review on human resources for health interventions to improve maternal health outcomes: Evidence from developing countries. 2010.

Uncertain reviews

Acheampong F, Anto BP, Koffuor GA. Medication safety strategies in hospitals--a systematic review. The International journal of risk & safety in medicine. 2014;26(3):117-31.

Alkhenizan A, Shaw C. Impact of accreditation on the quality of healthcare services: a systematic review of the literature. Annals of Saudi medicine. 2011;31(4):407-16.

Al-Mallah MH, Farah I, Al-Madani W, Bdeir B, Al Habib S, Bigelow ML, et al. The Impact of Nurse-Led Clinics on the Mortality and Morbidity of Patients with Cardiovascular Diseases: A Systematic Review and Meta-analysis. The Journal of cardiovascular nursing. 2015;31(1):89-95.

Bakitas MA, Elk R, Astin M, Ceronsky L, Clifford KN, Dionne-Odom JN, et al. Systematic Review of Palliative Care in the Rural Setting. Cancer control : journal of the Moffitt Cancer Center. 2015;22(4):450-64.

Balfour J, Abdulcadir J, Say L, Hindin MJ. Interventions for healthcare providers to improve treatment and prevention of female genital mutilation: a systematic review. BMC health services research. 2016;16(1):409.

Bannan DF, Tully MP. Bundle interventions used to reduce prescribing and administration errors in hospitalized children: a systematic review. Journal of clinical pharmacy and therapeutics. 2016;41(3):246-55.

Bashshur RL, Howell JD, Krupinski EA, Harms KM, Bashshur N, Doarn CR. The Empirical Foundations of Telemedicine Interventions in Primary Care. Telemedicine journal and e-health : the official journal of the American Telemedicine Association. 2016;22(5):342-75.

Boccia D, Hargreaves J, Lönnroth K, Jaramillo E, Weiss J, Uplekar M, et al. Cash transfer and microfinance interventions for tuberculosis control: review of the impact evidence and policy implications. The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease. 2011;15 Suppl 2:S37-49.

Booth A, Cantrell A, Preston L, Chambers D, Goyder E. What is the evidence for the effectiveness, appropriateness and feasibility of group clinics for patients with chronic conditions? A systematic review2015 2015/12/None.

Brocklehurst P, Mertz B, Jerković-Ćosić K, Littlewood A, Tickle M. Direct access to midlevel dental providers: an evidence synthesis. Journal of public health dentistry. 2014;74(4):326-35.

Candy B, France R, Low J, Sampson L. Does involving volunteers in the provision of palliative care make a difference to patient and family wellbeing? A systematic review of quantitative and qualitative evidence. International journal of nursing studies. 2014;52(3):756-68.

Chapman SM, Wray J, Oulton K, Peters MJ. Systematic review of paediatric track and trigger systems for hospitalised children. Resuscitation. 2016;109:87-109.

Coxeter P, Del Mar CB, McGregor L, Beller EM, Hoffmann TC. Interventions to facilitate shared decision making to address antibiotic use for acute respiratory infections in primary care. The Cochrane database of systematic reviews. 2015;11(11):CD010907.

Damiani G, Pinnarelli L, Sommella L, Vena V, Magrini P, Ricciardi W. The Short Stay Unit as a new option for hospitals: a review of the scientific literature. Medical science monitor : international medical journal of experimental and clinical research. 2011;17(6):SR15-9.

Gentry S, van Velthoven MHMMT, Tudor Car L, Car J. Telephone delivered interventions for reducing morbidity and mortality in people with HIV infection. Cochrane Database of Systematic Reviews. 2013;5(5):CD009189.

Harding R, Albertyn R, Sherr L, Gwyther L. Pediatric palliative care in sub-saharan Africa: a systematic review of the evidence for care models, interventions, and outcomes. Journal of pain and symptom management. 2014;47(3):642-51.

Hastings SE, Armitage GD, Mallinson S, Jackson K, Suter E. Exploring the relationship between governance mechanisms in healthcare and health workforce outcomes: a systematic review. BMC health services research. 2014;14(1):479.

Hines S, Munday J, Kynoch K. Effectiveness of nurse-led preoperative assessment services for elective surgery: a systematic review update. JBI database of systematic reviews and implementation reports. 2015;13(6):279-317.

Hotchkiss DR, Diana ML, Foreit KG. How can routine health information systems improve health systems functioning in low- and middle-income countries? Assessing the evidence base. Advances in health care management. 2012;12:25-58.

Housden L, Wong ST, Dawes M. Effectiveness of group medical visits for improving diabetes care: a systematic review and meta-analysis. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2013;185(13):E635-44.

Ireland S, Kent B. Telephone pre-operative assessment for adults: a comprehensive systematic review. JBI Library of Systematic Reviews. 2012;10(25):1452-503.

Kågesten A, Parekh J, Tunçalp O, Turke S, Blum RW. Comprehensive adolescent health programs that include sexual and reproductive health services: a systematic review. American journal of public health. 2014;104(12):e1-e14.

Lazarus JV, Safreed-Harmon K, Nicholson J, Jaffar S. Health service delivery models for the provision of antiretroviral therapy in sub-Saharan Africa: a systematic review. Tropical medicine & international health : TM & IH. 2014;19(10):1198-215.

Leidy Johanna Rueda D, Diná Lopes Monteiro da C. The efficacy of telephone use to assist and improve the wellbeing of family caregivers of persons with chronic diseases: a systematic review. JBI Library of Systematic Reviews. 2015;12(12):106-40.

McCormack L, Sheridan S, Lewis M, Boudewyns V, Melvin CL, Kistler C, et al. Communication and dissemination strategies to facilitate the use of health-related evidence. Evidence report/technology assessment. 2013(213):1-520.

Meid AD, Lampert A, Burnett A, Seidling HM, Haefeli WE. The impact of pharmaceutical care interventions for medication underuse in older people: a systematic review and meta-analysis. British journal of clinical pharmacology. 2015;80(4):768-76.

Mengistu TA, Tafere TE. Effect of antenatal care on institutional delivery in developing countries: a systematic review. JBI Library of Systematic Reviews. 2011;9(35):1447-70.

Mitchell GK, Burridge L, Zhang J, Donald M, Scott IA, Dart J, et al. Systematic review of integrated models of health care delivered at the primary?secondary interface: how effective is it and what determines effectiveness? Australian journal of primary health. 2015;21(4):391-408.

Palmas W, March D, Darakjy S, Findley SE, Teresi J, Carrasquillo O, et al. Community Health Worker Interventions to Improve Glycemic Control in People with Diabetes: A Systematic Review and Meta-Analysis. Journal of General Internal Medicine. 2015;30:1004-12.

Pérez-Escamilla R, Martinez JL, Segura-Pérez S. Impact of the Baby-friendly Hospital Initiative on breastfeeding and child health outcomes: a systematic review. Maternal & child nutrition. 2016;12(3):402-17.

Rinke ML, Bundy DG, Velasquez CA, Rao S, Zerhouni Y, Lobner K, et al. Interventions to Reduce Pediatric Medication Errors: A Systematic Review. Pediatrics. 2014;134(2):338-60.

Rudge MV, Lima SA, El Dib RP, Marini G, Magalhães C, Calderon Ide M. Effect of ambulatory versus hospital treatment for gestational diabetes or hyperglycemia on infant mortality rates: a systematic review. São Paulo medical journal = Revista paulista de medicina. 2013;131(5):331-7.

Sabater-Hernández D, Sabater-Galindo M, Fernandez-Llimos F, Rotta I, Hossain LN, Durks D, et al. A Systematic Review of Evidence-Based Community Pharmacy Services Aimed at the Prevention of Cardiovascular Disease. Journal of managed care & specialty pharmacy. 2016;22(6):699-713.

Salmoiraghi A, Hussain S. A Systematic Review of the Use of Telepsychiatry in Acute Settings. Journal of psychiatric practice. 2015;21(5):389-93.

Santos MT, Moura SC, Gomes LM, Lima AH, Moreira RS, Silva CD, et al. Telehealth application on the rehabilitation of children and adolescents. Revista paulista de pediatria : orgão oficial da Sociedade de Pediatria de São Paulo. 2014;32(1):136-43.

Saxon RL, Gray MA, Oprescu FI. Extended roles for allied health professionals: an updated systematic review of the evidence. Journal of multidisciplinary healthcare. 2014;7((Saxon R.L., robyn.saxon@health.qld.gov.au; Gray M.A.; Ioprescu F.) School of Health and Sports Sciences, University of the Sunshine Coast, Sippy Downs, Australia):479-88.

Stokes J, Panagioti M, Alam R, Checkland K, Cheraghi-Sohi S, Bower P. Effectiveness of Case Management for 'At Risk' Patients in Primary Care: A Systematic Review and Meta-Analysis. PloS one. 2015;10(7):e0132340.

Suksomboon N, Poolsup N, Nge YL. Impact of phone call intervention on glycemic control in diabetes patients: a systematic review and meta-analysis of randomized, controlled trials. PloS one. 2014;9(2):e89207.

Tao D, Xie L, Wang T, Wang T. A meta-analysis of the use of electronic reminders for patient adherence to medication in chronic disease care. Journal of Telemedicine and Telecare. 2015;21(1).

Tricco AC, Antony J, Ivers NM, Ashoor HM, Khan PA, Blondal E, et al. Effectiveness of quality improvement strategies for coordination of care to reduce use of health care services: a systematic review and meta-analysis. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2014;186(15):E568-78.

Pitt V, Lowe D, Hill S, Prictor M, Hetrick SE, Ryan R, et al. Consumer-providers of care for adult clients of statutory mental health services. Cochrane Database of Systematic Reviews. 2013;3(3):CD004807.

Weaver MS, Lönnroth K, Howard SC, Roter DL, Lam CG. Interventions to improve adherence to treatment for paediatric tuberculosis in low- and middle-income countries: a systematic review and meta-analysis. Bulletin of the World Health Organization. 2015;93(10):700-11B.

Wekesah FM, Mbada CE, Muula AS, Kabiru CW, Muthuri SK, Izugbara CO. Effective non-drug interventions for improving outcomes and quality of maternal health care in sub-Saharan Africa: a systematic review. Systematic reviews. 2016;5(1):137.

Willey B, Smith Paintain L, Mangham-Jefferies L, Car J, Armstrong Schellenberg J. Effectiveness of interventions to strengthen national health service delivery on coverage, access, quality and equity in the use of health services in low and lower middle income countries. 2013 2013.

World Health Organization, University of California SF. Task shifting - physicians (doctors) versus non-physicians (nurses or clinical officers) for initiation and maintenance of antiretroviral therapy. World Health Organization. 2013.

Yasmin F, Banu B, Zakir SM, Sauerborn R, Ali L, Souares A. Positive influence of short message service and voice call interventions on adherence and health outcomes in case of chronic disease care: a systematic review. BMC medical informatics and decision making. 2016;16:46.

Zhai YK, Zhu WJ, Hou HL, Sun DX, Zhao J. Efficacy of telemedicine for thrombolytic therapy in acute ischemic stroke: a meta-analysis. Journal of telemedicine and telecare. 2015;21(3):123-30.

Zhou K, Fitzpatrick T, Walsh N, Kim JY, Chou R, Lackey M, et al. Interventions to optimise the care continuum for chronic viral hepatitis: a systematic review and meta-analyses. The Lancet Infectious diseases. 2016.

Zwanikken PA, Dieleman M, Samaranayake D, Akwataghibe N, Scherpbier A. A systematic review of outcome and impact of master's in health and health care. BMC medical education. 2013;13:18.

Likely supplemental reviews

Abdulwahid MA, Booth A, Kuczawski M, Mason SM. The impact of senior doctor assessment at triage on emergency department performance measures: systematic review and meta-analysis of comparative studies. Emergency medicine journal : EMJ. 2015;33(7):504-13.

Adebayo EF, Uthman OA, Wiysonge CS, Stern EA, Lamont KT, Ataguba JE. A systematic review of factors that affect uptake of community-based health insurance in low-income and middle-income countries. BMC health services research. 2015;15(1):543.

Alghamdi M, Gashgari H, Househ M. A Systematic Review of Mobile Health Technology Use in Developing Countries. Studies in health technology and informatics. 2015;213:223-6.

Alkhaled L, Kahale L, Nass H, Brax H, Fadlallah R, Badr K, et al. Legislative, educational, policy and other interventions targeting physicians' interaction with pharmaceutical companies: a systematic review. BMJ open. 2014;4(7):e004880.

Altowaijri A, Phillips CJ, Fitzsimmons D. A systematic review of the clinical and economic effectiveness of clinical pharmacist intervention in secondary prevention of cardiovascular disease. Journal of managed care pharmacy : JMCP. 2013;19(5):408-16.

Amouzou A, Morris S, Moulton LH, Mukanga D. Assessing the impact of integrated community case management (iCCM) programs on child mortality: Review of early results and lessons learned in sub-Saharan Africa. Journal of global health. 2014;4(2):020411.

Aziz H, Hatah E, Makmor Bakry M, Islahudin F. How payment scheme affects patients' adherence to medications? A systematic review. Patient preference and adherence. 2016;10:837-50.

Bailey C, Blake C, Schriver M, Cubaka VK, Thomas T, Martin Hilber A. A systematic review of supportive supervision as a strategy to improve primary healthcare services in Sub-Saharan Africa. International journal of gynaecology and obstetrics: the official organ of the International Federation of Gynaecology and Obstetrics. 2015;132(1):117-25.

Baxter PE, Hewko SJ, Pfaff KA, Cleghorn L, Cunningham BJ, Elston D, et al. Leaders' experiences and perceptions implementing activity-based funding and pay-for-performance hospital funding models: A systematic review. Health policy (Amsterdam, Netherlands). 2015;119(8):1096-110.

Bbosa GS, Wong G, Kyegombe DB, Ogwal-Okeng J. Effects of intervention measures on irrational antibiotics/antibacterial drug use in developing countries: A systematic review. Health. 2014;6.

Bellows B, Bulaya C, Inambwae S, Lissner CL, Ali M, Bajracharya A. Family Planning Vouchers in Low and Middle Income Countries: A Systematic Review. Studies in family planning. 2016;47(4):357-70.

Bellows BW, Conlon CM, Higgs ES, Townsend JW, Nahed MG, Cavanaugh K, et al. A taxonomy and results from a comprehensive review of 28 maternal health voucher programmes. Journal of health, population, and nutrition. 2013;31(4 Suppl 2):106-28.

Benishek LA, Dugosh KL, Kirby KC, Matejkowski J, Clements NT, Seymour BL, et al. Prize-based contingency management for the treatment of substance abusers: a meta-analysis. Addiction (Abingdon, England). 2014;109(9):1426-36.

Beratarrechea A, Lee AG, Willner JM, Jahangir E, Ciapponi A, Rubinstein A. The impact of mobile health interventions on chronic disease outcomes in developing countries: a systematic review. Telemedicine journal and e-health : the official journal of the American Telemedicine Association. 2014;20(1):75-82.

Blank L, Baxter S, Woods HB, Goyder E, Lee A, Payne N, et al. What is the evidence on interventions to manage referral from primary to specialist non-emergency care? A systematic review and logic model synthesis. Health services and delivery research. 2015.

Bloomfield GS, Vedanthan R, Vasudevan L, Kithei A, Were M, Velazquez EJ. Mobile health for non-communicable diseases in Sub-Saharan Africa: a systematic review of the literature and strategic framework for research. Globalization and health. 2014;10(1):49.

Boksmati N, Butler-Henderson K, Anderson K, Sahama T. The Effectiveness of SMS Reminders on Appointment Attendance: a Meta-Analysis. Journal of medical systems. 2016;40(4):90.

Borchard A, Schwappach DL, Barbir A, Bezzola P. A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Annals of surgery. 2012;256(6):925-33.

Braet A, Weltens C, Sermeus W. Effectiveness of discharge interventions from hospital to home on hospital readmissions: a systematic review. JBI database of systematic reviews and implementation reports. 2016;14(2):106-73.

Brata C, Gudka S, Schneider CR, Clifford RM. A review of the provision of appropriate advice by pharmacy staff for self-medication in developing countries. Research in social & administrative pharmacy: RSAP. 2014;11(2):136-53.

Byrne A, Morgan A. How the integration of traditional birth attendants with formal health systems can increase skilled birth attendance. International journal of gynaecology and obstetrics: the official organ of the International Federation of Gynaecology and Obstetrics. 2011;115(2):127-34.

Campanella P, Vukovic V, Parente P, Sulejmani A, Ricciardi W, Specchia ML. The impact of Public Reporting on clinical outcomes: a systematic review and meta-analysis. BMC health services research. 2016;16(1):296.

Carter EB, Temming LA, Akin J, Fowler S, Macones GA, Colditz GA, et al. Group Prenatal Care Compared With Traditional Prenatal Care: A Systematic Review and Meta-analysis. Obstetrics and gynecology. 2016;128(3):551-61.

Chhina HK, Bhole VM, Goldsmith C, Hall W, Kaczorowski J, Lacaille D. Effectiveness of academic detailing to optimize medication prescribing behaviour of family physicians. Journal of pharmacy & pharmaceutical sciences : a publication of the Canadian Society for Pharmaceutical Sciences, Société canadienne des sciences pharmaceutiques. 2013;16(4):511-29.

Chin WY, Lam CL, Lo SV. Quality of care of nurse-led and allied health personnel-led primary care clinics. Hong Kong medical journal = Xianggang yi xue za zhi / Hong Kong Academy of Medicine. 2011;17(3):217-30.

Chishinga N, Godfrey-Faussett P, Fielding K, Ayles H. Effect of home-based interventions on virologic outcomes in adults receiving antiretroviral therapy in Africa: a meta-analysis. BMC public health. 2014;14(1):239.

Clark CE, Smith LF, Taylor RS, Campbell JL. Nurse led interventions to improve control of blood pressure in people with hypertension: systematic review and meta-analysis. BMJ (Clinical research ed). 2010;341(7771):c3995.

Cobos Muñoz D, Merino Amador P, Monzon Llamas L, Martinez Hernandez D, Santos Sancho JM. Decentralization of health systems in low and middle income countries: a systematic review. International journal of public health. 2016.

Conn VS, Ruppar TM, Enriquez M, Cooper PS, Chan KC. Healthcare provider targeted interventions to improve medication adherence: systematic review and meta-analysis. International journal of clinical practice. 2015;69(8):889-99.

Davis R, Parand A, Pinto A, Buetow S. Systematic review of the effectiveness of strategies to encourage patients to remind healthcare professionals about their hand hygiene. The Journal of hospital infection. 2014;89(3):141-62.

Davy C, Bleasel J, Liu H, Tchan M, Ponniah S, Brown A. Factors influencing the implementation of chronic care models: A systematic literature review. BMC family practice. 2015;16:102.

Decroo T, Rasschaert F, Telfer B, Remartinez D, Laga M, Ford N. Community-based antiretroviral therapy programs can overcome barriers to retention of patients and decongest health services in sub-Saharan Africa: a systematic review. International health. 2013;5(3):169-79.

Dempsey E, Pammi M, Ryan AC, Barrington KJ. Standardised formal resuscitation training programmes for reducing mortality and morbidity in newborn infants. The Cochrane database of systematic reviews. 2015;9(9):CD009106.

Devi BR, Syed-Abdul S, Kumar A, Iqbal U, Nguyen PA, Li YC, et al. mHealth: An updated systematic review with a focus on HIV/AIDS and tuberculosis long term management using mobile phones. Computer methods and programs in biomedicine. 2015;122(2):257-65.

do Amaral JJF, Victora CG. The effect of training in Integrated Management of Childhood Illness (IMCI) on the performance and healthcare quality of pediatric healthcare workers: a systematic review. Revista Brasileira de Saúde Materno Infantil. 2008;8(2):151-62.

Druetz T, Siekmans K, Goossens S, Ridde V, Haddad S. The community case management of pneumonia in Africa: a review of the evidence. Health policy and planning. 2013;30(2):253-66.

Dzakpasu S, Powell-Jackson T, Campbell OM. Impact of user fees on maternal health service utilization and related health outcomes: a systematic review. Health policy and planning. 2014;29(2):137-50.

Eichler R, Agarwal K, Askew I, Iriarte E, Morgan L, Watson J. Performance-based incentives to improve health status of mothers and newborns: what does the evidence show? Journal of health, population, and nutrition. 2013;31(4 Suppl 2):36-47.

Elder E, Johnston AN, Crilly J. Review article: Systematic review of three key strategies designed to improve patient flow through the emergency department. Emergency medicine Australasia : EMA. 2015;27(5):394-404.

Evans BA, Porter A, Gammon B, Mayes RH, Poulden M, Rees N, et al. A systematic review of rapid access models of care and their effects on delays in emergency departments. Emergency medicine journal : EMJ. 2015;32(6):e15-6.

Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS medicine. 2013;10(1):e1001362.

Gielen SC, Dekker J, Francke AL, Mistiaen P, Kroezen M. The effects of nurse prescribing: A systematic review. International journal of nursing studies. 2013;51(7):1048-61.

Gillespie BM, Chaboyer W, Thalib L, John M, Fairweather N, Slater K. Effect of Using a Safety Checklist on Patient Complications after Surgery: A Systematic Review and Meta-analysis. Anesthesiology. 2014;120(6):1380-9.

Gilmore B, McAuliffe E. Effectiveness of community health workers delivering preventive interventions for maternal and child health in low- and middle-income countries: a systematic review. BMC public health. 2013;13(1):847.

Gogia S, Sachdev HP. Home-based neonatal care by community health workers for preventing mortality in neonates in low- and middle-income countries: a systematic review. Journal of perinatology : official journal of the California Perinatal Association. 2016;36 Suppl 1(S1):S55-73.

Gogia S, Sachdev HS. Home visits by community health workers to prevent neonatal deaths in developing countries: a systematic review. Bulletin of the World Health Organization. 2010;88(9):658-66B.

Govindasamy D, Meghij J, Kebede Negussi E, Clare Baggaley R, Ford N, Kranzer K. Interventions to improve or facilitate linkage to or retention in pre-ART (HIV) care and initiation of ART in low- and middle-income settings - a systematic review. Journal of the International AIDS Society. 2014;17(1):19032.

Hamine S, Gerth-Guyette E, Faulx D, Green BB, Ginsburg AS. Impact of mHealth Chronic Disease Management on Treatment Adherence and Patient Outcomes: A Systematic Review. Journal of medical Internet research. 2015;17(2):e52.

Hecht L, Buhse S, Meyer G. Effectiveness of training in evidence-based medicine skills for healthcare professionals: a systematic review. BMC medical education. 2016;16(1):103.

Hurt K, Walker RJ, Campbell JA, Egede LE. mHealth Interventions in Low and Middle-Income Countries: A Systematic Review. Global journal of health science. 2016;8(9):54429.

Jia L, Meng Q, Yuan B, Fang L. Effects of drug cost sharing policy on the drug use, financial risks and moral hazard for the health insurance beneficiaries. Value in Health. 2014;17(7):A795.

Kamarudin G, Penm J, Chaar B, Moles R. Educational interventions to improve prescribing competency: a systematic review. BMJ open. 2013;3(8):e003291.

Kanters S, Park JJ, Chan K, Socias ME, Ford N, Forrest JI, et al. Interventions to improve adherence to antiretroviral therapy: a systematic review and network meta-analysis. The lancet HIV. 2016.

Ke KM, Blazeby JM, Strong S, Carroll FE, Ness AR, Hollingworth W. Are multidisciplinary teams in secondary care cost-effective? A systematic review of the literature. Cost effectiveness and resource allocation : C/E. 2013;11(1):7.

Khanal S, Burgon J, Leonard S, Griffiths M, Eddowes LA. Recommendations for the Improved Effectiveness and Reporting of Telemedicine Programs in Developing Countries: Results of a Systematic Literature Review. Telemedicine journal and e-health : the official journal of the American Telemedicine Association. 2015;21(11):903-15.

Kok MC, Dieleman M, Taegtmeyer M, Broerse JE, Kane SS, Ormel H, et al. Which intervention design factors influence performance of community health workers in low- and middle-income countries? A systematic review. Health policy and planning. 2014;30(9):1207-27.

Kondo KK, Damberg CL, Mendelson A, Motu'apuaka M, Freeman M, O'Neil M, et al. Implementation Processes and Pay for Performance in Healthcare: A Systematic Review. Journal of general internal medicine. 2016;31 Suppl 1:61-9.

Korachais C, Macouillard E, Meessen B. How User Fees Influence Contraception in Low and Middle Income Countries: A Systematic Review. Studies in family planning. 2016;47(4):341-56.

Körner M, Bütof S, Müller C, Zimmermann L, Becker S, Bengel J. Interprofessional teamwork and team interventions in chronic care: A systematic review. Journal of interprofessional care. 2015;30(1):1-14.

Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: A systematic review. Patient education and counseling. 2015;99(1):36-43.

Lee IH, Bloor K, Hewitt C, Maynard A. International experience in controlling pharmaceutical expenditure: influencing patients and providers and regulating industry - a systematic review. Journal of health services research & policy. 2014;20(1):52-9.

Lee SH, Nurmatov UB, Nwaru BI, Mukherjee M, Grant L, Pagliari C. Effectiveness of mHealth interventions for maternal, newborn and child health in low- and middle-income countries: Systematic review and meta-analysis. Journal of global health. 2016;6(1):010401.

Lehnbom EC, Stewart MJ, Manias E, Westbrook JI. Impact of medication reconciliation and review on clinical outcomes. The Annals of pharmacotherapy. 2014;48(10):1298-312.

L'Engle KL, Mangone ER, Parcesepe AM, Agarwal S, Ippoliti NB. Mobile Phone Interventions for Adolescent Sexual and Reproductive Health: A Systematic Review. Pediatrics. 2016;138(3):1-16.

Lin Y, Yin S, Huang J, Du L. Impact of Pay for performance on Behavior of Primary Care Physicians and Patient Outcomes. Journal of evidence-based medicine. 2015;9(1):8-23.

Liu X, Dou L, Zhang H, Sun Y, Yuan B. Analysis of context factors in compulsory and incentive strategies for improving attraction and retention of health workers in rural and remote areas: a systematic review. Human resources for health. 2015;13:61.

Liu X, Hotchkiss DR, Bose S. The effectiveness of contracting-out primary health care services in developing countries: A review of the evidence. Health Policy and Planning. 2007; 23(1): 1-13.

Luangasanatip N, Hongsuwan M, Limmathurotsakul D, Lubell Y, Lee AS, Harbarth S, et al. Comparative efficacy of interventions to promote hand hygiene in hospital: systematic review and network meta-analysis. BMJ (Clinical research ed). 2015;351:h3728.

Mann BS, Barnieh L, Tang K, Campbell DJ, Clement F, Hemmelgarn B, et al. Association between drug insurance cost sharing strategies and outcomes in patients with chronic diseases: a systematic review. PloS one. 2014;9(3):e89168.

Martínez-González NA, Djalali S, Tandjung R, Huber-Geismann F, Markun S, Wensing M, et al. Substitution of physicians by nurses in primary care: a systematic review and meta-analysis. BMC health services research. 2014;14:214.

Martínez-González NA, Rosemann T, Tandjung R, Djalali S. The effect of physician-nurse substitution in primary care in chronic diseases: a systematic review. Swiss medical weekly. 2015;145(no pagination):w14031.

Martínez-González NA, Tandjung R, Djalali S, Rosemann T. The impact of physician-nurse task shifting in primary care on the course of disease: a systematic review. Human resources for health. 2015;13:55.

Mbuagbaw L, Sivaramalingam B, Navarro T, Hobson N, Keepanasseril A, Wilczynski NJ, et al. Interventions for Enhancing Adherence to Antiretroviral Therapy (ART): A Systematic Review of High Quality Studies. AIDS patient care and STDs. 2015;29(5):248-66.

McCollum R, Gomez W, Theobald S, Taegtmeyer M. How equitable are community health worker programmes and which programme features influence equity of community health worker services? A systematic review. BMC public health. 2016;16(1):419.

McCulloch P, Rathbone J, Catchpole K. Interventions to improve teamwork and communications among healthcare staff. The British journal of surgery. 2011;98(4):469-79.

McGrady ME, Ryan JL, Gutiérrez-Colina AM, Fredericks EM, Towner EK, Pai AL. The impact of effective paediatric adherence promotion interventions: systematic review and meta-analysis. Child: care, health and development. 2015;41(6):789-802.

McMillan SS, Kendall E, Sav A, King MA, Whitty JA, Kelly F, et al. Patient-centered approaches to health care: a systematic review of randomized controlled trials. Medical care research and review : MCRR. 2013;70(6):567-96.

Mijovic H, McKnight J, English M. What does the literature tell us about health workers' experiences of task-shifting projects in sub-Saharan Africa? A systematic, qualitative review. Journal of clinical nursing. 2016;25(15-16):2083-100.

Montagu D, Goodman C, Berman P, Penn A, Visconti A. Recent trends in working with the private sector to improve basic healthcare: a review of evidence and interventions. Health policy and planning. 2016;31(8):1117-32.

Musa BM, Iliyasu Z, Yusuf SM, Uloko AE. Systematic review and metanalysis on community based interventions in tuberculosis care in developing countries. Nigerian journal of medicine: journal of the National Association of Resident Doctors of Nigeria. 2014;23(2):103-17.

Mwai GW, Mburu G, Torpey K, Frost P, Ford N, Seeley J. Role and outcomes of community health workers in HIV care in sub-Saharan Africa: a systematic review. Journal of the International AIDS Society. 2013;16(1):18586.

Nazar H, Nazar Z, Portlock J, Todd A, Slight SP. A systematic review of the role of community pharmacies in improving the transition from secondary to primary care. British journal of clinical pharmacology. 2015;80(5):936-48.

Nguyen DT, Leung KK, McIntyre L, Ghali WA, Sauve R. Does integrated management of childhood illness (IMCI) training improve the skills of health workers? A systematic review and meta-analysis. PloS one. 2013;8(6):e66030.

Nijmeijer KJ, Fabbricotti IN, Huijsman R. Is franchising in health care valuable? A systematic review. Health policy and planning. 2014;29(2):164-76.

Nilsson C, Lundgren I, Smith V, Vehvilainen-Julkunen K, Nicoletti J, Devane D, et al. Women-centred interventions to increase vaginal birth after caesarean section (VBAC): A systematic review. Midwifery. 2015;31(7):657-63.

Ofek Shlomai N, Rao S, Patole S. Efficacy of interventions to improve hand hygiene compliance in neonatal units: a systematic review and meta-analysis. European journal of clinical microbiology & infectious diseases : official publication of the European Society of Clinical Microbiology. 2015;34:887-97.

Ogbechie OA, Hsu J. Systematic review of benefit designs with differential cost sharing for prescription drugs. The American journal of managed care. 2015;21(5):e338-48.

Olisemeke B, Chen YF, Hemming K, Girling A. The Effectiveness of Service Delivery Initiatives at Improving Patients' Waiting Times in Clinical Radiology Departments: A Systematic Review. Journal of digital imaging. 2014;27(6):751-78.

Owusu-Addo E, Cross R. The impact of conditional cash transfers on child health in low- and middle-income countries: a systematic review. International journal of public health. 2014;59(4):609-18.

Pallas SW, Minhas D, Pérez-Escamilla R, Taylor L, Curry L, Bradley EH. Community Health Workers in Low- and Middle-Income Countries: What Do We Know About Scaling Up and Sustainability? American journal of public health. 2013;103(7):e74-82.

Patel J, Ahmed K, Guru KA, Khan F, Marsh H, Shamim Khan M, et al. An overview of the use and implementation of checklists in surgical specialities - a systematic review. International journal of surgery (London, England). 2014;12(12):1317-23.

Rashidian A, Omidvari AH, Vali Y, Mortaz S, Yousefi-Nooraie R, Jafari M, et al. The effectiveness of regionalization of perinatal care services--a systematic review. Public health. 2014;128(10):872-85.

Robinson DJ. An integrative review: triage protocols and the effect on ED length of stay. Journal of emergency nursing: JEN : official publication of the Emergency Department Nurses Association. 2013;39(4):398-408.

Roque MD, Herdeiro MT, Soares SI, Teixeira Rodrigues A, Granadeiro LA, Gusman AF. Educational interventions to improve prescription and dispensing of antibiotics: a systematic review. BMC public health. 2014;14(1):1276.

Ruizendaal E, Dierickx S, Peeters Grietens K, Schallig HD, Pagnoni F, Mens PF. Success or failure of critical steps in community case management of malaria with rapid diagnostic tests: a systematic review. Malaria journal. 2014;13(1):229.

Russ S, Rout S, Sevdalis N, Moorthy K, Darzi A, Vincent C. Do safety checklists improve teamwork and communication in the operating room? A systematic review. Annals of surgery. 2013;258(6):856-71.

Ruth Lv, Francke AL, Mistiaen P. Effects of nurse prescribing of medication: a systematic review. The Internet Journal of Healthcare Administration. 2008;5(2).

Sacks GD, Shannon EM, Dawes AJ, Rollo JC, Nguyen DK, Russell MM, et al. Teamwork, communication and safety climate: a systematic review of interventions to improve surgical culture. BMJ quality & safety. 2015;24(7):458-67.

Santschi V, Chiolero A, Colosimo AL, Platt RW, Taffé P, Burnier M, et al. Improving blood pressure control through pharmacist interventions: a meta-analysis of randomized controlled trials. Journal of the American Heart Association. 2014;3(2):e000718.

Schepman S, Hansen J, de Putter ID, Batenburg RS, de Bakker DH. The common characteristics and outcomes of multidisciplinary collaboration in primary health care: a systematic literature review. International journal of integrated care. 2015;15:e027.

Schmutz J, Manser T. Do team processes really have an effect on clinical performance? A systematic literature review. British journal of anaesthesia. 2013;110(4):529-44.

Schweizer ML, Reisinger HS, Ohl M, Formanek MB, Blevins A, Ward MA, et al. Searching for an optimal hand hygiene bundle: a meta-analysis. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2014;58(2):248-59.

Suwannakeeree W, Picheansathian W. Strategies to Promote Adherence to Treatment by Pulmonary Tuberculosis Patients: A systematic review. JBI Database of Systematic Reviews and Implementation Reports. 2012;10(11):615.

Sweeney S, Obure CD, Maier CB, Greener R, Dehne K, Vassall A. Costs and efficiency of integrating HIV/AIDS services with other health services: a systematic review of evidence and experience. Sexually transmitted infections. 2012;88(2):85-99.

Thakkar J, Kurup R, Laba TL, Santo K, Thiagalingam A, Rodgers A, et al. Mobile Telephone Text Messaging for Medication Adherence in Chronic Disease: A Meta-analysis. JAMA internal medicine. 2016;176(3):340-9.

Trehan A, Maruthappu M, Barnett-Vanes A, Carty M, McCulloch P. Does feedback of surgical outcome data improve surgical performance? A systematic review. Journal of the American College of Surgeons. 2014;219(4):e148.

Tripathi A, Kabra SK, Sachdev HP, Lodha R. Home visits by community health workers to improve identification of serious illness and care seeking in newborns and young infants from low- and middle-income countries. Journal of perinatology : official journal of the California Perinatal Association. 2016;36 Suppl 1(S1):S74-82.

Tshiananga JK, Kocher S, Weber C, Erny-Albrecht K, Berndt K, Neeser K. The effect of nurse-led diabetes self-management education on glycosylated hemoglobin and cardiovascular risk factors: a meta-analysis. The Diabetes educator. 2011;38(1):108-23.

Tsiachristas A, Wallenburg I, Bond CM, Elliot RF, Busse R, van Exel J, et al. Costs and effects of new professional roles: Evidence from a literature review. Health policy (Amsterdam, Netherlands). 2015;119(9):1176-87.

Turcotte-Tremblay AM, Spagnolo J, De Allegri M, Ridde V. Does performance-based financing increase value for money in low- and middle- income countries? A systematic review. Health economics review. 2015;6(1):30.

Uyei J, Coetzee D, Macinko J, Guttmacher S. Integrated delivery of HIV and tuberculosis services in Sub-Saharan Africa: A systematic review. International Initiative for Impact Evaluation (3ie). 2011.

Van Camp YP, Van Rompaey B, Elseviers MM. Nurse-led interventions to enhance adherence to chronic medication: systematic review and meta-analysis of randomised controlled trials. European journal of clinical pharmacology. 2013;69(4):761-70.

van Velthoven MHMMT, Tudor Car L, Gentry S, Car J. Telephone delivered interventions for preventing HIV infection in HIV-negative persons. Cochrane Database of Systematic Reviews. 2013;5(5):CD009190.

Wagner B, Filice GA, Drekonja D, Greer N, MacDonald R, Rutks I, et al. Antimicrobial stewardship programs in inpatient hospital settings: a systematic review. Infection control and hospital epidemiology : the official journal of the Society of Hospital Epidemiologists of America. 2014;35(10):1209-28.

Wald DS, Butt S, Bestwick JP. One-way versus two-way text messaging on improving medication adherence: meta-analysis of randomized trials. The American journal of medicine. 2015;128(10):1139.e1-5.

Wallace AS, Ryman TK, Dietz V. Experiences integrating delivery of maternal and child health services with childhood immunization programs: systematic review update. The Journal of infectious diseases. 2012;205 Suppl 1:S6-19.

Wallace J, Byrne C, Clarke M. Improving the uptake of systematic reviews: a systematic review of intervention effectiveness and relevance. BMJ open. 2014;4(10):e005834.

Watson SJ, Aldus CF, Bond C, Bhattacharya D. Systematic review of the health and societal effects of medication organisation devices. BMC health services research. 2016;16(1):202.

Wilcher R, Hoke T, Adamchak SE, Cates W. Integration of family planning into HIV services: a synthesis of recent evidence. AIDS (London, England). 2013;27 Suppl 1:S65-75.

World Health Organization, University of California SF. Electronic reminders for promoting adherence to ART among people living with HIV. World Health Organization. 2013.

World Health Organziation, University of California SF. Integration of HIV and TB services. World Health Organization. 2013.

Yamada J, Shorkey A, Barwick M, Widger K, Stevens BJ. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ open. 2015;5(4):e006808.

Zulu JM, Kinsman J, Michelo C, Hurtig AK. Integrating national community-based health worker programmes into health systems: a systematic review identifying lessons learned from low-and middle-income countries. BMC public health. 2014;14(1):987.

Contributions of authors

All of the authors contributed to drafting and revising the overview. All of the authors contributed important intellectual input to the overview.

Declarations of interest

Cristian A Herrera, Simon Lewin, Elizabeth Paulsen, Newton Opiyo, Tomas Pantoja, Gabriel Rada, and Andrew D Oxman are editors of the Cochrane Effective Practice and Organisation of Care (EPOC) Group. Agustín Ciapponi, Tomas Pantoja, Gabriel Rada, Cristian A Herrera, Andrew D Oxman, and Blanca Peñaloza are authors on some of the included reviews. Charles S Wiysonge, Gabriel Bastías, Sebastian Garcia Marti, and Charles I Okwundu have no relevant conflicts to declare.

Sources of support

Internal sources

  • Department of Family Medicine, School of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile.

  • Institute for Clinical Effectiveness and Health Policy, Buenos Aires, Argentina.

  • Norwegian Knowledge Centre for the Health Services, Oslo, Norway.

  • Department of Public Health, School of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile.

  • South African Medical Research Council, Cape Town, South Africa.

External sources

  • Norwegian Agency for Development Cooperation (Norad), Oslo, Norway.

  • National Research Foundation (CSW), South Africa.

  • The Effective Health Care Research Consortium which is funded by UK aid from the UK Government for the benefit of developing countries, UK.

Ancillary