Decision-support tools via mobile devices to improve quality of care in primary healthcare settings

  • Protocol
  • Intervention



This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

The objective of this review is to assess the effect of digital, clinical decision-support tools, accessible via mobile devices by primary healthcare providers in the context of primary care settings, on providers' adherence to recommended practices, time taken for appropriate management, providers' and patients' acceptability and satisfaction, health status, and resource use..


The use of clinical decision-support tools on mobile devices may help primary healthcare providers, including frontline healthcare workers in developing countries, to improve the quality of services provided. The provision of appropriate, evidence-based, quality healthcare services is a concern of global policy makers.

Description of the condition

To improve the quality and safety of preventative and clinical healthcare services remains a challenge. There is widespread recognition that the quality of healthcare services varies widely, and is often suboptimal across healthcare systems (Moja 2014). In primary healthcare, despite the availability of knowledge, and specific diagnostic, treatment, and management protocols, there is often a discrepancy between the knowledge and the application. This know-do gap in the quality of healthcare has been widely cited as a key barrier to improving healthcare outcomes (Blank 2013; Mohanan 2015). A range of systemic factors contribute to the deficiencies and challenges in delivering high-quality evidence-based healthcare services.

Globally, the World Health Organization (WHO) projects a shortfall of 12.9 million healthcare providers by 2035 (WHO 2014). Having the right type of provider, at the right time, at the right place, continues to be a challenge worldwide. Clustering of health personnel in capital cities and other urban areas, and out-migration from low- and middle-income countries (LMICs) to high-income countries (HICs), further diminishes the number of healthcare providers available in rural areas (Dussault 2006). Especially in LMICs, the lack of trained primary healthcare providers has prompted policy makers to explore a shift of key tasks from higher to lower cadres of health workers (Baker 2007; Lehman 2008; WHO 2007). While the transition of vital primary healthcare services to a lower cadre of workers is feasible, it requires ongoing training support to ensure that service quality and safety standards are maintained (Rednick 2014; WHO 2007).

In other clinical settings, even when an adequate number of providers with the right training are available, the quality of care is variable. Busy healthcare providers may struggle to stay abreast of current evidence and apply it consistently. They may also lack information on alternate risk-reduction approaches, and not be sure which treatments work best (Kocher 2010). Time-constraints may result in the omission of essential information for counselling the patient, and long work hours may result in increased errors (Bright 2012).

Standardized protocols, which provide critical information at the point-of-care, support decision-making and guide healthcare providers through the process of diagnosis and management. They can introduce efficiencies into the system, optimize the time with the client, and improve the overall quality of services (Bright 2012; Mickan 2014).

Description of the intervention

Several challenges encountered in clinical practice could benefit from the use of clinical decision-support systems. A clinical decision-support system (CDSS) is 'any electronic system designed to aid directly in clinical decision making, in which characteristics of individual patients are used to generate patient-specific assessments or recommendations that are then presented to clinicians for consideration' (Kwamoto 2005). The increasing ubiquity and affordability of smartphones and tablets has made it possible for clinical decision-support tools to become available to healthcare providers on handheld devices at the point-of-care. Mobile decision-support tools can potentially address some of the challenges faced by many healthcare systems regarding adequate training of healthcare providers, shifting clinical tasks from clinicians to frontline health workers with limited formal training, and improving the quality of preventative, diagnostic, and treatment care across a range of health issues.

Over the last decade, personal digital assistants (PDAs), and other wireless mobile devices, such as smartphones and tablets are increasingly being used at the point-of-care to obtain evidence and guidance on clinical conditions, do necessary calculations for drugs, and access other medical information. They even support more advanced clinical decision-support systems linked to a comprehensive patient database (Divall 2013). Where before, in most low-income settings, only basic features, such as voice calls (or interactive voice response (IVR)) and short message service (SMS) were commonplace, the pace of growth in mobile technology increasingly allows for a range of functionality — low-cost access to the internet, high quality cameras for still and video footage, applications stored on-device, pre-loaded audio or video clips and images, global positioning service (GPS), and the potential to connect additional sensors and devices — that allows for the development of very sophisticated point-of-care decision-support systems, even in low resource settings.

Mobile CDSS may vary in the range of functionality and applications to improve diagnoses, facilitate evidence-based screening, counselling, and treatment, and improve workflow efficiencies

Broadly, CDSS may serve the following functions:

  • guide the healthcare provider through process algorithms using if...then... rules, based on evidence-based clinical protocols;

  • provide the healthcare provider with a checklist, based on clinical protocols;

  • provide step-by-step guidance to screen clients by health status or risk status, possibly using models based on machine learning, where mathematical functions might be used to classify patients into risk groups.

CDSS may be integrated with electronic health records or stand alone. For the purpose of this review, we will focus only on mobile decision-support tools that are not integrated with an electronic health record or management system, or are integrated with such a system but can be used independent of it.

How the intervention might work

Approaches to using mobile clinical decision-support tools vary substantially across countries and contexts, in part, depending on the availability of technological and healthcare infrastructure, and the costs of mobile devices and data packages. In addition to the costs and infrastructure, the level of sophistication of a CDSS would also depend on the complexity of the disease, the purpose of the CDSS (for example, screening alone, or screening integrated with risk assessment and counselling), and the capacity of the healthcare providers to adopt such systems.

At the most basic level, a decision-support system might comprise of a process algorithm, and transfer a paper-based protocol into a digital format. For example, a series of digital If...then... logic-guided questions may be used to assess appropriate contraceptive choices for a client, based on demographic information and preferences. By inputting client data in a systematic way, a decision-support tool might be used to identify and prioritize clients into risk groups. Additional point-of care support mechanisms, such as automated algorithmic instructions that prompt healthcare workers to follow certain guidelines, and provide tailored counselling messages and recommendations, might be added to such a system. For example, in addition to the assessment for an appropriate contraceptive, the decision-support tool may provide a number of recommendations, and prompt the healthcare worker to discuss risks for each contraceptive method, and care and follow-up for the contraceptive method chosen by the client. At each step, the provider might be required to check off a counselling item that has been discussed, before the system moves to the next set of questions and recommendations. In theory, such a system would promote comprehensive evidence-based counselling, and improve the overall quality of care provided by the healthcare worker.

In high-income countries, CDSS is typically used in clinical settings by trained healthcare providers or medical students. In LMICs, decision-support tools are used for both in-clinic and community-based outreach visits, by a range of healthcare providers, including clinicians, midwives, and lay healthcare workers.

Why it is important to do this review

The use of decision-support tools for clinical decision-making has been well-established, and is supported by some emerging evidence. A systematic review of 148 randomized, controlled trials on the effectiveness of CDSS, reported that the use of CDSS was associated with significant improvements in a range of morbidity outcomes, healthcare process measures related to performing preventive services, diagnostic testing, and improved adherence to treatment regimens (Bright 2012). However, over the last decade, clinical decision-support tools have transitioned from being operated on stationary computer systems to wireless mobile devices, which provide unique opportunities for point-of-care assessments, diagnoses, and management. Furthermore, most healthcare systems in LMICs, especially in rural areas, do not have the required infrastructure for computerized CDSS. However, the use of these tools on wireless digital devices, make them accessible to healthcare providers in LMICs, which was not possible previously. Despite the substantial investments and global interest in using mobile digital devices to support clinical decision-making, specific evidence on the effectiveness of such interventions on clinical and public health practice is limited.

Two systematic reviews assessed whether the use of handheld computers, primarily personal digital assistants (PDAs), improved access to information and supported point-of-care clinical decision making (Divall 2013; Mickan 2014). Compared to paper resources, the studies suggested that using handheld computers improved access to information, adherence to clinical guidelines, and appropriate diagnostic decision making. One study reported improvements in data collection quality (Divall 2013). This review will build on the existing studies, to include the use of other mobile devices, such as smartphones and tablets — the most current forms of handheld digital devices, especially in low- and middle-income countries. One review assessed the feasibility of, and barriers to, using digital point-of-care decision-support tools by healthcare providers in Africa (Adepoju 2017). Based on largely descriptive and observational studies, conducted in seven sub-Saharan African countries, the review concluded that healthcare providers found mobile decision-support tools useful; however, they expressed concerns about altered workflows and increased workloads. The review identified technical and infrastructural support, and adequate training, as key barriers to adopting clinical decision-support tools in the sub-Saharan African context.

Digital, mobile, wireless technologies provide an innovative and accessible platform to accelerate health services and improve quality of care for some of the most hard-to-reach populations. Given the recent emergence of such technologies for health, there is considerable demand from ministries of health, donors, and decision makers for evidence-based guidance to invest in such technologies. In response to this global need, the WHO is developing guidelines to inform investments on digital health approaches. This review constitutes one of six reviews on the effectiveness of digital health interventions that will be used directly to inform these WHO Guidelines.


The objective of this review is to assess the effect of digital, clinical decision-support tools, accessible via mobile devices by primary healthcare providers in the context of primary care settings, on providers' adherence to recommended practices, time taken for appropriate management, providers' and patients' acceptability and satisfaction, health status, and resource use..


Criteria for considering studies for this review

Types of studies

We will include randomized trials, available as full-text studies, conference abstracts, and unpublished data. We will include studies regardless of their publication status and language of publication.

Types of participants

  • all cadres of healthcare providers (i.e. professionals, paraprofessionals, and lay health workers) who are providing healthcare services to patients, using digital, clinical decision-support tools in the context of a primary care setting;

  • other individuals or groups involved in the delivery of primary healthcare services. These individuals or groups could include administrative staff, managerial, and supervisory staff, who may or may not be based in a primary healthcare facility or in the community, but must be involved in supporting the delivery of primary healthcare services using digital, clinical decision-support tools.

  • clients or patients receiving care from primary healthcare providers who are using decision-support tools.

We will include participants regardless of their location, professional status, condition or demographic factors, such as age.

Types of interventions

We will include studies that compare digital, clinical decision-support tools accessible via mobile device with non-digital decision-support tools, or no intervention, in the context of primary care. We will include studies in which digital, decision-support tools are developed for use primarily on a mobile device, and are used by health workers for the purpose of service delivery, to follow clinical protocols, guide service delivery using checklists and job aids, or prioritize clients by risk or other health status in a primary healthcare setting.

By mobile devices, we mean mobile phones of any kind (but not analogue landline telephones), as well as tablets, personal digital assistants, and smartphones. We will include studies if a mobile device is used, and the tool is intended to be used in a mobile state. For example, if websites or other applications are used, they should be optimized for use on a mobile device, and healthcare workers should be trained to use the mobile device. We will include studies in which a laptop is used as a tablet, with applications customized for such use.

By primary healthcare services, we mean a combination of the following:

  • the first contact point of healthcare (Awofeso 2004), including care delivered at an individual level, community level, or both (Muldoon 2006), by individual healthcare providers or teams of providers, and intended to provide and co-ordinate care in settings where people work and live, or provide continuity of care (Muldoon 2006);

  • any healthcare that prevents illness, promotes health, is therapeutic, or rehabilitative (Global Health Watch 2011).

The intervention may be implemented in public or private healthcare facilities, in the community, or the homes of the patients. We will include studies in any country.

The comparisons for this review will be:

  • digital decision-support tools accessible via mobile device compared to non-mobile and non-digital decision-support tools (e.g. a mobile job-aid versus a paper job-aid);

  • digital decision-support tools accessible via mobile device compared to standard practice (i.e. non-digital intervention or no intervention).

Network meta analysis could allow us to statistically infer a non-digital/non-mobile comparison group by comparing a study that compares digital intervention to non-digital intervention with another study that compares one type of a digital intervention with another type of a digital intervention. However, this approach requires a relatively high level of homegeneity among interventions that are grouped together as one intervention, and requires that the populations and other aspects of the context and outcomes are also similar across all comparisons. The types of interventions included in this review will likely have important differences in the populations, both in terms of healthcare providers and the service users, and the contexts, and are likely to be much more varied than the trials that are typically conducive to network meta-analysis.

We will exclude:

  • studies in which the use of the digital decision-support tool is dependent on its integration with an electronic medical record or other types of client health-tracking tools;

  • studies in which the use of the decision-support tool is primarily for the purpose of training alone, and does not involve direct service delivery;

  • studies in which digital decision support is conducted on stationary computers or laptops alone;

  • studies that compare one type of mobile decision support with another type of mobile decision support;

  • studies in which patients use the digital decision-support systems;

  • pilot and feasibility studies (pilot study is defined as 'a version of the main study that is run in miniature to test whether the components of the main study can all work together', and feasibility study is defined as 'pieces of research done before a main study'; (Arain 2010)),

We will include studies in which digital decision-support tools are delivered as part of a wider package (such as sending messages to the client or provider, supporting the provider in prioritising clients, etc.), if the decision-support tool is the major component of the intervention.

Types of outcome measures

  1. Providers’ adherence to recommended practice, guidelines, or protocols (e.g. providing the service at the recommended time, referral as recommended, screening and prioritising as recommended).

  2. Time between presentation and appropriate management, including time for referrals and service linkages.

  3. Patients' or clients' health status and well-being, assessed through validated measures, if available.

  4. Provider acceptability and satisfaction with the intervention, assessed through validated measures, if available.

  5. Patients' or clients' acceptance of and satisfaction with the intervention, assessed through validated measures, if available.

  6. Resource use (e.g. human resources and time, training, supplies, and equipment).

  7. Unintended consequences that result in an adverse effect of the intervention (these could include misreading or misinterpretation of the data; transmission of inaccurate data, for instance incorrect underlying algorithms or clinical protocols; loss of verbal or non-verbal communication cues; decreased direct engagement with patient; issues of privacy and disclosure; loss (including theft) or misuse of device (in cases in which health workers are given the phones or tablets); interrupted workflow due to infrastructural constraints for battery recharging and network coverage; impacts on equity; disruptions on the delivery of health services, unforeseen ill-effects on patient outcomes).

Search methods for identification of studies

We will restrict the search from 2000 to the date of search. This is based on the increased availability and penetration of mobile devices in low- and middle-income countries starting in 2000 (International Telecommunications Union 2015).

Electronic searches

An independent information specialist (JE) will develop the search strategies in consultation with the review authors.

We will search the following databases for primary studies, from 2000 to the date of search.

  • Cochrane Central Register of Controlled Trials (CENTRAL; latest issue), in the Cochrane Library;

  • MEDLINE Ovid;

  • Embase Ovid;


  • WHO Global Health Library.

Appendix 1 lists the search strategy for MEDLINE Ovid. Search strategies are comprised of keywords and controlled vocabulary terms. We will not apply any limits on language. We will use a modified version of the Cochrane Highly Sensitive Search Strategy to identify randomized trials (Lefebvre 2011).

Searching other resources

We will search for ongoing trials in the following trial registries:

  • WHO ICTRP (World Health Organization International Clinical Trials Registry Platform;;

  • US National Institutes of Health Ongoing Trials Register (

We will search Epistemonikos ( for relevant systematic reviews and potentially eligible primary studies. In addition, the WHO will issue a call for papers through popular digital health communities of practice, such as the Global Digital Health Network and Implementing Best Practices, to identify additional primary studies and grey literature.

Grey literature

We will search for grey literature. The search portal for is more limited, therefore, we will review the titles and abstracts of all contributed content that is not referenced in Medline Ovid.

We will review reference lists of all included studies and relevant systematic reviews for additional potentially eligible primary studies. We will contact authors of included studies and reviews to clarify reported published information and to seek unpublished results and data.

Data collection and analysis

Selection of studies

A core team of two review authors (NH, NM), with assistance where necessary from an additional review author (SA), will be responsible for the selection of studies. We will download all titles and abstracts retrieved by electronic searching of all databases to a reference management database and remove duplicates (DistillerSR). We will use a machine learning classifier that is able to assign a probability score that a given record describes, or does not describe, a randomized trial. It was built based on 280,000 titles and abstracts from Embase, which have been manually labelled by the Cochrane Crowd (Wallace 2017). We will send all the search results through the classifier. Two review authors will independently screen the titles and abstracts of studies with a 10% probability or greater of being a randomized trial; one review author will screen those with a less than 10% probability of being a randomized trial.

We will retrieve the full-text study reports and publications of studies that are screened and included. Two review authors (NH, NM) will independently screen the full-texts to identify studies to include, and record reasons for excluding ineligible studies. We will resolve any disagreements through discussion, or if required, we will consult a third review author (SA).

We will list studies that initially appeared to meet the inclusion criteria, but were excluded after full-text review, in the 'Characteristics of excluded studies' table. We will collate multiple reports of the same study, so that each study rather than each report is the unit of interest in the review. We will also provide any information we can obtain about ongoing studies. We will record the selection process in sufficient detail to complete a PRISMA flow diagram (Liberati 2009).

Data extraction and management

We will modify the EPOC standard data collection form and adapt it for our study characteristics and outcome data (EPOC 2017a). We will identify key characteristics of the intervention for extraction based on the mHealth Evidence Review and Assessment (mERA) guidelines (Agarwal 2016). We will pilot the form on at least one study in the review. Two review authors (NH, NM) will independently extract the following study characteristics from the included studies.

  1. General information: title, reference details, author contact details, publication type, funding source, conflicts of interest of study authors.

  2. Population and setting: country, geographical location (rural, urban, or peri-urban, defined as outskirts of urban areas), healthcare setting (e.g. facility-based, home-based).

  3. Methods: function of the intervention, study design, unit of allocation, duration of participation.

  4. Participant characteristics: type of healthcare worker (function, age, length of training), description of clients serviced by the healthcare worker, description of any other participants in the intervention, withdrawal.

  5. Interventions: intervention purpose, components, type of technology (hardware and software characteristics) and model of delivery, type of mobile device(s) used (smartphone, tablet, feature phone, basic phone, laptops), phone ownership, content of the intervention, health provider training, interoperability, compliance with national guidelines, data security, comparison, fidelity assessment, duration of intervention.

  6. Outcomes: primary and other outcomes specified and collected, time points reported, adverse events, results of any subgroup analyses.

Two review authors (NM, NH) will independently extract outcome data from the included studies. We will note in the 'Characteristics of included studies' table if outcome data were reported in an unusable way. We will resolve disagreements by consensus, or by involving a third review author (SA).

Assessment of risk of bias in included studies

Two review authors (NH, NM) will independently assess risk of bias for each included study, using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2017), and the guidance from the EPOC group for assessing randomized trials (EPOC 2017b). Any disagreements will be resolved by discussion, or by involving a third review author (SA). We will assess the risk of bias according to the following domains: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, similarity of baseline characteristics, and any other bias. We will assess incomplete outcome data separately for different outcomes.

We will judge each potential source of bias as high, low, or unclear and provide a quote from the study report, together with a justification for our judgment in the 'Risk of bias' table. We will summarize the 'Risk of bias' judgments across different studies for each of the domains listed. We will consider blinding separately for different key outcomes where necessary. Where information on risk of bias relates to unpublished data, or correspondence with a trialist, we will note this in the 'Risk of bias' table. We will not exclude studies on the grounds of their risk of bias, but will clearly report the risk of bias when presenting the results of the studies. When considering treatment effects, we will take into account the risk of bias for the studies that contributed to that outcome. We will further perform assessment of quality of evidence using the GRADE approach (Guyatt 2008). We will summarize the findings in the ‘Summary of findings’ tables.

We will conduct the review according to this published protocol, and report any deviations from it in the 'Differences between protocol and review' section of the systematic review.

Measures of treatment effect

We will estimate the effect of the intervention using risk ratio (RRs) for dichotomous data, together with the appropriate, associated 95% confidence interval (CI), and mean difference (MD), or standardised mean difference (SMD) for continuous data, together with the appropriate, associated 95% CI (Higgins 2017). We will ensure that an increase in scores for continuous outcomes can be interpreted in the same way for each outcome, explain the direction to the reader, and report where the directions were reversed, if this was necessary.

Unit of analysis issues

For cluster-randomized trials that do not account adequately for the effects of the clustering on the effect estimate, we will adjust the analyses to avoid unit-of-analysis errors, if possible. If insufficient information is available to reanalyze the results, we will contact the authors of the primary paper to obtain necessary data. If these data are not available, and we are concerned there is a unit of analysis error, we will report the effect estimates without confidence intervals or P values.

Dealing with missing data

We will contact investigators in order to verify key study characteristics and obtain missing outcome data where possible (e.g. when a study is identified as abstract only). If this is not possible, we will report the data as missing, note this in the 'Risk of bias' tables, and not attempt to impute the missing values. For all outcomes, we will carry out the analyses, to the extent possible, on an intention-to-treat basis based on the available cases. However, in assessing adverse events, we will relate the results to the actual treatment received i.e. the analyses will be based on the participants who actually received the intervention, and the number of adverse events reported in the studies.

Assessment of heterogeneity

If we find studies that are similar enough to combine, we will conduct a meta-analysis. We will examine heterogeneity by visual inspection of forest plots, as well as using the I² statistic to measure heterogeneity among the trials in each analysis. If we identify substantial heterogeneity, we will explore it by subgroup analysis, listed below.

Assessment of reporting biases

We will attempt to contact study authors, asking them to provide missing outcome data. Where this is not possible, and the missing data are thought to introduce serious bias, we will explore the impact of including such studies in the overall assessment of results. If we are able to pool more than 10 trials within a comparison, we will create and examine a funnel plot to explore possible publication biases, interpreting the results with caution (Sterne 2011).

Data synthesis

Where intervention characteristics and outcome measures are similar across included studies, i.e. if the treatments, participants, and the underlying clinical question are similar enough for pooling to make sense, we will conduct a meta-analysis to estimate an overall effect size. If analyses, adjusted for potential confounders, are reported for either dichotomous or continuous outcomes, we will use estimates of effect from the primary analysis reported by the investigators, and convert these to risk ratios (RR), if possible. In cases where the adjusted analyses for dichotomous outcomes are reported using odds ratios (OR) and not risk ratios, we will use the Cochrane Collaboration’s statistical software, Review Manager 2014, to convert OR to RR before including the result in a meta-analysis.

A common way that trialists indicate they have skewed data, is by reporting medians and interquartile ranges. When we encounter this, we will note that the data are skewed, and consider the implication of this.

Where multiple trial arms are reported in a single trial, we will include only the relevant arms. If two comparisons (e.g. intervention A versus usual care and intervention B versus usual care) must be entered into the same meta-analysis, we will halve the control group to avoid double-counting.

Where studies are not similar enough to allow meta-analysis, we will report the results in a narrative format.

'Summary of findings' table and GRADE

Two review authors will independently assess the certainty of the evidence (high, moderate, low, and very low) using the five GRADE considerations (risk of bias, consistency of effect, imprecision, indirectness, and publication bias; (Guyatt 2008)). We will use the methods and recommendations described in the Cochrane Handbook for Systematic Reviews of interventions (Schünemann 2017), and the EPOC worksheets (EPOC 2017c), and use GRADEpro software (GRADEpro GDT). We will resolve disagreements on certainty ratings by discussion, and provide justification for decisions to down- or upgrade the ratings using footnotes in the table, and make comments to aid readers' understanding of the review, where necessary. We will use plain language statements to report these findings in the review (EPOC 2017c).

We will create a 'Summary of findings' table for the main comparisons for the following outcomes: providers’ adherence to recommended practice, guidelines, or protocols; time between presentation and appropriate management; patients' or clients' health status and well-being; provider acceptability and satisfaction with the intervention; patients' or clients' acceptance of and satisfaction with the intervention; resource use; unintended consequences that result in an adverse effect of the intervention. We will draw conclusions about the certainty of the evidence within the text of the review. If, during the review process, we become aware of an important outcome that we failed to list in our planned 'Summary of findings' table, we will include the relevant outcome and explain the reasons for this in the section 'Differences between protocol and review'.

We will consider whether there is any additional outcome information that we were not able to incorporated into the meta-analyses, note this in the comments, and state whether it supports or contradicts the information from the meta-analyses. If it is not possible to meta-analyse the data, we will summarize the results in the text.

Subgroup analysis and investigation of heterogeneity

We will perform subgroup analysis to assess the variation in the delivery of the intervention across different population groups, interventions, or setting characteristics, if possible. We will conduct subgroup analyses only if a sufficient number of trials are available to make statistically significant comparisons between groups. We plan to carry out the following subgroup analyses.

  1. Type of geographic setting – (for example, urban, rural, peri-urban; low- and middle-income countries), as we anticipate that the intervention may have different effects due to social and economic differences between settings.

  2. Type of healthcare setting – (for example, community-based, clinic-based), as we anticipate this may influence the way the intervention is delivered and its effects.

  3. Provider type (for example, lay provider, versus professional healthcare provider), as we anticipate that different types of providers may vary in their use of decision-support tools using a mobile device.

  4. Intervention characteristics (for example, smartphones versus tablets, functional characteristics, such as the use of decision support as checklists, screening tools, and for risk stratification and treatment, duration of implementation), phone ownership, mode of payment (if any), number of phones in use after a certain period of time, as the effect of decision-support tools may be different, depending on the purpose of its use.

  5. Health area (for example, chronic disease, infectious disease, maternal health, child health), as the effect of using decision-support tools may vary, based on the type of services accessed to address a health area.

We will assess for heterogeneity within each subgroup using forest plots, and the I² statistic. We will summarize the results of the subgroup analysis within the text of the review or state if meta-analysis is not possible or meaningful.

Sensitivity analysis

We will perform sensitivity analyses to assess the robustness of our conclusions and explore its impact on effect sizes. We will (i) restrict any meta-analysis to published studies only, (ii) remove studies from any meta-analyses that have a high risk of bias, based on the 'Risk of bias' assessment.


We acknowledge the help and support of Cochrane Effective Practice and Organisation of Care (EPOC). The authors would also like to thank Xavier Bosch-Capblanch and Daniela Gonçalves Bradley for providing comments to improve the protocol. We would also like to thank John Eyers for designing the search strategies and Vicki Pennick for copy-editing the protocol.

National Institute for Health Research, via Cochrane Infrastructure funding to the Effective Practice and Organisation of Care Group. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Systematic Reviews Programme, NIHR, NHS, or the Department of Health.

We are grateful to the Guideline Development Group of the Digital Health Guidelines for their constructive feedback in formulating the guiding questions for this systematic review.

Cochrane Response, an evidence services unit operated by Cochrane will provide screening and data extraction services for this review.


Appendix 1. MEDLINE search strategy

Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations, Ovid MEDLINE(R) Daily and Ovid MEDLINE(R) <1946 to Present> - Searched 19th July 2017

1 Decision Support Systems, Clinical/ (6639)

2 Decision Making/ (82937)

3 Decision Support Techniques/ (17324)

4 Diagnosis, Computer-assisted/ (21663)

5 Decision Support Systems, Management/ (961)

6 Expert Systems/ (3380)

7 Point-of-Care Systems/ (10238)

8 Guideline Adherence/ or Clinical Protocols/ or Checklist/ (56976)

9 Therapy, Computer-Assisted/ (6325)

10 "Drug Therapy, Computer-Assisted"/ (1641)

11 Electronic Prescribing/ (847)

12 Clinical Laboratory Information Systems/ or Clinical Pharmacy Information Systems/ (3192)

13 ((decision* adj3 (make or makes or making or made or support* or algorithm* or aid or aids or app or apps or application* or technique*)) or expert system* or job-aid* or "job aid*").ti,ab,kw. (145619)

14 ((therap* or prescrib* or prescript* or diagnos*) adj2 (computer* or digital or electronic)).ti,ab,kw. (7255)

15 (((guideline* or protocol*) adj4 (adher* or comply or complian* or observ*)) or checklist*).ti,ab,kw. (46010)

16 or/1-15 (343799)

17 Cell Phones/ (7064)

18 Smartphone/ (1222)

19 MP3-Player/ (167)

20 Computers, Handheld/ (3086)

21 ((cell* or mobile*) adj1 (phone* or telephone* or technolog* or device*)).ti,ab,kw. (13082)

22 (handheld or hand-held).ti,ab,kw. (9940)

23 (smartphone* or smart-phone* or cellphone* or mobiles).ti,ab,kw. (5505)

24 ((personal adj1 digital) or (PDA adj3 (device* or assistant*)) or MP3 player* or MP4 player*).ti,ab,kw. (1294)

25 (samsung or nokia).ti,ab,kw. (817)

26 (windows adj3 (mobile* or phone*)).ti,ab,kw. (42)

27 android.ti,ab,kw. (1524)

28 (ipad* or i-pad* or ipod* or i-pod* or iphone* or i-phone*).ti,ab,kw. (1957)

29 (tablet* adj3 (device* or computer*)).ti,ab,kw. (989)

30 Telemedicine/ (16654)

31 Webcasts as topic/ (284)

32 Text Messaging/ (1650)

33 Telenursing/ (174)

34 (mhealth or m-health or "mobile health" or ehealth or e-health or "electronic health" or "digital health" or uhealth or u-health).ti,ab,kw. (15673)

35 (telemedicine or tele-medicine or telehealth or tele-health or telecare or tele-care or telenursing or tele-nursing or telepsychiatry or tele-psychiatry or telemonitor* or tele-monitor* or teleconsult* or tele-consult* or telecounsel* or tele-counsel* or telecoach* or tele-coach*).ti,ab,kw. (13866)

36 (webcast* or web-cast*).ti,ab,kw. (215)

37 (((text* or short or voice or multimedia or multi-media or electronic or instant) adj1 messag*) or instant messenger).ti,ab,kw. (3421)

38 (texting or texted or texter* or ((sms or mms) adj (service* or messag*)) or interactive voice response* or IVR or voice call* or callback* or voice over internet or VOIP).ti,ab,kw. (2543)

39 (Facebook or Twitter or Whatsapp* or Skyp* or YouTube or "You Tube" or Google Hangout*).ti,ab,kw. (4093)

40 Mobile Applications/ (2217)

41 "mobile app*".ti,ab,kw. (1720)

42 Reminder Systems/ (3053)

43 (remind* adj3 (text* or system* or messag*)).ti,ab,kw. (1403)

44 Medical informatics/ or Medical informatics applications/ (12864)

45 Nursing informatics/ or Public health informatics/ (2461)

46 ((medical or clinical or health or healthcare or nurs*) adj3 informatics).ti,ab,kw. (5006)

47 Computer-Assisted Instruction/ (11260)

48 ((interactive or computer-assisted) adj1 (tutor* or technolog* or learn* or instruct* or software or communication)).ti,ab,kw. (2213)

49 or/17-48 (104026)

50 randomized controlled (469749)

51 controlled clinical (94451)

52 randomized.ab. (403554)

53 placebo.ab. (189035)

54 drug therapy.fs. (2020527)

55 randomly.ab. (280309)

56 trial.ab. (423300)

57 groups.ab. (1724593)

58 or/50-57 (4120672)

59 exp animals/ not (4441809)

60 58 not 59 (3557949)

61 16 and 49 and 60 (2446)

62 limit 61 to yr="2000 -Current" (2270)

Contributions of authors

Conceiving the protocol: CG, TT, SL, SA, GM, MS

Designing the protocol: CG, TT, SL, SA, NM, NH, MS, GM

Co-ordinating the protocol: SA, TT, CG, SL

Writing the protocol: SA, TT, CG

Providing general advice on the protocol: CG, SL, TT, GM

Securing funding for the protocol: GM

Declarations of interest

SA: The author was commissioned by the WHO to conduct this review. CG: None known. TT: None known. SL: I am the Joint Co-ordinating Editor for Cochrane Effective Practice and Organisation of Care. NM: I previously worked for Enhanced Reviews Ltd, a company that conducts systematic reviews, mostly for the public sector. Since June 2016, I have been employed by Cochrane Response, an evidence services unit operated by Cochrane. Cochrane Response was contracted by the WHO to produce this review. NH: Since June 2016, I have been employed by Cochrane Response, an evidence services unit operated by Cochrane. Cochrane Response was contracted by the WHO to produce this review. MS: None known. GM: Owns stock in Apple computers

Sources of support

Internal sources

  • No sources of support supplied

External sources

  • This work was funded by the UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), a cosponsored program executed by the World Health Organization (WHO), Switzerland.


This protocol is based on standard text and guidance provided by Cochrane Effective Practice and Organisation of Care (EPOC).