Digital tracking, provider decision support systems, and targeted client communication via mobile devices to improve primary health care

  • Protocol
  • Intervention



This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

In the context of primary healthcare settings, we aim to assess:

  • effects of digitally tracking clients' health service use and status combined with decision support conducted via mobile device;

  • effects of digitally tracking clients’ health service use and status combined with TCCs accessible via mobile device; and

  • effects of digitally tracking clients' health service use and status combined with decision support conducted via mobile device and TCCs accessible via mobile device.


Description of the condition

The World Bank/World Health Organization (WHO)/US Agency for International Development (USAID) Roadmap for Health Measurement and Accountability Post-2015 (MA4Health) report underlined that “Public health and clinical care cannot be delivered safely, with high quality, and in a cost-effective manner, without seamless, sustainable, and secure data and information exchanges at all levels of the health system” (Haazen 2015). The report further specifies that countries should have comprehensive databases and electronic tracking systems that support the delivery of quality health services and allow providers to follow up with clients over a period of time, whether within the facility or in the community. An effective healthcare system should have the capacity to store, share, and use health data, so that providers can use information to improve not just clinical care, but also treatment co-ordination and disease management. Better data on health status are critical for addressing various coverage and quality of care bottlenecks with the goal of achieving universal healthcare coverage (Froen 2016).

Such functional health information systems, when available, can provide additional opportunities for improving the quality and safety of preventative and clinical healthcare services. Too often, providers lack information on which treatments work best for which clients - whether this void is due to limited training, insufficient time to keep abreast of current evidence, or the sheer complexity of the disease. In primary health care, despite the availability of knowledge and of specific diagnostic, treatment, and management protocols, a discrepancy exists between knowledge and its application. This ‘know-do’ gap in the quality of health care has been widely cited as a key barrier to optimal healthcare outcomes (Blank 2013; Mohanan 2015).

For clients receiving primary healthcare services, interactions with providers during healthcare appointments remain key opportunities to receive evidence-based clinical information customised to the individual's specific health needs. Limited interactions with the healthcare system between clinical appointments often result in poor engagement in care, for example, poor adherence to a recommended clinical care plan or medication regimen. In low-resource settings, a shortage of trained healthcare professionals combined with a low provider-to-patient ratio often translates into additional barriers for clients who wish to receive counselling or information that would help them manage their health or learn about available preventative health measures.

Description of the intervention

Several obstacles to maintaining robust client health records, improving the clinical practices of providers, and engaging with clients may be addressed by digital interventions using mobile devices. Digital tracking is characterised by longitudinal follow-up of a client's health status conducted by digitally entering and accessing data on healthcare services utilised by the client. Digital tracking of a client's health information and service utilisation may involve digitisation of traditionally paper health records (such as medical registers and logbooks of services with a longitudinal care plan for antenatal and postnatal care, immunisation or diabetes). By using such digitised records, providers can uniquely identify each client and can digitally follow client interactions with the healthcare system/services for health concerns that require continuity of care (e.g. pregnancy, diabetes).

Systems designed to track and manage preventative and clinical care will allow providers to capture, store, access, and share client information during the care encounter. Tracking of health status enables individual care planning, close management, and potentially prevention of episodes of acute illness and declining health status (WHO/ITU 2012). Additionally, tracking may facilitate aggregation, analysis, and synthesis of client data from multiple sources to enable better decisions for client care and more accurate study of disease and service utilisation trends.

In several high-income countries (HICs), sophisticated computerised health records have been widely implemented over the past decade. In low- and middle-income countries (LMICs), where computerised infrastructure may be limited, mobile devices are widely used instead of stationary computers to leapfrog the lack of infrastructure while digitising client health records.

Such digital client health records may be integrated with other tools that provide information to healthcare providers and to clients through their mobile devices. For example, mobile clinical decision support systems (CDSSs) may couple clinical practice guidelines with an accurate base of client information from the digital health record to identify drug interactions, client risks, and appropriate management of disease. A CDSS is “any electronic system designed to aid directly in clinical decision making, in which characteristics of individual clients are used to generate client-specific assessments or recommendations that are presented to clinicians for considerations” (Kawamoto 2005). Mobile CDSSs may vary in range of functionality and applications to improve diagnosis, while facilitating evidence-based screening, counselling, and treatment, and improving workflow efficiencies.

On the client side, digital health records may be integrated with systems designed to improve client adherence to the clinical care plan via targeted communication with clients on their mobile devices. Targeted client communication in this context is defined as transmission of targeted health content to a specified population or to individuals within a predefined health or demographic group. This transmitted information can fall along a continuum of tailored to untailored communication, to include transmission of individualised notifications according to a specific individual clinical care plan, as well as transmission of predetermined content developed for the identified population group (Hawkins 2008). Examples of targeted client communication (TCC) may include appointment reminders, medication reminders, notification of test results, or information on specific health conditions that pertains to the client. To define appropriate populations for TCC, providers need to identify and subscribe eligible individuals into a system that allows transmission of health information. Additionally, the health system must determine the timing and content of transmitted information rather than have clients seeking information on-demand.

How the intervention might work

Longitudinal tracking of client health information varies widely depending on the availability of health infrastructure. In HICs, providers may track client health information using sophisticated electronic medical records. It is expected that such records will conform to nationally recognised interoperability standards, can draw from multiple sources, support a range of disease groups and comorbidities, and can be shared and controlled by healthcare providers and clients (Kahn 2009). These records may be integrated with CDSSs relevant to a specialty or to a high-priority hospital condition (Moja 2014) and may offer additional features that support education and automated follow-up with clients according to their specific health condition.

In most LMICs, such systems tend to vary widely according to available infrastructure and human resources, as well as disease groups. For example, in peri-urban and urban centres that include an extant computerised infrastructure, digital health records customised to the workflows of individual clinics and hospitals may be available. In most primary healthcare settings, the transition from paper records to digital records is still in its infancy. Interoperability standards are not well defined or used in practice, precluding establishing and sharing of digital health records. Longitudinal digital records used to support reproductive, maternal, newborn, and child health, while following care provided to women and infants from prenatal to postpartum stages, are often referred to as electronic registries, or eRegistries (Froen 2016). Like electronic medical records, eRegistries provide an organised system for collecting, managing, and analysing data by using reproductive, maternal, newborn, and child health data to improve women's and children’s health, and serve as an entry point for accessing a range of preventative and curative services and promoting health.

Digital health records may drive multiple mobile applications that can draw on stored client data to guide a provider through next steps (e.g. CDSSs) or to offer additional support to the client in the form of information and reminders (e.g. as TCC). Providers may load electronic databases onto a handheld mobile device to manage diaries, input new client data that later can be synchronised with an external source (Divall 2013), access summary dashboards, and retrieve information and decision support aids based on client-specific information. Broadly, CDSSs may serve the following functions.

  • Guide the healthcare provider through process algorithms using 'if..then' rules based on evidence-based clinical protocols.

  • Provide the healthcare provider with a checklist based on clinical evidence-based protocols.

  • Present step-by-step guidance for screening clients by health status or risk status, possibly using models based on machine learning whereby mathematical functions might be used to classify clients into risk groups.

Providers may use the data stored in digital health records to generate TCC delivered to the client’s mobile device to reach clients near-instantaneously with information that is specifically tailored to their needs. A variety of communication modalities such as text messages, phone calls, interactive voice response systems, unstructured supplementary service data (USSD) messages, multimedia messages, pictures, and in-app alerts might be used in communicating with clients. Such targeted communication with clients may serve several functions.

  • Targeted health event alerts to clients may alert clients about diagnostics results or availability of results. Efficient delivery of diagnostic results may expedite follow-up, appropriate treatment, and ongoing engagement with care.

  • Targeted health content based on care plan or demographic health status may be used to improve clients’ knowledge about their health condition, and may positively influence their behaviour and healthcare practices.

  • Targeted alerts and reminders may be sent to clients to improve adherence to medication regimens and attendance at clinic appointments.

Interventions that integrate digital health records with CDSSs or TCCs are multi-faceted and may vary widely. For example, the mTika immunisation registry system supports vaccinators in registering and tracking children for receipt of recommended childhood vaccinations during the first year of life. Caregivers of children receive appointment reminders via short message service (SMS) to their registered phone number one week before and one day before the scheduled vaccination appointment. In addition to tracking and sending targeted messages to caregivers, mTika offers CDSS components such as vaccination schedules calculated on the basis of the child's date of birth. Another example is the Safer Deliveries programme in Zanzibar (Uddin 2016). This programme helps community health workers (CHWs) register pregnant women via mobile phones. CHWs then visit these women in their homes and offer antenatal care via mobile-based CDSSs throughout pregnancy while preparing women to deliver at a health facility (Dtree International 2017).

Why it is important to do this review

Digital, mobile, and wireless technologies provide an innovative and accessible platform for accelerating health services and improving quality of care. Health records available on mobile devices can facilitate delivery of critical care outside formal healthcare settings within the community, as well as in areas where healthcare facilities may be lacking critical infrastructure. Some evidence shows how such digital health records, when combined with additional tools such as CDSSs, may improve the quality of health services and healthcare outcomes.

A systematic review of 148 randomised trials on effects of CDSS reported significant improvements in healthcare process measures related to preventative services, diagnostic testing, and treatment (Bright 2012). Another systematic review assessed the effectiveness of CDSSs that are integrated with electronic health records. This review of 25 randomised trials concluded that integrated CDSSs did not affect mortality but significantly prevented disease morbidity (Moja 2014). Although this evidence is promising, it does not speak to the effectiveness of interventions provided via mobile devices, especially in low-income settings, where such interventions are in early stages of development.

Several systematic reviews have examined use of TCCs for improving health knowledge, behaviour, and outcomes (Free 2013; Gurol-Urganci 2013; Vodopivec-Jamsek 2012; Whittaker 2016). Much of the available evidence is of low or moderate quality and includes limited studies from low-resource settings that are not conducted in primary healthcare settings. A systematic review of eight randomised trials assessed effects of mobile phone messaging reminders on improving attendance at healthcare appointments. This review concluded that mobile phone text messaging reminders, similar to phone call reminders, were associated with increased attendance at healthcare appointments (Gurol-Urganci 2013). Whittaker 2016 included 12 studies of moderate to high quality and found beneficial impact of mobile phone-based smoking cessation interventions on six-month cessation outcomes. Another systematic review of 75 controlled trials on the effectiveness of mobile technology-based health behaviour change interventions and disease management interventions delivered to healthcare consumers suggested benefits of text messaging for improving antiretroviral treatment adherence and smoking cessation (Free 2013). However, this review found limited evidence from LMICs and noted an overall lack of high-quality trials. None of these reviews focussed on interventions that integrate TCCs with longitudinal digital health records.

The present review assesses evidence on tracking systems accessible via mobile devices integrated with CDSSs and/or TCCs. Given the recent emergence of digital technologies for health, ministries of health, donors, and decision-makers have demanded evidence-based guidance on the value of digital tools in strengthening health system gaps. In response to this global need, the World Health Organization is developing guidelines to inform investments on digital health approaches. This review on digital interventions that use combinations of CDSSs, TCCs, and electronic health records constitutes one of a suite of such reviews. Results of this review will be used to directly inform WHO guidelines on the effectiveness of these strategies in addressing health system shortfalls.


In the context of primary healthcare settings, we aim to assess:

  • effects of digitally tracking clients' health service use and status combined with decision support conducted via mobile device;

  • effects of digitally tracking clients’ health service use and status combined with TCCs accessible via mobile device; and

  • effects of digitally tracking clients' health service use and status combined with decision support conducted via mobile device and TCCs accessible via mobile device.


Criteria for considering studies for this review

Types of studies

We will include randomised trials along with trials using the following types of non-randomised study design.

  • Non-randomised trials, where assignment is not random.

  • Controlled before-after studies, provided they include at least two intervention sites and two control sites.

  • Interrupted time series studies, if the intervention occurred at a clearly defined point in time and investigators documented at least three data points before and at least three data points after the intervention.

We will include published studies, conference abstracts, and unpublished data. We will include studies irrespective of their publication status and language of publication.

Types of participants

  • All cadres of healthcare providers (i.e. professionals, paraprofessionals, and lay health workers) who are involved in providing healthcare services to patients in a primary healthcare setting

  • Other individuals or groups involved in client registration or tracking. These individuals or groups could include administrative staff and managerial and supervisory staff. Participants may be based in a primary healthcare facility or in the community but must be involved in supporting delivery of primary healthcare services

  • Patients/clients receiving primary healthcare services

By “primary healthcare services”, we mean a combination of the following.

  • First contact point of health care (Awofeso 2004), including care delivered at an individual or community level, or both (Muldoon 2006), by non-specialist healthcare providers, and intended to bring care to where people work and live (Muldoon 2006), as well as to co-ordinate or provide continuity of care (WHO 2008).

  • Any rehabilitative, therapeutic, preventative, and promotive health care (Global Health Watch 2011).

Types of interventions

For the purposes of this systematic review, we will include multi-faceted interventions that aim to improve the quality and efficiency of delivering healthcare services. These multi-faceted interventions comprise a digital system that allows providers to longitudinally follow up on clients by entering and accessing data on healthcare services utilised by the client (i.e. digital tracking of clients) combined with other interventions aimed at improving either client adherence to treatment or quality of services received by the client. To clarify the relationship between interventions and outcomes, we present a logic model (Figure 1).

Figure 1.

Figure 1: Logic model outlining the three interventions and associated outcomes.

We will include studies in which the intervention involves digital tracking of clients combined with digital decision support via mobile device; or digital tracking of clients combined with targeted client communications via mobile device; or digital tracking of clients combined with both of these interventions, in the context of a primary healthcare setting.

In each case, the provider should be able to enter client data using a mobile interface and/or should be able to access individual client data using a mobile device. In contexts with more developed infrastructure, it may be preferable for providers to enter data on desktop computers. We will include studies that test interventions by using client information as entered via desktops, so long as the record is accessible on a mobile interface. Interventions may include systems by which clinical decision support tools are available on mobile devices and are used to enter relevant client data via these mobile devices. Similarly, the system may be set up in such a way that targeted messages are sent to clients' mobile devices. For studies included in this review, in which investigators used digital devices to provide maternity care from the prenatal to the postpartum period and included the function of birth notification, we will include only prenatal outcomes.

By 'mobile devices', we mean mobile phones of any kind (but not analogue landline telephones), as well as tablets, personal digital assistants, and smartphones.

As shown in Figure 1, comparisons for this review will include standard practice and interventions that included completed non-digital systems.

We will exclude the following.

  • Studies that perform digital tracking of client healthcare services on laptop alone, or in which the client health record is not accessible via a mobile device.

  • Studies that provide decision support on stationary computers or laptops alone.

  • Pilot and feasibility studies (pilot study defined as “a version of the main study that is run in miniature to test whether the components of the main study can all work together”, and feasibility study defined as “pieces of research done before a main study” (Arain 2010)).

  • Studies in which digital decision support or targeted client communication is provided as a standalone intervention or is not linked to client health record.

  • Studies in which untargeted client communication is provided.

  • Studies that include interventions targeted at notification of new births that measure only postnatal outcomes attributable to timely birth notification.

  • Studies that do not include a direct face-to-face provider-client service delivery interaction.

Types of outcome measures

InterventionOutcomes specific to interventionsOutcomes common across all three interventions
Tracking + CDSS
  • Providers’ adherence to recommended practice (e.g. providing service at the recommended time; referral as recommended; screening and prioritising as recommended)

  • Time between presentation and appropriate management

  • Quality of data about services provided (accuracy, timeliness, completeness of data)

  • Clients' utilisation of primary health care and/or services health status

  • Provider acceptability of/satisfaction with the intervention, assessed

  • Patient/client acceptability of, or satisfaction with, the intervention

  • Resource use (e.g. human resources/time, including additional time spent by providers when managing/transitioning dual paper and digital reporting systems; training, supplies, and equipment)

  • Unintended consequences that result in adverse effects of the intervention (these could include providers' time spent on administrative tasks; misinterpretation of data; transmission of inaccurate data, for instance, through incorrect data entries; loss of verbal or non-verbal communication cues; issues of privacy and disclosure; failure of, or delay in, message delivery; loss or misuse of devices; interrupted workflow due to infrastructural constraints for battery recharge and network coverage; impact on equity)

Tracking + TCC
  • Clients’ timeliness of receiving and accessing healthcare services and information (e.g. partner notification, receipt of test results)

Tracking + CDSS + TCC
  • Providers’ adherence to recommended practice, guidelines, or protocols (e.g. providing service at the recommended time; referral as recommended; screening and prioritising as recommended)

  • Time between presentation and appropriate management

  • Clients’ timeliness of receiving and accessing healthcare services and information (e.g. partner notification, receipt of test results)

CDSS: clinical decision support system.

TCC: targeted client communication.

Search methods for identification of studies

We will restrict the search from 2000 to the date the search is conducted. This is based on increased availability and penetration of mobile devices in low- and middle-income countries starting in 2000 (International Telecommunications Union 2015).

Electronic searches

An independent Information Specialist (JE) will develop search strategies in consultation with the review authors.

We will search the following databases for primary studies, from 2000 to date of search.

  • Cochrane Central Register of Controlled Trials (CENTRAL), in the Cochrane Library.

  • MEDLINE, Ovid.

  • Embase, Ovid.

  • Population Information Online (POPLINE), K4Health.

  • Global Health Library, WHO.

Appendix 1 lists the search strategy for MEDLINE Ovid. Search strategies comprise keywords and controlled vocabulary terms. We will not apply any limits on language.

Searching other resources

We will search for ongoing trials in the following trial registries and will contact study authors to request further information and data, if available.

  • WHO ICTRP (World Health Organization International Clinical Trials Registry Platform;

  • US National Institutes of Health Ongoing Trials Register (

We will search Epistemonikos ( for relevant systematic reviews and potentially eligible primary studies. Additionally, the WHO will issue a call for papers through popular digital health communities of practice such as the Global Digital Health Network, as well as grey literature, to identify additional primary studies.

Grey literature

In addition to the above databases, we will search for grey literature. The search portal for is limited; therefore we will review titles and abstracts of all contributed literature that is not referenced in MEDLINE Ovid.

We will review reference lists of all included studies and relevant systematic reviews for additional potentially eligible primary studies. We will contact authors of included studies/reviews to clarify reported published information and to seek unpublished results/data.

Data collection and analysis

Selection of studies

A core team of two review authors (NH, HB), with assistance when necessary from one additional review author (SA), will be responsible for selection of studies. We will download all titles and abstracts retrieved by electronic searching to a reference management database and will remove duplicates. Two review authors (NH, HB) will independently screen titles and abstracts for inclusion. We will retrieve full-text study reports/publications of all relevant studies, and two review authors (NH, HB) will independently screen these for inclusion. We will record all reasons for exclusion of ineligible studies. We will resolve disagreements through discussion, or, if required, we will consult a third review author (SA).

We will list in the 'Characteristics of excluded studies' table studies that initially appeared to meet the inclusion criteria but that we later excluded. We will collate multiple reports of the same study, so that each study rather than each report is the unit of interest in the review. We will report in an 'Ongoing studies table' all information that we can obtain about ongoing studies. We will record the study selection process in sufficient detail to complete a PRISMA flow diagram (Liberati 2009).

Data extraction and management

We will modify the EPOC standard data extraction form and will adapt it for study characteristics and outcome data relevant to this review (EPOC 2017a). We will identify key characteristics of the intervention for abstraction based on mHealth Evidence Review and Assessment (mERA) guidelines (Agarwal 2016). We will pilot the form on at least one study in the review. Two review authors (NH, HB) will independently extract the following study characteristics from the included studies.

  • General information: title, reference details, study author contact details, publication type, funding source, conflicts of interest of study authors.

  • Population and setting: country, geographical location (rural, urban, peri-urban), healthcare setting (e.g. facility-based, community-based).

  • Methods: function of the intervention, study design, unit of allocation, duration of the study.

  • Participant characteristics: type of healthcare worker (function, age, length of training), description of clients serviced by the healthcare worker, description of any other participants in the intervention, withdrawals.

  • Interventions.

    • For tracking and CDSS interventions: intervention purpose; components of tracking interventions and CDSS interventions; type of technology and mode of delivery; content of the intervention; type of mobile device/s used (smartphone, tablet, feature phone, basic phone, laptops); healthcare provider training; interoperability; compliance with national guidelines; data security; comparison; fidelity assessment; duration of the intervention.

    • For tracking and TCC interventions: intervention purpose; intervention components' mode, timing, frequency, and duration of intervention delivery; content of the intervention; type of mobile device/s used (smartphone, tablet, feature phone, basic phone, laptops); interoperability; compliance with national guidelines; data security; comparison; fidelity assessment.

    • For tracking, CDSS, and TCC review: We will extract all characteristics listed in the sub-bullets above.

  • Outcomes: primary and other outcomes (continuous and dichotomous) for each intervention group, time points reported, adverse events, results of any subgroup analyses.

We will note in the 'Characteristics of included studies' table if outcome data were reported in a unusable way. We will resolve disagreements by reaching consensus or by involving a third review author (SA).

Assessment of risk of bias in included studies

Two review authors (NH, HB) will independently assess risk of bias for each study by using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions Section 8.5 (Higgins 2017) and guidance from the EPOC group (EPOC 2017b). We will resolve any disagreements by discussion or by consultation with a third review author (SA). We will assess risk of bias for randomised/non-randomised trials and controlled before-after studies using the following criteria: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, baseline outcomes measurement, similarity of baseline characteristics, and other bias. For interrupted time series studies, we will assess risk of bias using these seven standard criteria (EPOC 2017b): independence of the intervention from other changes, prespecified shape of the intervention effect, independence of intervention and data collection procedures, objectivity of the outcomes, bias resulting from missing outcome measures, selective outcome reporting, and other biases such as seasonality.

We will judge risk of each potential source of bias as high, low, or unclear, and will provide a quote from the study report together with a justification for our judgement in the 'Risk of bias' table. We will summarise 'Risk of bias' judgements across different studies for each of the domains listed. We will consider blinding separately for different key outcomes when necessary (e.g. for unblinded outcome assessment, risk of bias for all-cause mortality may be very different than for a patient-reported pain scale). When information on risk of bias relates to unpublished data or correspondence with a trialist, we will note this in the 'Risk of bias' table. We will not exclude studies on the grounds of their risk of bias, but will clearly report risk of bias when presenting results of studies. When considering treatment effects, we will take into account risk of bias for studies that contribute to that outcome. We will further perform assessment of evidence quality using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach (Guyatt 2008). We will summarise findings in ‘Summary of findings’ tables.

We will conduct the review according to this published protocol and will report deviations from it in the 'Differences between protocol and review' section of the systematic review.

Measures of treatment effect

We will report preintervention and postintervention means and proportions for intervention and comparison groups. When possible, we will estimate effects of the intervention using risk ratios/risk differences for dichotomous data, together with appropriate associated 95% confidence intervals, and mean differences or standardised mean differences for continuous data, together with 95% appropriate associated confidence intervals. We will ensure that an increase in scores for continuous outcomes can be interpreted in the same way for each outcome, will explain the direction to the reader, and will report when directions were reversed if this was necessary. For interrupted time series, we will report preintervention and postintervention slopes of analysis, differences between slopes, and differences between intercepts at the first intervention point and predicted intercept by intervention. If interrupted time series data have been analysed incorrectly, we will reanalyse the data when possible (EPOC 2017c).

Unit of analysis issues

If cluster-randomised trials or controlled before-after studies are included in the review, we will report cluster-adjusted risk ratios or differences and their 95% confidence intervals. If the analysis was not adjusted for clustering, we will use the intracluster correlation coefficient (ICC), if available, to adjust the confidence interval. If the ICC is not available, we will present results without a measure of variance or precision of effect for outcomes for which we find a unit of analysis error (EPOC 2017d).

Dealing with missing data

We will contact investigators to verify key study characteristics and to obtain missing outcome data when possible (e.g. when a study is identified as abstract only). If this is not possible, we will report the data as missing, will note this in the 'Risk of bias' tables, and will not attempt to impute missing values. For all outcomes, we will carry out analyses, to the extent possible, on an intention-to-treat basis using available cases. However, in assessing adverse events, we will relate results to actual treatment received (i.e. analyses will be based on participants who actually received the intervention and adverse events reported in the studies).

Assessment of heterogeneity

If we find a sufficient number of studies that evaluate similar interventions and report similar outcomes, we will conduct a meta-analysis. We will examine heterogeneity by visually inspecting forest plots and by using the I² statistic to measure heterogeneity among the trials in each analysis (Higgins 2017). If we identify substantial heterogeneity, we will explore this by performing prespecified subgroup analysis.

Assessment of reporting biases

We will attempt to contact study authors to ask them to provide missing outcome data. When this is not possible and missing data are thought to introduce serious bias, we will explore the impact of including such studies in the overall assessment of results. In cases where adjusted analyses for dichotomous outcomes are reported using odds ratios and not risk ratios, we will use Cochrane statistical software (Review Manager 2014) to convert odds ratios to risk ratios before including results in a meta-analysis. If we are able to pool more than 10 trials, we will create funnel plots to explore possible publication bias or other causes of asymmetry (Sterne 2011). We will interpret results of the funnel plot with caution, as funnel plot calculations for dichotomous outcomes measured as risk ratios are not well developed, and statistical funnel plot results may not be representative in cases of small-study effects.

Data synthesis

When studies evaluate interventions that are similar, we will group them together and will summarise key characteristics of each study in tables, to facilitate comparison across studies. We will undertake meta-analyses using a random-effects model only when this is meaningful (i.e. when intervention, context, and outcomes are similar enough for pooling to make sense). Trialists commonly indicate when they have skewed data by reporting medians and interquartile ranges. When we encounter this, we will note that the data are skewed and will consider the implications of this. When a single trial reports multiple trial arms, we will include only the relevant arms. When we must enter two comparisons (e.g. intervention A vs usual care and intervention B vs usual care) into the same meta-analysis, we will halve the control group to avoid double-counting.

'Summary of findings' table and GRADE

We will create a 'Summary of findings' table for the main intervention comparisons and will include the most important outcomes to permit conclusions about certainty of evidence provided within the text of the review. We will include the following outcomes in the 'Summary of findings' table.

  • Providers' adherence to recommended practice.

  • Clients' utilisation of primary healthcare services, including adherence to recommended treatments.

  • Clients' timeliness of receiving and accessing healthcare services.

  • Quality of data on services provided.

  • Clients' health status.

If during the review process, we become aware of an important outcome that we failed to list in our planned 'Summary of findings' table, we will include the relevant outcome and will explain the reasons for it in the section titled 'Differences between protocol and review'.

Two review authors will independently assess the certainty of evidence (high, moderate, low, and very low) using the five GRADE considerations (risk of bias, consistency of effect, imprecision, indirectness, and publication bias) (Guyatt 2008). We will use methods and recommendations as described in Section 8.5 and Chapter 12 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2017), as well as EPOC worksheets (EPOC 2017e), along with GRADEpro software (GRADEpro GDT 2014). We will resolve disagreements on certainty ratings by discussion, will provide justification for decisions to downgrade or upgrade the ratings using footnotes in the table, and will make comments to aid readers' understanding of the review when necessary. We will use plain language statements to report these findings in the review (EPOC 2017f).

We will consider whether we are unable to incorporate any additional outcome information into meta-analyses; we will note this in the comments and will state if this supports or contradicts information from the meta-analyses. If it is not possible to meta-analyse the data, we will summarise in the text results of analysis.

Subgroup analysis and investigation of heterogeneity

We will perform subgroup analysis to assess variation in delivery of the intervention across different population groups, interventions, or setting characteristics, if possible. We will conduct subgroup analysis only if a sufficient number of trials are available for statistically significant comparisons between groups. We will assess heterogeneity within each subgroup by using forest plots and the I² measure. We will summarise results of subgroup analysis within the text of the review, if meta-analysis is not possible or meaningful. We plan to carry out subgroup analyses by each of the following categories, depending on availability of data.

Tracking + CDSSTracking + TCCTracking + CDSS + TCC
  • Intervention characteristics (e.g. smartphones vs tablets)

  • Functional characteristics, such as use of decision support as checklists, as screening tools, and for risk stratification and treatment, as effects of decision support tools may be different depending on the purpose for which they are used

  • Intervention characteristics (e.g. communication modalities including SMS, IVR, voice, or multimedia)

  • Intervention purpose (e.g. appointment reminders, medication reminders, health information)

  • Intervention timing or frequency

  • Intervention characteristics (e.g. smartphones vs tablets)

  • Functional characteristics, such as use of decision support as checklists, as screening tools, and for risk stratification and treatment, as effects of decision support tools may be different depending on the purpose for which they are used

  • Intervention characteristics (e.g. communication modalities including SMS, IVR, voice, or multimedia)

  • Intervention purpose (e.g. appointment reminders, medication reminders, health information)

  • Intervention timing or frequency

  • Type of geographic setting (e.g. urban, rural, peri-urban; low- and middle-income countries), as we anticipate that the intervention may have different effects owing to social and economic differences between settings

  • Type of healthcare setting (e.g. community-based, clinic-based), which may influence the way the intervention is delivered and its effects

  • Provider type (e.g. lay provider vs professional healthcare provider), as we anticipate that different types of providers may vary in their use of decision support tools provided via a mobile device

  • Health area (e.g. chronic disease, infectious disease, maternal health, child health), as effects of using decision support tools may vary by types of services used to address needs in different health areas

CDSS: clinical decision support system.

IVR: interactive voice response.

SMS: short message service.

TCC: targeted client communication.

Sensitivity analysis

We will perform sensitivity analyses as defined a priori to assess the robustness of our conclusions and will explore their impact on effect sizes. This will involve restricting analysis to published studies and removing from meta-analysis studies that have high risk of bias as determined by risk of bias assessment.


We acknowledge the help and support of Cochrane Effective Practice and Organisation of Care (EPOC). We would like to thank the following editors and peer referees who provided comments to improve the protocol: Elizabeth Paulsen, Michel Wensing, Marit Johansen, and Marco Bardus. We would also like to thank John Eyers for designing the search strategies and Dolores Matthews for copy-editing the protocol.

The Norwegian Satellite of Effective Practice and Organisation of Care (EPOC) receives funding from the Norwegian Agency for Development Co-operation (Norad), via the Norwegian Institute of Public Health, to support review authors in production of their reviews.

We are grateful to the Guideline Development Group of Digital Health Guidelines for constructive feedback in formulating the guiding questions for this systematic review.


Appendix 1. MEDLINE search strategy

Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations, Ovid MEDLINE(R) Daily, and Ovid MEDLINE(R) <1946 to Present> - Searched 24 July 2017

1 exp Health Personnel/ (453813)

2 Volunteers/ (8883)

3 "Delivery of Health Care"/ma or "Delivery of Health Care, Integrated"/ma (1059)

4 (((health or medical or healthcare or frontline or front-line) adj (personnel or worker* or auxiliar* or staff or professional* or assistant* or provider*)) or doctor* or physician* or GP or general practitioner? or family doctor or nurse* or midwi* or clinical officer* or pharmacist* or dentist* or vaccinator* or supervisor* or ((birth or childbirth or labor or labour) adj (attendant? or assistant?))).ti,ab,kw. (887792)

5 (((lay or voluntary or volunteer? or untrained or unlicensed or nonprofessional? or non professional? or support or link or outreach or out reach) adj5 (worker? or visitor? or attendant? or aide or aides or support$ or person$ or helper? or carer? or caregiver? or care giver? or consultant? or assistant? or staff)) or promotores or promotora or promotoras or embajadoras or comodrones or abuela or "lay advocate*" or "lay health" or "lay advisors" or "lay educators" or "lay counselor*" or "lay counsellor*" or "lady health worker*" or "lay facilitator*" or "natural helpers" or linkworker? or monitrice* or consejeras).ti,ab,kw. (816640)

6 (paraprofessional? or paramedic or paramedics or paramedical worker? or paramedical personnel or allied health personnel or allied health worker? or support worker? or home health aide?).ti,ab,kw. (7531)

7 ((community or village? or peer or indigenous or treatment) adj3 (health worker? or health care worker? or healthcare worker? or health advisor* or volunteer* or educator* or facilitator* or distributor* or extension worker* or supporter* or counselor* or counsellor*)).ti,ab,kw. (9092)

8 (doula? or douladural? or barefoot doctor?).ti,ab,kw. (410)

9 or/1-8 (1870556)

10 Medical Records/ (65183)

11 Registries/ (71806)

12 Electronic Health Records/ (13270)

13 Health Records, Personal/ (1222)

14 Health Smart Cards/ (36)

15 Medical Order Entry Systems/ (1991)

16 Medical Records Systems, Computerized/ (19239)

17 "Forms and Records Control"/ or Clinical Coding/ or Medical Record Linkage/ (13887)

18 Health Information Management/ or Health Information Exchange/ (1070)

19 Management Information Systems/ or Decision Support Systems, Management/ (4496)

20 Decision Support Systems, Clinical/ or Decision Making/ or Decision Support Techniques/ or Checklist/ (109170)

21 Health Information Systems/ (887)

22 Ambulatory Care Information Systems/ (1189)

23 Hospital Information Systems/ (10858)

24 (register or registers or registration or registry or registries or ((tracking or monitor*) adj2 (client* or patient*)) or ((digital or digiti?ed or electronic or computeri?ed or medical or clinical or patient* or health) adj2 (record or records)) or health smart card* or HMIS or ((information or decision* or case*) adj (management or system* or support))).ti,ab,kw. (410716)

25 ((decision* adj3 (make or makes or making or made or support* or algorithm* or aid or aids or app or apps or application* or technique*)) or checklist* or expert system* or job-aid* or "job aid*").ti,ab,kw. (173541)

26 (district health information or DHIS2 or Motech or Mobile Technology for Health or service record or logbook or log book or eregistr* or e-registr* or (vital event* adj3 track*)).ti,ab,kw. (829)

27 or/10-26 (735630)

28 Cell Phones/ (7071)

29 Smartphone/ (1230)

30 MP3-Player/ (167)

31 Computers, Handheld/ (3086)

32 ((cell* or mobile*) adj1 (phone* or telephone* or technolog* or device*)).ti,ab,kw. (13075)

33 (handheld or hand-held).ti,ab,kw. (9937)

34 (smartphone* or smart-phone* or cellphone* or mobiles).ti,ab,kw. (5502)

35 ((personal adj1 digital) or (PDA adj3 (device* or assistant*)) or MP3 player* or MP4 player*).ti,ab,kw. (1294)

36 (samsung or nokia).ti,ab,kw. (816)

37 (windows adj3 (mobile* or phone*)).ti,ab,kw. (42)

38 android.ti,ab,kw. (1527)

39 (ipad* or i-pad* or ipod* or i-pod* or iphone* or i-phone*).ti,ab,kw. (1958)

40 (tablet* adj3 (device* or computer*)).ti,ab,kw. (994)

41 Text Messaging/ (1653)

42 (mhealth or m-health or "mobile health" or ehealth or e-health or "electronic health" or "digital health" or uhealth or u-health).ti,ab,kw. (15659)

43 (((text* or short or voice or multimedia or multi-media or electronic or instant) adj1 messag*) or instant messenger).ti,ab,kw. (3415)

44 (texting or texted or texter* or ((sms or mms) adj (service* or messag*)) or interactive voice response* or IVR or voice call* or callback* or voice over internet or VOIP).ti,ab,kw. (2539)

45 (Facebook or Twitter or Whatsapp* or Skyp* or YouTube or "You Tube" or Google Hangout*).ti,ab,kw. (4097)

46 Mobile Applications/ (2226)

47 "mobile app*".ti,ab,kw. (1718)

48 Reminder Systems/ (3053)

49 (remind* adj3 (text* or system* or messag*)).ti,ab,kw. (1401)

50 Medical informatics/ or Medical informatics applications/ (12869)

51 Nursing informatics/ or Public health informatics/ (2461)

52 ((medical or clinical or health or healthcare or nurs*) adj3 informatics).ti,ab,kw. (5001)

53 Computer-Assisted Instruction/ (11261)

54 ((interactive or computer-assisted) adj1 (tutor* or technolog* or learn* or instruct* or software or communication)).ti,ab,kw. (2213)

55 or/28-54 (86533)

56 randomised controlled (470060)

57 controlled clinical (94456)

58 multicenter (232984)

59 pragmatic clinical (620)

60 (randomis* or randomiz* or randomly).ti,ab. (749068)

61 groups.ab. (1724247)

62 (trial or multicenter or multi center or multicentre or multi centre).ti. (212388)

63 (intervention? or effect? or impact? or controlled or control group? or (before adj5 after) or (pre adj5 post) or ((pretest or pre test) and (posttest or post test)) or quasiexperiment* or quasi experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or time series or time point? or time trend? or repeated measur*).ti,ab. (8115377)

64 non-randomised controlled trials as topic/ (181)

65 interrupted time series analysis/ (313)

66 controlled before-after studies/ (264)

67 or/56-66 (9069279)

68 exp Animals/ (21500389)

69 Humans/ (17057280)

70 68 not (68 and 69) (4443109)

71 (2320969)

72 meta (83223)

73 (184299)

74 (697322)

75 (443138)

76 cochrane database of systematic reviews.jn. (13397)

77 comment (697322)

78 (systematic review or literature review).ti. (96184)

79 or/70-78 (7781506)

80 67 not 79 (6297340)

81 9 and 27 and 55 and 80 (5268)

82 limit 81 to yr="2000 -Current" (5030)

Contributions of authors

Conceiving the protocol: CG, TT, SL, SA, GM, MS.

Designing the protocol: CG, TT, SL, SA, LV, NM, NH, MS, GM.

Co-ordinating the protocol: SA, TT, CG, SL.

Writing the protocol: SA, TT, CG, LV.

Providing general advice on the protocol: CG, SL, MS TT, GM.

Securing funding for the protocol: GM.

Declarations of interest

Smisha Agarwal: The author was commissioned by the WHO to conduct this review.

Lavanya Vasudevan: none known.

Tigest Tamrat: none known.

Claire Glenton: none known.

Simon Lewin: The author is the joint co-ordinating editor for the Cochrane Effective Practice and Organisation of Care Review Group.

Hanna Bergman: Since May 2016, the author has been employed by Cochrane Response, an evidence services unit operated by the Cochrane Collaboration. Cochrane Response was contracted by the WHO to produce this review.

Nicholas Henschke: Since June 2016, the author has been employed by Cochrane Response, an evidence services unit operated by the Cochrane Collaboration. Cochrane Response was contracted by the WHO to produce this review.

Garrett L Mehl: owns stock in Apple Computer.

Marita Fønhus: none known.

Sources of support

Internal sources

  • No sources of support supplied

External sources

  • UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), a cosponsored program executed by the World Health Organization (WHO), Switzerland.

    Provided funding for the review


This protocol is based on standard text and guidance provided by Cochrane Effective Practice and Organisation of Care.