Types of studies

We will include randomised trials along with trials using the following types of non‐randomised study design.

• Non‐randomised trials, where assignment is not random.

• Controlled before‐after studies, provided they include at least two intervention sites and two control sites.

• Interrupted time series studies, if the intervention occurred at a clearly defined point in time and investigators documented at least three data points before and at least three data points after the intervention.

We will include published studies, conference abstracts, and unpublished data. We will include studies irrespective of their publication status and language of publication.

Types of participants

• All cadres of healthcare providers (i.e. professionals, paraprofessionals, and lay health workers) who are involved in providing healthcare services to patients in a primary healthcare setting

• Other individuals or groups involved in client registration or tracking. These individuals or groups could include administrative staff and managerial and supervisory staff. Participants may be based in a primary healthcare facility or in the community but must be involved in supporting delivery of primary healthcare services

• Patients/clients receiving primary healthcare services

By “primary healthcare services”, we mean a combination of the following.

• First contact point of health care (Awofeso 2004), including care delivered at an individual or community level, or both (Muldoon 2006), by non‐specialist healthcare providers, and intended to bring care to where people work and live (Muldoon 2006), as well as to co‐ordinate or provide continuity of care (WHO 2008).

• Any rehabilitative, therapeutic, preventative, and promotive health care (Global Health Watch 2011).

Types of interventions

For the purposes of this systematic review, we will include multi‐faceted interventions that aim to improve the quality and efficiency of delivering healthcare services. These multi‐faceted interventions comprise a digital system that allows providers to longitudinally follow up on clients by entering and accessing data on healthcare services utilised by the client (i.e. digital tracking of clients) combined with other interventions aimed at improving either client adherence to treatment or quality of services received by the client. To clarify the relationship between interventions and outcomes, we present a logic model (Figure 1).

Figure 1

---

---

Figure 1: Logic model outlining the three interventions and associated outcomes.

We will include studies in which the intervention involves digital tracking of clients combined with digital decision support via mobile device; or digital tracking of clients combined with targeted client communications via mobile device; or digital tracking of clients combined with both of these interventions, in the context of a primary healthcare setting.

In each case, the provider should be able to enter client data using a mobile interface and/or should be able to access individual client data using a mobile device. In contexts with more developed infrastructure, it may be preferable for providers to enter data on desktop computers. We will include studies that test interventions by using client information as entered via desktops, so long as the record is accessible on a mobile interface. Interventions may include systems by which clinical decision support tools are available on mobile devices and are used to enter relevant client data via these mobile devices. Similarly, the system may be set up in such a way that targeted messages are sent to clients' mobile devices. For studies included in this review, in which investigators used digital devices to provide maternity care from the prenatal to the postpartum period and included the function of birth notification, we will include only prenatal outcomes.

By 'mobile devices', we mean mobile phones of any kind (but not analogue landline telephones), as well as tablets, personal digital assistants, and smartphones.

As shown in Figure 1, comparisons for this review will include standard practice and interventions that included completed non‐digital systems.

We will exclude the following.

• Studies that perform digital tracking of client healthcare services on laptop alone, or in which the client health record is not accessible via a mobile device.

• Studies that provide decision support on stationary computers or laptops alone.

• Pilot and feasibility studies (pilot study defined as “a version of the main study that is run in miniature to test whether the components of the main study can all work together”, and feasibility study defined as “pieces of research done before a main study” (Arain 2010)).

• Studies in which digital decision support or targeted client communication is provided as a standalone intervention or is not linked to client health record.

• Studies in which untargeted client communication is provided.

• Studies that include interventions targeted at notification of new births that measure only postnatal outcomes attributable to timely birth notification.

• Studies that do not include a direct face‐to‐face provider‐client service delivery interaction.

Types of outcome measures

Electronic searches

An independent Information Specialist (JE) will develop search strategies in consultation with the review authors.

We will search the following databases for primary studies, from 2000 to date of search.

• Cochrane Central Register of Controlled Trials (CENTRAL), in the Cochrane Library.

• MEDLINE, Ovid.

• Embase, Ovid.

• Population Information Online (POPLINE), K4Health.

• Global Health Library, WHO.

Appendix 1 lists the search strategy for MEDLINE Ovid. Search strategies comprise keywords and controlled vocabulary terms. We will not apply any limits on language.

Searching other resources

We will search for ongoing trials in the following trial registries and will contact study authors to request further information and data, if available.

• WHO ICTRP (World Health Organization International Clinical Trials Registry Platform; www.who.int/ictrp).

• US National Institutes of Health Ongoing Trials Register ClinicalTrials.gov (www.clinicaltrials.gov).

We will search Epistemonikos (www.epistemonikos.org) for relevant systematic reviews and potentially eligible primary studies. Additionally, the WHO will issue a call for papers through popular digital health communities of practice such as the Global Digital Health Network, as well as grey literature, to identify additional primary studies.

Selection of studies

A core team of two review authors (NH, HB), with assistance when necessary from one additional review author (SA), will be responsible for selection of studies. We will download all titles and abstracts retrieved by electronic searching to a reference management database and will remove duplicates. Two review authors (NH, HB) will independently screen titles and abstracts for inclusion. We will retrieve full‐text study reports/publications of all relevant studies, and two review authors (NH, HB) will independently screen these for inclusion. We will record all reasons for exclusion of ineligible studies. We will resolve disagreements through discussion, or, if required, we will consult a third review author (SA).

We will list in the 'Characteristics of excluded studies' table studies that initially appeared to meet the inclusion criteria but that we later excluded. We will collate multiple reports of the same study, so that each study rather than each report is the unit of interest in the review. We will report in an 'Ongoing studies table' all information that we can obtain about ongoing studies. We will record the study selection process in sufficient detail to complete a PRISMA flow diagram (Liberati 2009).

Data extraction and management

We will modify the EPOC standard data extraction form and will adapt it for study characteristics and outcome data relevant to this review (EPOC 2017a). We will identify key characteristics of the intervention for abstraction based on mHealth Evidence Review and Assessment (mERA) guidelines (Agarwal 2016). We will pilot the form on at least one study in the review. Two review authors (NH, HB) will independently extract the following study characteristics from the included studies.

• General information: title, reference details, study author contact details, publication type, funding source, conflicts of interest of study authors.

• Population and setting: country, geographical location (rural, urban, peri‐urban), healthcare setting (e.g. facility‐based, community‐based).

• Methods: function of the intervention, study design, unit of allocation, duration of the study.

• Participant characteristics: type of healthcare worker (function, age, length of training), description of clients serviced by the healthcare worker, description of any other participants in the intervention, withdrawals.

• Interventions.

  • For tracking and CDSS interventions: intervention purpose; components of tracking interventions and CDSS interventions; type of technology and mode of delivery; content of the intervention; type of mobile device/s used (smartphone, tablet, feature phone, basic phone, laptops); healthcare provider training; interoperability; compliance with national guidelines; data security; comparison; fidelity assessment; duration of the intervention.

  • For tracking and TCC interventions: intervention purpose; intervention components' mode, timing, frequency, and duration of intervention delivery; content of the intervention; type of mobile device/s used (smartphone, tablet, feature phone, basic phone, laptops); interoperability; compliance with national guidelines; data security; comparison; fidelity assessment.

  • For tracking, CDSS, and TCC review: We will extract all characteristics listed in the sub‐bullets above.

• Outcomes: primary and other outcomes (continuous and dichotomous) for each intervention group, time points reported, adverse events, results of any subgroup analyses.

We will note in the 'Characteristics of included studies' table if outcome data were reported in a unusable way. We will resolve disagreements by reaching consensus or by involving a third review author (SA).

Assessment of risk of bias in included studies

Two review authors (NH, HB) will independently assess risk of bias for each study by using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions Section 8.5 (Higgins 2017) and guidance from the EPOC group (EPOC 2017b). We will resolve any disagreements by discussion or by consultation with a third review author (SA). We will assess risk of bias for randomised/non‐randomised trials and controlled before‐after studies using the following criteria: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, baseline outcomes measurement, similarity of baseline characteristics, and other bias. For interrupted time series studies, we will assess risk of bias using these seven standard criteria (EPOC 2017b): independence of the intervention from other changes, prespecified shape of the intervention effect, independence of intervention and data collection procedures, objectivity of the outcomes, bias resulting from missing outcome measures, selective outcome reporting, and other biases such as seasonality.

We will judge risk of each potential source of bias as high, low, or unclear, and will provide a quote from the study report together with a justification for our judgement in the 'Risk of bias' table. We will summarise 'Risk of bias' judgements across different studies for each of the domains listed. We will consider blinding separately for different key outcomes when necessary (e.g. for unblinded outcome assessment, risk of bias for all‐cause mortality may be very different than for a patient‐reported pain scale). When information on risk of bias relates to unpublished data or correspondence with a trialist, we will note this in the 'Risk of bias' table. We will not exclude studies on the grounds of their risk of bias, but will clearly report risk of bias when presenting results of studies. When considering treatment effects, we will take into account risk of bias for studies that contribute to that outcome. We will further perform assessment of evidence quality using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach (Guyatt 2008). We will summarise findings in ‘Summary of findings’ tables.

We will conduct the review according to this published protocol and will report deviations from it in the 'Differences between protocol and review' section of the systematic review.

Measures of treatment effect

We will report preintervention and postintervention means and proportions for intervention and comparison groups. When possible, we will estimate effects of the intervention using risk ratios/risk differences for dichotomous data, together with appropriate associated 95% confidence intervals, and mean differences or standardised mean differences for continuous data, together with 95% appropriate associated confidence intervals. We will ensure that an increase in scores for continuous outcomes can be interpreted in the same way for each outcome, will explain the direction to the reader, and will report when directions were reversed if this was necessary. For interrupted time series, we will report preintervention and postintervention slopes of analysis, differences between slopes, and differences between intercepts at the first intervention point and predicted intercept by intervention. If interrupted time series data have been analysed incorrectly, we will reanalyse the data when possible (EPOC 2017c).

Unit of analysis issues

If cluster‐randomised trials or controlled before‐after studies are included in the review, we will report cluster‐adjusted risk ratios or differences and their 95% confidence intervals. If the analysis was not adjusted for clustering, we will use the intracluster correlation coefficient (ICC), if available, to adjust the confidence interval. If the ICC is not available, we will present results without a measure of variance or precision of effect for outcomes for which we find a unit of analysis error (EPOC 2017d).

Dealing with missing data

We will contact investigators to verify key study characteristics and to obtain missing outcome data when possible (e.g. when a study is identified as abstract only). If this is not possible, we will report the data as missing, will note this in the 'Risk of bias' tables, and will not attempt to impute missing values. For all outcomes, we will carry out analyses, to the extent possible, on an intention‐to‐treat basis using available cases. However, in assessing adverse events, we will relate results to actual treatment received (i.e. analyses will be based on participants who actually received the intervention and adverse events reported in the studies).

Assessment of heterogeneity

If we find a sufficient number of studies that evaluate similar interventions and report similar outcomes, we will conduct a meta‐analysis. We will examine heterogeneity by visually inspecting forest plots and by using the I² statistic to measure heterogeneity among the trials in each analysis (Higgins 2017). If we identify substantial heterogeneity, we will explore this by performing prespecified subgroup analysis.

Assessment of reporting biases

We will attempt to contact study authors to ask them to provide missing outcome data. When this is not possible and missing data are thought to introduce serious bias, we will explore the impact of including such studies in the overall assessment of results. In cases where adjusted analyses for dichotomous outcomes are reported using odds ratios and not risk ratios, we will use Cochrane statistical software (Review Manager 2014) to convert odds ratios to risk ratios before including results in a meta‐analysis. If we are able to pool more than 10 trials, we will create funnel plots to explore possible publication bias or other causes of asymmetry (Sterne 2011). We will interpret results of the funnel plot with caution, as funnel plot calculations for dichotomous outcomes measured as risk ratios are not well developed, and statistical funnel plot results may not be representative in cases of small‐study effects.

Data synthesis

When studies evaluate interventions that are similar, we will group them together and will summarise key characteristics of each study in tables, to facilitate comparison across studies. We will undertake meta‐analyses using a random‐effects model only when this is meaningful (i.e. when intervention, context, and outcomes are similar enough for pooling to make sense). Trialists commonly indicate when they have skewed data by reporting medians and interquartile ranges. When we encounter this, we will note that the data are skewed and will consider the implications of this. When a single trial reports multiple trial arms, we will include only the relevant arms. When we must enter two comparisons (e.g. intervention A vs usual care and intervention B vs usual care) into the same meta‐analysis, we will halve the control group to avoid double‐counting.

Subgroup analysis and investigation of heterogeneity

We will perform subgroup analysis to assess variation in delivery of the intervention across different population groups, interventions, or setting characteristics, if possible. We will conduct subgroup analysis only if a sufficient number of trials are available for statistically significant comparisons between groups. We will assess heterogeneity within each subgroup by using forest plots and the I² measure. We will summarise results of subgroup analysis within the text of the review, if meta‐analysis is not possible or meaningful. We plan to carry out subgroup analyses by each of the following categories, depending on availability of data.

Sensitivity analysis

We will perform sensitivity analyses as defined a priori to assess the robustness of our conclusions and will explore their impact on effect sizes. This will involve restricting analysis to published studies and removing from meta‐analysis studies that have high risk of bias as determined by risk of bias assessment.