Scolaris Content Display Scolaris Content Display

Cochrane Database of Systematic Reviews Protocol - Intervention

Collaborative writing applications in healthcare: effects on professional practice and healthcare outcomes

This is not the most recent version

Collapse all Expand all

Abstract

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

  1. To assess the effects of the use of collaborative writing applications on process outcomes (including the behaviour of healthcare professionals) and patient outcomes

  2. To critically appraise and summarise current evidence on the resource use, costs, and cost‐effectiveness associated with collaborative writing applications to improve professional practices and patient outcomes

  3. To explore the effects of different collaborative writing application features (e.g., open versus closed) and different implementation factors (e.g., the presence of a moderator) on process outcomes and patient outcomes

Background

Description of the condition

Clinical practice does not always reflect best evidence. High proportions of inappropriate care have been reported in different healthcare systems and settings (Grol 2003). This has a huge impact on both patient outcomes and healthcare costs. Passive dissemination of evidence has not been proven enough to promote uptake of research results, so specific strategies to encourage implementation of research‐based recommendations and changes in practices have been advocated (Bero 1998).

Description of the intervention

Information and communication technologies have been suggested as one possible solution for improving research uptake and increase evidence‐based practice (Health Canada 1999). Governments are investing billions of dollars to implement information and communication technologies, hoping to improve care and reduce costs (Black 2011; Catz 2003; Health Canada 2007). Some information and communication technologies like computer physician order entry are effective in improving the quality and safety of healthcare (Durieux 2008; van Rosse 2009). However, the impact of other information and communication technologies on healthcare professional performance has been modest in other contexts (e.g. computer generated reminders (Shojania 2009a), nursing record systems (Urquhart 2009), clinical decision support systems (Moxey 2010; Pearson 2009; Riggio 2009; Shojania 2009b), handheld and mobile computers (Prgomet 2009), or have yet to be proven (Black 2011; Currell 2000; Martin 2008; McGowan 2009). Due to many different innovative and expanded capabilities, social media such as collaborative writing applications may offer greater benefit than these older information and communication technologies (Eysenbach 2008).

Collaborative writing applications (Chu 2010; Eapen 2007) are a category of social media that has surged in popularity in recent years, including within the healthcare sector (Chu 2010; Eysenbach 2008; Heilman 2011; McLean 2007). Although no two applications are identical, all consist of software that allows users to create online content that anyone can edit or supplement (Kaplan 2010). Thus, Internet users have turned to wikis to produce a Wikipedia entry to stop tuberculosis (Boulos 2006; Heilman 2011; Leuf 2001); to Google Knol to exchange research on influenza at the Public Library of Science (Levy 2008; Public Library of Science 2011; Manber 2007); and to Google Docs to review the literature on emergency medicine (Archambault 2010a; Archambault 2010b; Eapen 2007; Phadtare 2009). For the purposes of our study and having referred to the writing on the subject (Boulos 2006; Eapen 2007; Kaplan 2010), we defined “collaborative writing applications” as a category of social media that enables the joint and simultaneous editing of a web page or an online document by at least two users (e.g., wikis, Google Docs) (Kaplan 2010).

While there exist different types of collaborative writing applications, wikis are perhaps the most popular with Wikipedia being the most famous wiki. A wiki is defined as a page or collection of interlinked web pages designed to enable anyone who accesses it to easily contribute or modify its content (Boulos 2006; Chu 2010). A wiki allows anyone who has access to add, edit, or delete the content of a web site. Wiki software (e.g., MediaWiki) keeps track of all changes and acknowledges authors, which makes it possible to jump back to previous wiki versions easily and it is always clear to see who changed what. Another well‐known collaborative writing application is the Google Docs online office suite of word processing, spreadsheet and slideshow applications that allow users to access their documents from any computer via a web browser. Google Docs applications share some common features with wikis in that they allow multiple users to add, edit, or delete the content of a document. However, Google Docs applications differ from wikis in that they don't use the same type of software and that Google Docs applications allow simultaneous collaborative writing compared to wikis that don't allow simultaneous editing of the same text section. A scoping review about collaborative writing applications identified that wikis and Google Docs are the main types of collaborative writing applications used in healthcare (Archambault 2013). This scoping review also identified a host of other collaborative writing applications that offer similar collaborative authoring of documents such as certain knowledge management applications (e.g., Google Sites (Dhillon 2011), Microsoft Sharepoint (Yates 2011), social media platforms (e.g., MijnZorgNet (van de Belt 2014), Atlassian Confluence (Seebregts 2009), MinJournal (Moen 2009), and certain virtual learning environments that contain wiki pages (e.g., Blackboard (Varga‐Atkins 2010). These collaborative writing applications also integrate additional features that favour conversation and deliberation between users such as blogs and social networking services. Although blogs and social networking applications are not collaborative writing applications because they don't allow the editing of a common document, they do offer additional conversational features that can support the processes of collaborative writing. Finally, collaborative writing applications also include hybrid wikis such as Wikibreathe (Gupta 2010), Orthochina (Ma 2008), and FreyaWIKI (Van Der Schoor‐Knijnenburg 2009), which are collaborative writing applications custom‐built specifically for certain projects.

How the intervention might work

A recent scoping review has identified many different mechanisms of action that could explain how collaborative writing applications might work to improve the implementation of best practices (Archambault 2013). Among these mechanisms, the most frequently reported in the literature were that collaborative writing applications improve collaboration (n=30) (Adams 2010; Andrus 2010; Bookstaver 2011; Ciesielka 2008; Cinnamon 2010; Cobus 2009; Culley 2012; Dhillon 2011; Gupta 2010; Hamilton 2008; Hamm 2009; Hawkins 2010; Hollinderbäumer 2013; Hulbert‐Williams 2010; Jones 2010; Kardong‐Edgren 2009; Llambí 2011; Meenan 2009; Mirk 2010; Montano 2010; Morley 2012; Morose 2007; Muir 2010; Philip 2008; Shaw 2010; Streeter 2007; Van Der Schoor‐Knijnenburg 2009; Welsh 2007; Williams 2011), influence behaviour change domains (n=24) (Andrus 2010; Bookstaver 2011; Ciesielka 2008; Cinnamon 2010; Cobus 2009; Hamilton 2008; Hamm 2009; Hickerson 2009; Hulbert‐Williams 2010; Kardong‐Edgren 2009; Kitson‐Reynolds 2009; Koerner 2011; Llambí 2011; McGee 2008; Moeller 2010; Musil 2011; Phadtare 2009; Philip 2008; Shaw 2010;Stutsky 2009; Van Der Schoor‐Knijnenburg 2009; Wan 2009; Williams 2011), positively impact learning (n=20) (Adams 2010; Andrus 2010; Bookstaver 2011; Ciesielka 2008; Culley 2012; Dhillon 2011; Hamilton 2008; Hawkins 2010; Hulbert‐Williams 2010; Jones 2010; Koerner 2011; Lanning 2010; Llambí 2011; McGee 2008; Miller 2009; Moeller 2010; Musil 2011; Shaw 2010; Stutsky 2009; Umland 2011; Williams 2011), and facilitate knowledge management and increase access to information (n=17) (Bookstaver 2011; Chiarella 2009; Cinnamon 2010; Culley 2012; Felsen 2010; Gerber 2010; Hamilton 2008; Koerner 2011; Kohli 2011; Meenan 2009; Muir 2010; Streeter 2007; Welsh 2007; Williams 2011; Wright 2009; Yates 2011; Yu 2011). Among the behaviour change domains that are influenced by the use of collaborative writing applications, self‐efficacy, motivation, emotion, skills and knowledge have been reported as being positively influenced by the use of collaborative writing applications.

More specifically, it is expected that collaborative writing applications will support collaboration between multiple stakeholders (including patients, healthcare professionals, decision makers and researchers) who will participate in the construction of their content. This sharing of evidence‐based knowledge tools within a community of healthcare stakeholders using collaborative writing applications could increase the social influences this community of stakeholders has on the practice of evidence‐based healthcare (Godin 2008). As it is often very hard to get all stakeholders in a healthcare institution to meet in the same place and at the same time to discuss care processes, the implementation of new protocols and new care pathways, collaborative writing applications could help in gathering the feedback from a wide range of stakeholders. Thus, collaborative writing applications would allow feedback to be openly and quickly generated about the new care processes being implemented or to comment on older processes that would need to improve based on new evidence.

Moreover, by involving knowledge users in the creation and dissemination of knowledge (Eysenbach 2008), collaborative writing applications, which are highly accessible, web‐based, interactive vehicles of communication, have the potential to empower users to apply knowledge in practice. Collaborative writing applications facilitate the adoption of proactive behaviours among knowledge users who would remain passive targets of knowledge translation initiatives in the past. Eysenbach 2008 describes this as turning the consumer into a ‘prosumer’ or co‐producer of information. Applied to the healthcare field, this transformation of the traditional knowledge user into an active agent for the implementation and dissemination of knowledge has the potential to increase healthcare professionals’ self‐efficacy to implement evidence‐based knowledge into practice (Godin 2008; Michie 2005). By actively getting healthcare professionals to participate in the creation of evidence‐based knowledge tools within a collaborative writing application, a user’s own knowledge of the evidence would increase (Godin 2008).

The type of interaction between stakeholders and knowledge users and the feedback they provide will differ based on the type of collaborative writing applications used. Collaborative writing applications can be open or public such as Wikipedia, which can be edited by nearly anyone in the world and can also be seen by anyone. There are also partially public collaborative writing applications, which can be seen by anyone, but can only be edited by certain members of a restricted community (e.g., Ganfyd (GANFYD 2014), which is a wiki viewed by all but only edited by health professionals). There are also closed or private collaborative writing applications part of central knowledge management systems (e.g., Intelink (U.S. Government 2013)) or online learning systems (e.g., Blackboard), which are edited by members of the institution and are only visible to members of the institution. Hence, based on who has access to view and edit the content in these applications, the target and the effect of collaborative writing applications will be different.

Finally, collaborative writing applications have different features or implementation facilitators that can make them more or less effective in changing professional behaviours and patient outcomes. Among these factors, the following have been shown to be potentially important to consider (Archambault 2013): the level of openness of the collaborative writing application (i.e., the level of control about who is allowed to edit the content within the collaborative writing application); the presence of a moderator; the presence of a formal community of practice; the disclosure of authorship; the number and type of conversational features added to the collaborative writing application (e.g., email alerts integrated to the collaborative writing application and integration with other social media platforms); the format of knowledge being shared within the collaborative writing application (e.g., knowledge tools versus textbook‐style knowledge); and being part of a theory‐based intervention. This former feature is important because developing an intervention based on a behaviour change framework (e.g., the Theoretical Domains Framework (Michie 2008), the Theory of Planned Behaviour (Godin 1996), or the Technology Acceptance Model (Davis 1989)), helps implementers target specific behaviour change domains that are important to consider when implementing a complex intervention.

Why it is important to do this review

Even though collaborative writing application use is increasing and are already being used by different stakeholders in healthcare (professionals, patients, decision makers and researchers), questions remain about the safety (Devgan 2007; Goodman 2006; Mason 2009), reliability (Arita 2009; Callis 2009; Clauson 2008; Pender 2009; Rosenzweig 2006), lack of traditional authorship (Giles 2006; Kittur 2007), and the legal implications for decision making regarding the use of collaborative writing applications in healthcare (Cohen 2009, Jain 2009). Researchers question clinicians’ intentions to use the applications in their practice (Archambault 2010b), and to contribute knowledge collaboratively (Archambault 2010a; Heilman 2011; Logan 2010). Furthermore, it is unknown if collaborative writing applications are effective at enhancing the delivery of healthcare (e.g., by empowering patients in decision making (Adams 2010; Vogel 2011), by improving healthcare communication and education (Chu 2010; Collier 2010; ; Kim 2010; Naik 2010; Phadtare 2009;Tangient LLC 2013; Varga‐Atkins 2010), and benefiting health in developing countries (Godlee 2004; Heilman 2011; Seebregts 2009).

Hence, there have been calls to study how collaborative writing applications could improve the delivery of healthcare, decrease its cost (Eysenbach 2008; Mandl 2008), and increase the access to knowledge within developing countries (de Silva 2010; Heilman 2011). Specifically focusing on collaborative writing applications in healthcare is important since not all social media share the same mechanisms of action (Kietzmann 2011). While researchers have conducted scoping and systematic reviews on internet and communication technologies (Black 2011; Gagnon 2010), social media in healthcare (Cheston 2013; Chou 2013; Eysenbach 2004; Hollinderbäumer 2013; Moorhead 2013; von Muhlen 2012), and research on Wikipedia in general (Okoli 2012), there is yet to be a systematic review on the impact of collaborative writing applications on professional practices and patient outcomes.This knowledge gap has been confirmed by a scoping review on collaborative writing applications in healthcare (Archambault 2013).

Present researchers exploring collaborative writing applications to harness the power of collaboration in healthcare would benefit from the results of a systematic review focusing on the effects of collaborative writing applications on professional practice and patient outcomes. Furthermore, it is important to consider whether use of collaborative writing applications is worthwhile to stimulate best practices given the incremental costs (resource use) and benefits (effects) which may be associated with collaborative writing applications. Public non‐profit organisations, professional organisations and academic organisations who are using or considering using collaborative writing applications to help them disseminate information and implement knowledge could also benefit greatly from a better understanding of how collaborative writing applications impact professional practice, patient outcomes and healthcare costs (Sabouni 2014; WHO 2013).

Objectives

  1. To assess the effects of the use of collaborative writing applications on process outcomes (including the behaviour of healthcare professionals) and patient outcomes

  2. To critically appraise and summarise current evidence on the resource use, costs, and cost‐effectiveness associated with collaborative writing applications to improve professional practices and patient outcomes

  3. To explore the effects of different collaborative writing application features (e.g., open versus closed) and different implementation factors (e.g., the presence of a moderator) on process outcomes and patient outcomes

Methods

Criteria for considering studies for this review

Types of studies

We will include randomised controlled trials (RCTs), non‐randomised controlled trials (NRCTs), controlled before‐after (CBA) studies, interrupted time series (ITS) studies and repeated measures studies (RMSs).

For the economic analysis part of this review, we will include all previous study designs having conducted full economic evaluations (cost‐effectiveness analyses, cost‐utility analyses and cost‐benefit analyses), cost analyses or comparative resource utilisation studies.

Types of participants

We will consider healthcare stakeholders such as healthcare professionals, decision makers, researchers and patients. We will include studies enrolling single groups of stakeholders (e.g., patients) as the only target of a collaborative writing application intervention only if the ultimate goal is to improve professional practices (e.g., involving patients in the development of clinical practice guidelines using a collaborative writing application). Moreover, we will include studies enrolling healthcare students as long as the majority of the participants are certified health professionals.

Types of interventions

We will consider studies that evaluated any collaborative writing application intervention aimed at improving processes of care, patient outcomes and healthcare costs. Our definition of a collaborative writing application intervention includes all of the following elements:

  • content that is collaboratively created;

  • a web page that allows the joint and/or simultaneous editing accessed by a group of authors;

  • a web based software to keep track of the revisions to the original document;

  • a public or private access.

Since we will compare the specific effects of wikis compared to other collaborative writing applications, we will include wiki interventions if they contain all the following elements:

  • a web page or a collection of interlinked web pages designed to enable anyone who accesses it to easily contribute or modify its content using a web browser;

  • a wiki software that keeps track of all changes and all the authors (and their internet protocol addresses) making the changes (e.g., MediaWiki and Tiki Wiki CMS Groupware);

  • a collaboratively created content;

  • a public access (e.g., Wikipedia, community websites) or a private access (e.g., corporate intranets, knowledge management systems).

We will exclude blogs, microblogs (e.g., Twitter), online communities of practice, discussion forums, media sharing websites and social bookmarking websites that do not include any collaborative writing feature defined as a common page that allows the joint and/or simultaneous editing by more than one person.

We will conduct the following comparisons:

  • collaborative writing application intervention compared to no intervention;

  • collaborative writing application intervention versus any single intervention (e.g., a static web page);

  • collaborative writing application intervention compared to a wiki intervention;

  • multifaceted intervention where a collaborative writing application is included versus any multifaceted intervention without a collaborative writing application; and

  • a collaborative writing application with an implementation strategy compared to a wiki/collaborative writing application without an implementation strategy.

A single intervention would be any static resource (e.g., a traditional web page) intervention where users do not have the capability in editing the content of the page. A multifaceted intervention would be any intervention where a collaborative writing application is a part of the intervention but is not the only active ingredient. For example, a study could explore the impact of a virtual learning environment that contains a wiki and other applications (e.g., discussion boards, social networks, live virtual classroom meetings) compared to traditional teacher‐led classroom meetings or other educational interventions like problem‐based learning groups.

Types of outcome measures

Primary outcomes

We will include studies if they report at least one outcome involving a healthcare process outcome or a clinical endpoint. Our healthcare process outcomes will include but will not be limited to:

  • any objective measure of professional performance (such as number of tests ordered, prescriptions for a particular drug, adverse events (e.g., medication errors, rates of complications), process and performance indicators, adherence to clinical practice guidelines);

  • interprofessional collaboration;

  • cost of care delivery (economic data);

  • length of hospital stay.

Patient outcomes will include but will not be restricted to: mortality and other markers of health (e.g., blood pressure, quality of life).

If and only if the primary outcome of interest is met then will we include other secondary outcomes (Secondary outcomes).

Secondary outcomes

Being adaptive and dynamic interventions, collaborative writing applications are complex interventions that need an adapted framework to measure and understand the mechanism of action leading to their effect. For this reason, we will use a specific framework developed to evaluate collaborative adaptive interactive technologies (O'Grady 2009).

This framework is based on the following themes.

  • People affected by the website.

  • Content of the website.

  • Technology of the website.

  • Human‐computer interaction between the person and the website.

  • Effects on the greater healthcare community that cut across three core evaluation phases:

    • formative phase;

    • summative; and

    • outcome.

The key issue will be to measure the different mechanisms involved in using collaborative writing applications as an adaptive and dynamic intervention. In order to achieve this, we will use the five themes and three phases of evaluation as mentioned above to categorize our secondary outcomes. To complement this framework, we will also use a taxonomy developed to describe the intermediate processes, including the professional behaviours, that need to be measured to understand the underlying mechanisms of action in play with collaborative writing applications (Archambault 2013). These mechanisms of action (collaboration, learning, knowledge management, access to information, increased self‐efficacy, motivation, skills and knowledge) will be measured as intermediate process outcomes that are part of the causal chain leading collaborative writing applications to improve healthcare practices and patient outcomes. Even if some of these variables will be self‐reported, their inclusion will help identify how the different collaborative writing applications are actually working and eventually establish a dose‐effect relationship between the processes, professional behaviours and patient outcomes.

In summary, we will include the following secondary outcomes.

  • Outcomes about the people affected by the collaborative writing application, which will include measures of:

    • self‐efficacy;

    • motivation;

    • skills;

    • knowledge; and

    • satisfaction.

  • Outcomes about the content of the website, which will include measures of:

    • quality of content; and

    • credibility of the content.

  • Outcomes about the technology of the collaborative writing application, which will include measures of:

    • number of visits; and

    • number of edits.

  • Outcomes about the human‐computer interaction, which will include measures of:

    • usability;

    • collaboration;

    • knowledge management.

  • Outcomes about the greater healthcare community, which will include measures of:

    • adverse events related to using information from collaborative writing application; and

    • service utilisation (e.g., number of visits to a physician) due to use of collaborative writing application.

Economic data

We will be including the following broad categories of economic data in our review of existing economic evaluations:

  • measures of resource use;

  • measures of costs; and

  • measures of cost‐effectiveness.

More specifically, measures of resource use will include: length of hospital stay, length of emergency room stay, length of intensive care unit stay, length of rehabilitation care, number of planned and unplanned outpatient consultations, use of nursing home care, number of healthcare professionals working on updating and maintaining up‐to‐date decision tools and total time spent by healthcare professionals in a care unit (hospital or clinic) creating and maintaining up‐to‐date decision tools.

Measures of costs will include direct medical costs: cost of hospital care, cost of emergency room care, cost of intensive care unit care, cost of rehabilitation care, cost of planned and unplanned outpatient visits, cost of nursing home care, cost of time spent creating and maintaining care unit (hospital or clinic) decision support tools (e.g., care protocols, order sets, patient decision aids) and chronological cost accrual dedicated to maintaining up‐to‐date decision support tools. Non‐medical costs (e.g., patient loss of productivity) will be included in our analysis if authors report them. We will collect costs at the level of patients and at the level of healthcare organizational units (clusters) where appropriate depending on the outcome. For example, we will extract cost data at the patient level for cost of hospital care (e.g., mean cost of hospital care per patient). However, we will extract cost data at the hospital level for maintaining up‐to‐date decision support tools (e.g., mean total cost to pay professionals in a hospital to maintain up‐to‐date decision support tools). We will adjust cluster‐level cost data to cluster size.

Measures of cost‐effectiveness will include: incremental cost‐effectiveness ratios (ICERs), incremental cost‐per quality adjusted life‐years (QALY) and cost‐benefit ratios.

Search methods for identification of studies

Electronic searches

A draft search strategy for MEDLINE and additional databases based on the MEDLINE strategy has been developed by the Cochrane Effectice Practice and Organisation of Care (EPOC) Group Trials Search Co‐ordinator in consultation with the authors (Appendix 1). A second information specialist peer‐reviewed this strategy using the Peer Review of Electronic Search Strategies (PRESS) criteria (Sampson 2008). We will search the Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE) for related systematic reviews.

In addition, we will search the following databases for primary studies:

  • MEDLINE (In‐Process and other non‐indexed citations) (1946‐present) (OvidSP);

  • EMBASE (1947‐present) (OvidSP);

  • CINAHL (Cumulative Index to Nursing and Allied Health Literature) (1980‐) (EBSCOhost);

  • PubMed;

  • Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library);

  • Health Technology Assessment (HTA) Database (The Cochrane Library);

  • National Health Service (NHS) Economic Evaluation Database (The Cochrane Library);

  • Health Economic Evaluations Database (1994‐present);

  • Dissertations and Theses Full Text (1861‐present) (ProQuest);

  • Education Resources Information Center (ERIC);

  • Science Citation Index Expanded (SCI‐EXPANDED) (1900‐present);

  • Social Sciences Citation Index (SSCI) (1900‐present);

  • Conference Proceedings Citation Index‐ Science (CPCI‐S) (1990‐present);

  • Conference Proceedings Citation Index‐ Social Science & Humanities (CPCI‐SSH) (1990‐present).

We will employ two methodological filters: the 'Cochrane Highly Sensitive Search Strategy (sensitivity‐ and precision‐maximizing version ‐ 2008 revision) to identify randomized trials' (Higgins 2011), and a Cochrane EPOC Group methodology filter to identify non‐RCT designs. We will not apply any limits on language or date of publication. However, in emerging technology areas, database controlled vocabulary, such as Medical Subject Headings (MeSH) found in MEDLINE and PubMed, lags behind. Thus, in MEDLINE, papers about wikis may be indexed using very general terms such as "Internet" or "Online Software." In the draft Medline strategy published here, the number of results found by these broad terms is not excessive and no date limit has been used. In other databases, depending on the search terms, a date limit corresponding to the emergence of wiki technology may be used for practical reasons.

We will search for economic analysis reviews in MEDLINE and EMBASE using the filter strategies developed by the Canadian Agency for Drugs and Technologies in Health (Glanville 2009).

Searching other resources

Grey literature

We will conduct a grey literature search to identify studies not indexed in the databases listed above. We will use the following sources will include the sites listed below:

We will document additional sources, if any, in the full review.

Trial registries

We will search the following trial registries for ongoing and completed trials:

Others

We will also:

  • screen individual journals and conference proceedings (e.g., handsearching);

  • review reference lists of all included studies, relevant systematic reviews and primary studies;

  • reference lists of identified studies will be searched for additional RCTs and health economics studies;

  • contact the authors of relevant studies or reviews to clarify reported published information or to seek unpublished results/data;

  • contact researchers with expertise relevant to the review topic or EPOC interventions;

  • contact experts from the Campbell and Cochrane Economics Methods Group;

  • conduct cited reference searches for all included studies in citations indexes;

  • contact the authors of existing projects using collaborative writing applications on the web (Appendix 2).

Data collection and analysis

Selection of studies

Two independent authors (PA,AP) will assess the eligibility of papers identified. We will screen all titles and abstracts according to pre‐established inclusion criteria. We will obtain the full‐text copies of papers in order to fully assess their eligibility. Any disagreements will be resolved by discussion between the two authors, and where consensus is not reached, we will consult a third author (TvdB). We will document the selection process in a Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) flow diagram (Moher 2009), and also illustrate our findings in a 'Characteristics of excluded studies' table. This table will only present the excluded studies with the reasons of exclusion.

We will describe characteristics of ongoing studies such as the primary author, research question(s), methods and outcome measures together with an estimate of the reporting date.

Data extraction and management

Two review authors (PA,AP) will independently undertake data extraction from all included studies using a modified version of the Cochrane EPOC Group data extraction template. Any discrepancies in judgement will be resolved by discussion and consensus, or by consulting a third review author (TvdB) if necessary. Where information is missing, we will attempt to contact the study authors, and if data are not available the study will be recorded as an 'Included study without data.

In addition to standardized data extraction for EPOC interventions, we will use collect the following characteristics of the intervention.

  • Clinical area addressed.

  • Type of healthcare professional (e.g., physician, nurse, physiotherapist, student nurse, resident).

  • Pre‐licensure healthcare professionals (student nurses, residents).

  • Type of healthcare stakeholder (e.g., patients, consumer, decision maker, researcher).

  • Conceptual or theoretical underpinnings of the intervention (i.e., part of a theory based intervention).

  • Known effectiveness of the intervention for changing of healthcare professional behaviours (e.g., evidence‐based intervention).

  • Number of components included in the intervention.

  • Source and authors of the intervention (e.g., professional organisation, governmental agency, health professionals training schools).

  • Mode of delivery (e.g., individuals or groups).

  • Frequency/timing of the intervention and its duration.

  • Type of collaborative writing application (i.e., Wiki, Google Knol, Google Docs, Google Wave).

  • Presence of a moderator.

  • Presence of a pre‐existing community of practice or community of learners.

  • Open versus closed wiki or collaborative writing application.

  • Authorship clearly stated.

  • Type of conversational features integrated with the collaborative writing application (e.g., discussion pages, email, comment page).

  • Format of knowledge being shared within the collaborative writing application (e.g., reminders, care pathways, textbook format).

  • Formal teaching about how to use a wiki.

For economic analysis data extraction, we will develop a data collection form specifically for use with health economic studies, based on the template used to produce the National Health Service (NHS) Economic Evaluation Database (EED) structured abstracts (Craig 2007).

Assessment of risk of bias in included studies

Two review authors (PA,AP) will independently assess the risk of bias of included studies, with disagreements resolved by discussion and consensus. The two authors will assess and report on the risk of bias of included RCTs, NRCTs, and CBA studies in accordance with the suggested nine risk of bias criteria for Cochrane EPOC Group reviews (Cochrane EPOC Group 2014):

  1. adequate sequence generation;

  2. allocation concealment;

  3. blinding;

  4. incomplete outcome data addressed;

  5. free of selective reporting;

  6. free of other bias;

  7. baseline outcomes similar;

  8. free of contamination; and

  9. baseline characteristics similar.

For ITS studies and RMSs, we will use the seven risk of bias criteria for Cochrane EPOC Group reviews (Cochrane EPOC Group 2014):

  1. independence from other changes in time;

  2. shape of the intervention effect pre‐specified;

  3. intervention unlikely to affect data collection;

  4. blinding;

  5. incomplete data outcome;

  6. free from selective outcome reporting; and

  7. free from other biases.

For all study types, we will assess each risk of bias criterion as "High risk", "Low risk", or "Unclear risk", and also present a table describing these assessments of the risk of bias for each study and outcome. We will also prepare a summary table presenting an assessment of the risk of bias within and across studies for important outcomes.

We will assign an overall assessment of the risk of bias (high, moderate or low risk of bias) to each of the included studies using the approach suggested in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). Studies with low risk of bias for all key domains or where it seems unlikely for bias to seriously alter the results will be considered to have a low risk of bias. Studies where risk of bias in at least one domain is unclear or judged to have some bias that could plausibly raise doubts about the conclusions will be considered to have an unclear risk of bias. Studies with a high risk of bias in at least one domain or judged to have serious bias that decreased the certainty of the conclusions will be considered to have a high risk of bias.

For included health economic studies, we will assess their risk of bias using the Cochrane Collaboration’s tool for assessing risk of bias as well as the Consolidated health economic evaluation reporting standards (CHEERS) developed by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) (Higgins 2011; Husereau 2013).

Measures of treatment effect

Reporting

For each study, we will report data in natural units. Where baseline results are available from RCTs, NRCTs, CBA studies, ITS studies, and RMSs, we will report pre‐ and post‐intervention means or proportions for both study and control groups. We will calculate the unadjusted and adjusted (for any baseline imbalance) absolute change from baseline with 95% confidence intervals (CIs). For ITS studies, we will report the main outcomes in natural units and two effect sizes: the change in the level of outcome immediately after the introduction of the intervention and the change in the slopes of the regression lines. Both of these estimates are necessary for interpreting the results of each comparison. For example, there could have been no change in the level immediately after the intervention, but there could have been a significant change in slope.

Analytic approach

We will use the statistical methods proposed in Grimshaw 2004 to guide data analysis and presentation. Our statistical analysis will be based upon consideration of dichotomous practitioner process outcomes, continuous practitioner process outcomes, dichotomous patient outcome measures and continuous patient outcome measures. In cases where there is insufficient data to calculate these effect sizes, we will present results of studies as reported by the authors.

Where studies report more than one outcome, we will extract data on the primary outcome (as defined by the authors of the study). However, if the study reported more than one outcome and none of them were denoted as the primary variable, we will rank the effect sizes for all the variables and take the median value. In all cases, the direction of the effect size will be standardized so that a positive difference between post‐intervention percentages or means will indicate a positive outcome.

Primary analyses

Where possible, we will present the results for all comparisons using a standard method of presentation. We will report results for categorical and continuous outcomes separately. For comparisons of RCTs, NRCTs, CBA studies, ITS studies, and RMSs, we will report (separately for each study design) median effect size across included studies, interquartile ranges (IQRs) of effect sizes across included studies and range of effect sizes across included studies. We will calculate standardized effect sizes for continuous measures by dividing the difference in mean scores between the intervention and comparison group in each study by the standard deviation of the comparison group. This results in a ‘scale free’ estimate of the effect for each study, which can be interpreted and pooled across studies regardless of the original scale of measurement used in each study (Laird 1990). We will also report, when available, the pre‐ and post‐intervention study and control data in natural units with statistical significance provided across groups. For ITS studies, we will follow the recommendation of Ramsay 2003 and compute, where possible, a difference in slopes and/or a level effect.

Secondary analyses

Secondary analyses will explore consistency of primary analyses with other types of endpoints.

Unit of analysis issues

Methods for re‐analysis of RCTs, NRCTs and CBA studies with potential unit of analysis errors

Comparisons that randomise or allocate clusters (healthcare professionals or organisations) but do not account for clustering during analysis have 'potential unit of analysis errors' resulting in artificially extreme P values and overly narrow CIs (Ukoumunne 1999). Therefore, when possible, we will attempt to re‐analyse studies with potential unit of analysis errors, by contacting primary authors for missing information. If a comparison is re‐analysed, we will quote the P value and annotate it with 're‐analysed'. If this is not possible, we will report only the point estimate.

Methods for re‐analysis of ITS studies comparisons with inappropriate analysis

Where possible, we will use time series regression to re‐analyse each comparison. We will estimate the best‐fit pre‐intervention and post‐intervention lines using linear regression and autocorrelation adjusted for using the Cochrane‐Orcutt method where appropriate (Draper 1981). We will test for first‐order autocorrelation statistically using the Durbin‐Watson statistic and higher order autocorrelations will be investigated using the autocorrelation and partial autocorrelation function.

Dealing with missing data

All efforts will be made to contact the original authors if missing data is present in an included study. If this is impossible, we will clearly state if the data seems to be ‘missing at random’. If this is the case, we will analyse only the available data and ignore the missing data (Higgins 2011). If data seems ‘not to be missing at random’, we will clearly state our assumptions and we will either impute the missing data with replacement values, and treat these as if they were observed (e.g., last observation carried forward, imputing an assumed outcome such as assessing all were poor outcomes, imputing the mean, imputing based on predicted values from a regression analysis). We will also explore the impact of missing values on our results with the use of statistical models to allow for missing data, making clearly stated assumptions about their relationship with the available data. We will perform sensitivity analyses to assess how sensitive our results are to reasonable changes in the stated assumptions (Sensitivity analysis). Finally, we will address the potential of missing data on the findings of our review in our discussion section.

Assessment of heterogeneity

We will analyse included studies to determine whether there are studies sufficiently similar in participants characteristics (e.g., age, gender), study design, type of collaborative writing application (e.g., wiki, Google Docs), type of knowledge contained in collaborative writing application (e.g., wikis sharing reminders), environmental setting (e.g., critical care), health condition (e.g., acute disease) and outcome measurement to allow for a meta‐analysis of their combined data using a random‐effects model. If studies are too heterogeneous, we will present a descriptive review of included studies using a narrative summary along with extracted data in tables and figures.

Where meta‐analysis is possible, we will assess statistical heterogeneity between trials using the Chi2 test and the I2 statistic (a Chi2 P value of less than 0.05 or an I2 value equal to or more than 50% will be considered to indicate substantial heterogeneity). If heterogeneity is identified, we will undertake subgroup analysis to investigate its possible sources (Subgroup analysis and investigation of heterogeneity). We will also conduct a meta‐regression if there are enough studies to assess the effect of the possible sources of heterogeneity.

Assessment of reporting biases

If sufficient studies are found, we will assess for publication bias graphically using funnel plots and statistically using Begg and Egger tests (Egger 1997). While funnel plot asymmetry may indicate publication bias, this is not always the case. We will consider all possible explanations for this asymmetry and discuss them in the review.

Data synthesis

We expect to find statistical heterogeneity, given the range of disparate settings where wikis and collaborative writing applications will be tested and the difference of content within these different collaborative writing applications. This makes it unlikely that statistical pooling will be feasible, but if there appears to be a body of studies amenable to meta‐analysis, then we will display their results graphically, and view them to assess heterogeneity. It is unlikely that this review will find many studies for inclusion in a meta‐analysis, and so statistical tests for heterogeneity will be insensitive. We will nevertheless carry out these tests. We will conduct any subsequent meta‐analyses using a random‐effects model. In comparisons of homogeneous groups of studies with different outcome measures, we will consider using a single effect size measure (standardised mean differences).

We will provide narrative and qualitative summaries. We will group studies by criteria included in this protocol (e.g., type of collaborative writing application), and vote counts made of studies grouped according to simple trichotomies on outcome: effective/no difference/harm. We will perform formal qualitative textual analysis for each of these groups of studies, looking for common themes in the description of the intervention within each outcome group, and systematic differences between groups of studies with different outcomes.

We will produce a 'Summary of findings' table with selected studies according to the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).

We will list all the outcomes that are important to patients and other decision makers. Consultation and feedback with the current decision makers on our team have identified that all the primary outcomes of our review will be included in this Summary of findings table. In addition, we will list all negative outcomes identified (e.g., quality of content, adverse events related to collaborative writing application knowledge use).

Subgroup analysis and investigation of heterogeneity

We will prepare tables and bubble plots comparing effect sizes of studies grouped according to potential effect modifiers to investigate heterogeneity. For example, the following variables will guide our exploration of heterogeneity: the type of collaborative writing application (wikis compared to Google Docs), the age of wiki users/editors (‘internet generation’ (born between 1976 and 1994) compared to participants born before the ‘internet generation’). We will consider the following subgroups to explore the following effect modifiers of the intervention on the magnitude of effects observed across studies:

  • open versus closed collaborative writing applications;

  • clear authorship identification;

  • the presence of a moderator; and

  • being part of a theory‐based intervention.

Sensitivity analysis

We will undertake sensitivity analyses for the allocation of missing data by best and worst case analysis and will also undertake sensitivity analysis on the basis of our evaluation of the risk of bias of each study. Since studies will not be blinded and already contain one domain with high risk of bias, our sensitivity analyses will include studies with at least two high risk of bias domains.

Economics issues

As described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011), we will summarise methodological characteristics and results of included economic evaluations using additional tables (Additional tables), supplemented by a narrative summary that will compare and evaluate methods used and principal results between studies. We will tabulate unit cost data when available. We will report the currency and price year which applies to measures of costs in each original study alongside measures of costs, incremental costs and incremental cost‐effectiveness, by study. Where details of currency and price year are available in original studies, we will convert measures of costs, incremental costs and cost‐effectiveness to 2014 US Dollar value, using implicit price deflators for gross domestic product (GDP) and purchasing power parities (PPPs). We will use the Campbell and Cochrane Economics Methods Group (CCEMG) and the Evidence for Policy and Practice Information and Coordinating Centre (EPPI‐Centre) 'CCEMG‐EPPI Cost Converter' (http://eppi.ioe.ac.uk/costconversion/), which is web‐based tool for adjusting estimates of cost expressed in one currency and price year to a specific target currency and price year (Shemilt 2010).