مداخلاتی جهت یادآوری و فراخوانی بیمار برای بهبود نرخ ایمنسازی
Referencias
منابع مطالعات واردشده در این مرور
منابع مطالعات خارجشده از این مرور
منابع اضافی
منابع دیگر نسخههای منتشرشده این مرور
Characteristics of studies
Characteristics of included studies [ordered by study ID]
Ir a:
| Methods | Study design: randomized trial; allocated children without grouping children in same family Study aim: evaluate effectiveness of mail and telephone interventions in increasing immunization rates among children less than 7 years of age in family practice residency clinic | |
| Participants | Inclusion: children actively enrolled in family practice residency clinic; not up‐to‐date with immunizations | |
| Interventions | Intervention: sent postcard reminder to parents, indicating types of immunizations needed by child, and urging parents to make appointment; made telephone calls to parents of unimmunized children, 6 weeks after postcard intervention; written in English; n = 213 | |
| Outcomes | Number and percent of children immunized: intervention 8.8 percentage point increase over controls | |
| Notes | 13% of postcards sent were returned as undeliverable Approximately 1% of families in practice Spanish‐speaking; postcards may not have been understood; 17.8% of telephones were disconnected 41 of 177 intervention families not reached by telephone Results seem to be inconsistently reported for children brought up‐to‐date with immunizations; reversed in study abstract compared with results in Table 3 | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "immunization records of the 519 children were entered into a minicomputer” Probably randomized within computer even though method not explicitly specified |
| Allocation concealment (selection bias) | Unclear risk | Charts of infants were reviewed for immunizations received; supplemented these data with immunizations recorded in health department registry; entered immunization records into minicomputer prior to randomizing children; procedures not explicitly described |
| Blinding of participants and personnel (performance bias) | Unclear risk | No information provided |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified for review of practice billing codes and charts |
| Incomplete outcome data (attrition bias) | Unclear risk | Number of reviewed records not specified; asked parents by telephone about immunizations received Immunization status confirmed by reviewing practice billing codes and charts and county health department's records Did not confirm or record immunizations received at other sites |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Randomized patients within residency clinic; authors noted there may have been some contamination of control group because children with different surnames living in same household could have been assigned to different study groups |
| Baseline measurement | Low risk | "Immunization records of the county health department were reviewed." At baseline, reviewed charts of all infants actively enrolled in practice and older than 2 months and less than 7 years of age; supplemented information from health department immunization registry; selected participants behind on immunizations |
| Methods | Study design: randomized trial Study aim: evaluate effect of 3 types of computer‐generated mailed reminders on influenza immunization rates | |
| Participants | Inclusion: adult patients aligned with primary care physician within health system and at high risk for influenza complications based on age 65 years or older, or diagnosis of asthma, diabetes, end stage renal disease, sickle cell disease, ischemic cardiomyopathy, or nephrotic syndrome | |
| Interventions | Intervention group 1: generic postcard to patient, standard message; n = 6169 Intervention group 2: personalized postcard from patient's primary care physician; n = 6252 Intervention group 3: personalized letter from the patient's primary care physician, addressed to specific patient; message tailored to specific health risk of patient; n = 6151 Comprehensive program included: walk‐in influenza vaccination clinics during October at all health system outpatient clinic locations, display of posters and take‐home postcards in clinic entrances and waiting areas, toll‐free information telephone line, developed program logo and theme used in all print media, and standard message in printed materials was based on Health Belief Model | |
| Outcomes | Number and percent receiving influenza vaccination | |
| Notes | Patient reminders were one component of comprehensive influenza immunization program. Used billing data to calculate rate of immunizations in study groups; some vaccinations may have been received at unaffiliated clinics, some of which provided vaccinations free of charge Authors identified possible threshold effect | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Patients were identified as being eligible for study using computerized appointment scheduling system and demographic data and computerized billing data; patients were randomized to 4 groups; method is not described |
| Allocation concealment (selection bias) | Unclear risk | Randomized patients into one of 4 groups; method not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | No information provided |
| Blinding of outcome assessment (detection bias) | Low risk | Measured outcomes using billing data; blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Used billing data to obtain immunization rates Authors note possibility that some participants may have received immunizations at non‐study clinics, some of which offered free vaccinations |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes to answer study questions |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Identified eligible patients using computerized billing data Obtained data on date of birth, sex, race, and marital status Demographic characteristics similar between study groups |
| Methods | Study design: randomized trial Study duration: participants followed for 14 weeks; recruitment began 1 January 2013; study follow‐up ended 31 August 2013 Study aim: determine effectiveness of short message service reminders on immunization receipt | |
| Participants | Inclusion: women or caregivers were recruited into study soon after delivery or during third and seventh day visits after delivery of baby; must have cell phone and resident of Kadoma city Exclusion: no cell phone n = 304 | |
| Interventions | Intervention group: short message service reminders indicating next appointment date and health education; 7, 3, and 1 day before immunization due date; repeated for 6‐, 10‐, and 14‐week appointments; message indicated immunization protects your child against deadly diseases, and reminder of vaccination appointment; n = 152 Control group: informed mothers or caregivers about next scheduled immunization visit and provided routine health education; n = 152 | |
| Outcomes | Receipt of scheduled vaccines at 6, 10, and 14 weeks 6 weeks OPV1, Penta1 and PCV: 97% versus 82%; 15 percentage point difference 10 weeks OPV2, Penta2 and PCV2: 96% versus 80%; 16 percentage point difference 14 weeks OPV3, Penta3 and PCV3: 95% versus 75%; 20 percentage point difference | |
| Notes | Immunizations may have been measured at the date due or day after | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Participants were assigned by computer‐generated random numbers to intervention and control groups |
| Allocation concealment (selection bias) | Low risk | Participants were assigned by computer‐generated random numbers to intervention and control groups |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding of outcome assessment not specified Entered and analyzed data in Epi Info 7 |
| Incomplete outcome data (attrition bias) | Low risk | 152 participants randomized and analyzed in each group; obtained outcomes by telephone follow‐up and clinic immunization registry; compared data sources |
| Selective reporting (reporting bias) | Low risk | Reported pre‐specified outcomes of interest |
| Other bias | Low risk | Did not identify other sources of bias |
| Baseline measurement | Low risk | Reported characteristics of mothers; similar for marital status, place of residence, educational levels, employment status, religion, and median age |
| Methods | Study design: randomized trial Study aim: evaluated effect of patient and provider reminders on immunization rates and other preventive services | |
| Participants | Inclusion: patients with recorded telephone number, at least 1 clinic visit within 18 months of study, 40 to 60 years of age, and house officer or general medicine fellow assigned as primary physician Exclusion: residence in nursing home or long‐term care psychiatric facility | |
| Interventions | Intervention group 1: mailed memo to patient, and physician reminder clipped to chart; individualized patient reminder, specified which preventive services were needed and when they should be obtained Preventive services included: blood pressure check, dental exam, ocular pressure measurement, stool exam for occult blood, influenza, pneumococcal and tetanus vaccinations, mammogram, and Papanicolaou smears; n = 350 | |
| Outcomes | Immunization rates for Intervention group 1: Pneumococcal: 0.8 percentage point increase over control group Tetanus: 8.2 percentage point increase over control group Influenza: 16 percentage point increase over control group | |
| Notes | Multiple interventions, including patient and provider reminders "Limited and variable follow‐up times" for outcome measures because intent was to complete study within a 12‐month period with same group of house staff Number of patients not meeting inclusion criteria was higher than expected; this limited power to detect differences in outcomes between study groups | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Used computerized clinic database to select eligible patients; "they were randomly assigned to three study groups"; specific method not described |
| Allocation concealment (selection bias) | Unclear risk | Specific allocation procedure not specified; participants potentially meeting eligibility were selected using computerized clinic database |
| Blinding of participants and personnel (performance bias) | Unclear risk | Study hypothesis "was not revealed" to physicians; physicians received reminders in groups 1 and 2; blinding not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified Reviewed outpatient medical records Re‐interviewed 20% random sample of each study group to identify whether preventive services were obtained at non‐affiliated clinics, specifically for dental and ophthalmologic services |
| Incomplete outcome data (attrition bias) | Unclear risk | Obtained outcome data by reviewing full outpatient medical records at least 4 months after telephone interview to assess whether services were obtained within medical clinic, other clinics, or emergency department |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included outcomes for all 8 preventive services, including 3 immunization types |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Used patient telephone interviews and clinic chart reviews to develop individualized schedule of preventive care services for each participant; if patient's belief about whether a service had been received differed from chart, based recommended need for service on patient's recall Obtained demographic data for all eligible patients, including age, sex, race, and distance from medical center Study groups similar in need for preventive services and demographic characteristics |
| Methods | Study design: randomized trial Study duration: interventions conducted between 12 May 2010 and 19 July 2010; assessed receipt of vaccinations at 4 weeks and 1 year after randomization Study aim: evaluate whether adolescent immunization rates can be increased by calling parents or guardians, or parents or guardians and adolescents | |
| Participants | Inclusion: billing codes for physical exam at adolescent practice within 3 years prior to 13 May 2010; not received MCV4; not received Tdap in past 5 years; or received only 1 VAR, but did not have documented history of chickenpox Age: 13 to 17 year olds Exclusion: in custody of Department of Children and Families or Department of Youth Services; having sibling enrolled in study; or having no record of any immunizations or only influenza vaccinations Setting: Adolescent Medicine Practice at Boston Children's Hospital, Boston, Massachusetts (USA) n = 424 allocated; 142 to control; 141 to parent reminder only; and 141 to parent and adolescent reminder; 1099 assessed for eligibility; excluded 675 | |
| Interventions | Intervention group 1: telephone calls to parent or guardian only, indicating adolescent was overdue for immunizations; study investigator made calls and used telephone script to briefly describe vaccine‐preventable illnesses, inquire about immunizations received in other locations, and offer to schedule vaccination appointment Up to 4 call attempts were made in 1 week until content was delivered or parent asked not to be contacted; did not leave voicemail messages Made telephone calls between 9 am and 7 pm on weekdays only Medical interpreters were used, when necessary; n = 141 Intervention group 2: telephone calls to parent or guardian and adolescent, indicating adolescent was overdue for immunizations; similar script; parents were asked permission to contact adolescent; n = 141 Control: no specific outreach regarding immunizations; usual care; n = 142 | |
| Outcomes | Used intention‐to‐treat analysis Primary outcome: new record of 1 or more of the 3 vaccines of interest, Tdap, MCV4 or VAR within 4 weeks after the first phone call attempt Secondary outcomes: receipt of 1 or more of 3 vaccines within 1 year after the intervention; receipt of any other vaccines by 4 weeks or 1 year after the Group 1 intervention: 7.4 percentage point increase over control group, for 1 or more of 3 vaccines within 4 weeks; 14.4% versus 7.0% Group 2 intervention: 7.5 percentage point increase over control group, for 1 or more of 3 vaccines within 4 weeks; 14.5% versus 7.0% | |
| Notes | Only reached 30 adolescents by telephone in Group 2 intervention Power calculations: a priori power calculations indicated 174 participants were required in each group to detect "15%" difference between groups with 80% power; study group sizes did not meet this estimate; post‐hoc power calculations indicated that actual participant numbers and data provided enough power to detect "12%" difference between outcomes for intervention and control groups | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomization software was used to develop randomization assignment lists; "assignments were designated in randomly permuted blocks of 6 or 9" |
| Allocation concealment (selection bias) | Low risk | Randomized adolescents using randomization software |
| Blinding of participants and personnel (performance bias) | Unclear risk | "trial was not blinded to investigators"; however, it is not clear whether the outcome could be influenced by lack of blinding |
| Blinding of outcome assessment (detection bias) | Low risk | "Immunization status was assessed by using … electronic medical record"; outcome measurement is not expected to be influenced by lack of blinding by investigators |
| Incomplete outcome data (attrition bias) | Low risk | Assessed immunization status using hospital's electronic medical record Reviewed immunization records at 4 weeks and 1 year after intervention to determine vaccination status During telephone calls, asked parents whether immunizations were received at different location; if so, asked parents to have records mailed or faxed to practice Changed script during study to ask all parents to have records of immunizations received in other locations mailed or faxed to practice Did not reach 269 of 424 participants at 4 weeks and 270 of 424 participants at 1 year (Figure 1) |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included outcomes for all 3 immunization types |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Reviewed hospital billing and immunization databases to identify adolescents with physical exam billing code at adolescent practice within 3 years before 13 May 2010, and met eligibility criteria Study groups were similar with respect to age, sex, race and ethnicity, insurance type, and vaccines needed |
| Methods | Study design: randomized trial Study aim: evaluate and compare effectiveness of telephone and letter reminders at improving influenza vaccination rates | |
| Participants | Inclusion: listed in active patient computer files of family medical center; high risk for influenza and complications; not yet received influenza vaccination in current season | |
| Interventions | Intervention group 1: mailed form letter using first class mail; letter emphasized influenza could pose serious threat because of certain health conditions, and patient's physician recommended influenza vaccination; signed by influenza vaccination director; n = 267 No appointment was needed; patients were informed of cost Intervention group 2: personal telephone reminder with same information as letter; added reference to each patient's diagnosis and physician; made up to 2 telephone call attempts, 1 during daytime hours and 1 in evening; used standard script to provide uniform information; n = 258 | |
| Outcomes | Number and percent receiving influenza vaccination | |
| Notes | Authors indicated outside efforts to encourage vaccination, such as local media promotions, may have influenced vaccination rates | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Eligible patients "were randomly assigned by computer to one of three groups" |
| Allocation concealment (selection bias) | Low risk | Identified patients from active computer files; randomly allocated participants to study groups by computer |
| Blinding of participants and personnel (performance bias) | Low risk | "To avoid bias, physicians at the Family Medical Center were not informed of the purpose or nature of the study." |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified; "clinic nurses used a standard form to keep a record of all patients who received their vaccination during the study period." |
| Incomplete outcome data (attrition bias) | Low risk | Clinic nurses collected immunization data for all patients during study period, using standard form |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included immunization outcomes for all 3 study arms |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Influenza immunization data not available prior to 1984; determined influenza immunization status at baseline for persons considered at high risk for influenza and related complications prior to randomization |
| Methods | Study design: group randomized trial Study duration: followed each infant for 9 to 12 months until 12 months of age; recruited from August to November 2012; cell phone reminder and recall occurred for 14 months, 2012 August through 2013 September Study aim: evaluated the effect of reminder‐recall intervention and primary care immunization provider training on routine immunization completion among infants | |
| Participants | Inclusion: age 0 to 12 weeks at first immunization visits; parents living in study communities Exclusion: no cell phone; infant died; left service area n = 605 | |
| Interventions | Intervention group 1: 2 cell reminder phone calls to child's parent or other contact person; made 2 and 1 day before immunization appointment; recall for missed appointments; n = 148 Intervention group 2: Primary Health Care Immunization Providers’ Training (PHCIPT); 2 days refresher training on theory and practice of immunization conducted for nurses, midwives, community health officers, and community health extension workers; 4 modules adapted from World Health Organization immunization training manual; not our intervention; n = 150 Intervention group 3: telephone reminder and recall with provider training; n = 147 Control group: usual care; no intervention; n = 150 | |
| Outcomes | Receipt of routine immunizations at 12 months of age Intervention group 1: 98.6% versus 57.3%; 41.3 percentage point difference | |
| Notes | Data not included in meta‐analyses | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomly selected 2 local government areas from urban and 2 from suburban area; used ballot system to allocate areas into 3 intervention and 1 control group; randomly selected 1 ward from each area and purposively selected 1 primary health services facility from each ward |
| Allocation concealment (selection bias) | Unclear risk | As above |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel was not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding was not specified Researchers and research assistants used paper‐based immunization data system; integrated data collection into health facility activities; collected data using immunization records and cards, and qualitative feedback from mothers in reminder‐recall group |
| Incomplete outcome data (attrition bias) | Low risk | 10 of 605 participants were lost to follow‐up |
| Selective reporting (reporting bias) | Low risk | Outcomes reported for study questions |
| Other bias | Low risk | Did not identify other sources of bias |
| Baseline measurement | High risk | Groups were similar with respect to mother’s age Groups differed for children’s mean age at first immunization visit, children’s sex, family type, birth order, family’s religion, maternal education, and place of delivery |
| Methods | Study design: randomized trial Study aim: evaluate effectiveness of simple vaccination reminder at increasing influenza vaccination | |
| Participants | Inclusion: active patients Exclusion: nursing home resident, allergy to influenza vaccine or eggs Age: at least 65 years | |
| Interventions | Intervention: postcard reminder; short message on 3‐inch by 5‐inch card, mailed in business envelope with physician's return address; message indicated flu season was coming, some people are at greater risk for influenza and complications, flu shots can decrease risks with minimal side effects, and it is needed each year; also provided instructions for where to obtain flu shots; n = 262 analyzed | |
| Outcomes | Percent of participants receiving influenza vaccination | |
| Notes | Eligibility criteria specify 65 years of age or older; however, introduction specified over 65 years of age 1 site had used mailed reminders in past Power calculations: number of patients was sufficient to detect vaccination increase from baseline of 30% to at least 45%, with 90% power | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Study personnel assisted each site with providing a roster of active patients for the study; eligible patients were randomly assigned to study groups by unspecified method |
| Allocation concealment (selection bias) | Unclear risk | Randomly assigned patients to study or control groups Allocation method not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Obtained outcome data from clinic records; process not described; questionnaires were mailed to patients to estimate compliance; blinding not specified |
| Incomplete outcome data (attrition bias) | High risk | Mailed follow‐up questionnaires to randomized patients to estimate compliance because many patients obtain influenza vaccinations outside study clinics; 77.1% of participants responded to vaccination status questionnaire |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; influenza vaccination rates are reported for intervention and control groups |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Unclear risk | Prior year vaccination rates obtained by questionnaire |
| Methods | Study design: randomized trial Identified target populations by late September 1989 in 2 intervention groups, and in December 1989 for control group physicians Study aim: evaluate effect of population‐based tracking system, postcard reminder, and immunization tracking chart on increasing influenza vaccinations | |
| Participants | Inclusion: patients active in private physician office setting affiliated with 1 teaching hospital; cared for at least once in physician's office within 2 years of study start | |
| Interventions | Intervention: postcard reminder and provider poster or chart; n = 3,604 Poster included 11‐inch by 17‐inch chart, displaying target population for each physician, the patient denominator; used chart to track percent of target patients immunized each week, over time | |
| Outcomes | Percent of patients receiving influenza vaccination Odds ratio 2.0, CI 0.67 to 5.93, adjusted for intra‐practice variation | |
| Notes | Randomized at practice or provider level, analyzed at patient level Data not entered in RevMan Potential for under‐reporting of vaccinations obtained at county health department because incomplete linkage of patients with primary care providers and inaccuracies in spelling of primary care clinicians' names in health department records Reported intervention costs | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Stratified 13 private practice groups based on estimated numbers of patients > 65 years in physicians' practices; randomized practices based on stratification; method not described |
| Allocation concealment (selection bias) | Unclear risk | Allocated 13 private practice groups to 3 study groups: 17 physicians to control group, 13 physicians to clinician poster group, and 15 physicians to postcard and clinician poster group; method not specified |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified; tracked immunization rates on physician‐specific posters in intervention and control practices; insufficient information to assess whether high risk or low risk |
| Incomplete outcome data (attrition bias) | Low risk | Outcome data collection procedures were extensive; specific number and pe cent of participants with outcome data not specified Tracked influenza vaccinations using computer‐generated billing codes in 4 provider groups Asked physicians and clinic staff to record all influenza immunizations given to persons 65 years and older and graph percent of target population on poster Study coordinators visited office personnel in intervention clinics approximately every 2 weeks during study period to verify that charts were updated All participating practices billed USD 8.00 administrative fee for influenza vaccination; used data to help determine number of vaccinations given Obtained immunization data from county health department at study end Did not record verbal reports of vaccinations received outside physicians' offices |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; influenza vaccination rates are reported for 2 intervention and 1 control group, stratified by type of practice |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Unclear risk | Baseline measurement not described Used computer‐generated patient lists to identify target population in some practices Used billing records and treatment files to identify patients not computerized |
| Methods | Study design: randomized trial Study aim: assess and compare effect of patient‐specific letters and appointment postcards on well‐child appointment show rates and immunizations | |
| Participants | Inclusion: newborn infants enrolled at clinic, but not those receiving well care from first author of paper | |
| Interventions | Intervention group 1: sent letter to parents 1 week before scheduled well‐care appointment patient‐specific and visit‐specific reminder letters designed using Health Belief Model; specified appointment date and time, and age‐specific interventions to be received by patients; n = 87 Intervention group 2: sent postcard reminder to parents 1 week before each scheduled well‐care appointment; only specified appointment date and time; n = 96 | |
| Outcomes | Number and percent receiving 3 DTP by 7 months of age | |
| Notes | Letters reminded patients of appointments and discussed several topics Postcards reminded patients of clinic appointment date and time only, but not specific immunizations needed | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Methods to randomize patients not specified |
| Allocation concealment (selection bias) | Unclear risk | Random allocation of patients; process not described |
| Blinding of participants and personnel (performance bias) | Low risk | "Medical group providers were blinded to study group assignment" |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified for chart auditing process |
| Incomplete outcome data (attrition bias) | Low risk | Charts were audited after patients completed study to determine "date of DTP immunizations received" |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Enrolled infants; interviewed mothers to obtain demographic and socioeconomic data Compared demographic data between study groups, differences not identified |
| Methods | Study design: randomized trial, stratified by age and diagnosis Study aim: evaluate and compare effectiveness of letters and informational brochure on increasing influenza vaccination among persons who had not received vaccine in prior year and at high risk of getting influenza or complications | |
| Participants | Inclusion: patients cared for in general medical clinic of 1 hospital; at high risk for influenza complications Exclusion: received influenza vaccination in previous year; living in nursing home; severe disabling mental, visual, or hearing impairment Defined high risk as: 65 years and older, or diagnosed with diabetes, chronic lung disorders, or chronic heart disorders | |
| Interventions | Intervention group 1: standard letter and informational brochure; developed using multi‐attribute utility‐based messages; sent to patients approximately 10 days before 2‐week special flu shot clinic in October; n = 66 Intervention group 2: augmented letter; added statement to standard letter that 70% of veterans from medical center were vaccinated last year; n = 57 Intervention group 3: augmented letter and informational brochure; n = 55 | |
| Outcomes | Number and percent receiving influenza immunization | |
| Notes | Control group includes patient reminder (standard letter), so no true control group Study participants had not received influenza vaccination in previous year, not general population Active influenza vaccination program had been operational in study setting since 1978; included sending mailed letters to patients at high risk of influenza, inviting them to receive vaccine | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Patients identified as high risk were stratified into risk groups and randomly assigned to one of 4 groups; allocation method not described |
| Allocation concealment (selection bias) | Unclear risk | Stratified high risk patients without history of influenza vaccination at the start of year 2 of larger study into those 65 years and older and less than 65 years, then randomly assigned to 1 of 4 groups; allocation process not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified Mailed vaccination status questionnaire to each participant to obtain outcome data; conducted second mailing and telephone follow‐up, if needed; also used clinic vaccination records |
| Incomplete outcome data (attrition bias) | Low risk | 83% of participants remained in study at end of intervention Compared self‐report of immunization with clinic records; 94% agreement between data sources |
| Selective reporting (reporting bias) | Unclear risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | High risk | Control group includes patient reminder (standard letter), so no true control group Study participants had not received influenza vaccination in previous year, so not general population |
| Baseline measurement | Low risk | All eligible patients had not received an influenza vaccination in the prior year |
| Methods | Study design: randomized trial Study duration: 3‐ to 4‐month follow‐up period; extracted baseline data from Medicaid billing files on 28 December 2010; during June 2011, reassessed vaccination status with claims files, including vaccinations administered through 30 April 30 2011 Study aim: evaluate effectiveness of recall letter, sent to parents of Medicaid beneficiaries, in improving immunization series completion among young children | |
| Participants | Inclusion: parents of children enrolled in Montana Medicaid; known not to have completed some vaccinations with routinely recommended series; birth dates from 2 December 2008 through 1 May 2009 Age: 19 to 23 months of age Exclusion: children known to have completed vaccination series; or home address outside Montana Setting: Montana Medicaid program and Montana Department of Public Health and Human Services; state‐wide (USA) n = 878 eligible for study participation; recall letter sent to 438 parents of eligible children | |
| Interventions | Intervention: sent 1 state‐generated recall letter to parents, reminding them to take their children to health services providers to receive missed vaccinations; did not list specific missing vaccinations; n = 438 Control: no letter sent from state; n = 440 | |
| Outcomes | Received all needed childhood vaccinations Intervention group: 4 percentage points over control group; not statistically significant | |
| Notes | 1865 children enrolled in Montana Medicaid were 19 to 23 months of age at the time of the study; of these 47% were not up‐to‐date with immunizations Individual practices may have delivered reminder‐recall interventions; 21% of respondents to survey of Montana Medicaid health services providers indicated use of immunization reminder or recall strategies Power calculations: if 250 participated in each study group, study had 99.9% likelihood of detecting statistically significant difference with 15 percentage point difference, or 72% likelihood with 6 percentage point difference | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "Using the Comprehensive Clinic Assessment Software Application random generator tool, … randomly assigned." |
| Allocation concealment (selection bias) | Low risk | Centrally allocated children enrolled in Montana Medicaid using random number generator tool |
| Blinding of participants and personnel (performance bias) | Low risk | Blinding of participants or personnel not specified; however, health services providers that administered vaccinations and submitted Medicaid claims were not involved with intervention or data collection |
| Blinding of outcome assessment (detection bias) | Low risk | Blinding not specified; used Medicaid billing records to determine vaccination receipt |
| Incomplete outcome data (attrition bias) | Low risk | Reassessed vaccination status 3 months after recall letter was sent using Medicaid claims and immunization registry data; health services "providers have up to 1 year to bill Medicaid for vaccines administered, so delays in billing for some vaccines might hide some differences in vaccination coverage between intervention and control cohorts." Missing outcome data are not expected to differ between groups |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes for each immunization type |
| Other bias | Unclear risk | Individual practices may have delivered reminder‐recall interventions; 21% of respondents to survey of Montana Medicaid health services providers indicated use of immunization reminder or recall strategies |
| Baseline measurement | Low risk | Extracted data from Medicaid billing records and web‐based immunization registry to determine whether children received all immunizations in vaccination series Study groups did not differ for age, sex, American Indian‐Alaskan Native classification, population density within county of residence, and number of missing vaccinations |
| Methods | Study design: randomized intervention study Study duration: assessed HPV vaccination status 3 months after mailing; 12‐month evaluation period; 13 February 2013 to 12 February 2014 Study aim: evaluated effectiveness of reminder letter on HPV vaccine 3‐dose series completion | |
| Participants | Inclusion: female members of health system for at least 1 year prior to study; received at least 1 dose of HPV4 during 3‐month period before 13 February 2013; valid address in membership file Age: 9 to 26 years when received first HPV4 dose Exclusion: more than 2 doses of HPV4; unresolved pregnancy; had not met the minimum HPV vaccine dosing intervals specified by ACIP; terminated health plan membership during evaluation period Setting: Kaiser Permanente Southern California Health Plan n = 12,205 | |
| Interventions | Customized reminder letter; 9th or 10th grade reading level; English and Spanish; indicating HPV4 immunization schedule, date of first dose and telephone numbers; encouraging follow‐up vaccination visits; sent to patients if 12 to 26 years and to parents if 9 to 11 years; 4 waves of mailings were scheduled quarterly; therefore, letters did not reach participants when a dose was due; n = 9760 Control group: usual care; author does not have information about reminder or recall systems used in individual clinical practices; n = 2445 System‐wide provider reminders in electronic medical records for intervention and control group | |
| Outcomes | HPV vaccine 3‐dose series completion Intervention group: 56.4% versus 46.6%; 9.8 percentage point difference | |
| Notes | Inconsistency in study method descriptions | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomization process not described; inconsistent description of allocation process; either 80% of eligible persons were randomly selected for intervention group and 20% for control group; or patients were randomized |
| Allocation concealment (selection bias) | Unclear risk | Randomization process not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants or personnel not described |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding of outcome assessment and outcomes data source(s) not described |
| Incomplete outcome data (attrition bias) | Unclear risk | Data source(s) and follow‐up not described Not all reminder letters were successfully delivered; some were returned as undeliverable (n = 388; 4%); intention‐to‐treat analysis |
| Selective reporting (reporting bias) | Low risk | Reported results for all study questions |
| Other bias | Unclear risk | Methods are not fully described |
| Baseline measurement | Low risk | HPV4 vaccination history, including 1 or 2 doses, and age, race and ethnicity, Medicaid insurance, and length of managed care membership were reported and similar across study groups |
| Methods | Study design: randomized trial Study duration: October to December 2010 Study aim: evaluate efficacy of registry‐based letter and telephone recall intervention on rates of pneumococcal conjugate vaccine (PCV7) | |
| Participants | Inclusion: all children included in immunization registry Age: 6 weeks to 22 months Exclusion: siblings of included participants; registry documentation of having received PCV7; duplicate registry record; moved; died Setting: primary care clinic of The Children's Hospital, Denver, Colorado; teaching clinic predominantly serving Medicaid beneficiaries and uninsured patients (USA) n = 1234; 610 intervention and 624 control participants | |
| Interventions | Intervention: letter and telephone call from vaccination registry; English‐Spanish letter; indicated new vaccine protected against some types of specified infections; recommended in children less than 2 years of age; letter signed by 11 attending physicians; instructed all clinic trainees about dosing schedule and indications for PCV7; research nurse made up to 6 telephone calls per participant, beginning 10 days after letter was sent; during daytime, weekend, and evening hours; asked parents questions about recall letter and gave information about PCV7; encouraged parents to make vaccination appointments for children; n = 610 Control: no intervention; clinic did not routinely contact patients by telephone or letter to remind them of appointment reminders or interventions; n = 624 | |
| Outcomes | Receipt of one or more doses of pneumococcal (PCV7) vaccine during 2‐month study period Intervention: 2.8 percentage points above control group; 23% versus 20.2% Intervention group, received reminder letter and call: 9.4 percentage points above control group; 29.6% versus 20.2% Used intention‐to‐treat analysis | |
| Notes | All attending physicians of the clinic agreed to immunize all children less than 24 months of age with PCV7, a new vaccine at time of study Abundant supply of PCV7 vaccine during study period Immunization registry in operation since May 1998 Power calculations: with estimated sample size of 1410, study would have 80% power to detect 5 percentage point difference in immunization rates between intervention and control groups with 5% significance level Difficult to contact intervention group | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "All children aged 6 weeks to 22 months were selected from an immunization registry database." Used Microsoft Excel 96 "to randomly assign subjects to study arms." |
| Allocation concealment (selection bias) | Low risk | Randomized children using Microsoft Excel 97; one child was randomly selected if eligible siblings |
| Blinding of participants and personnel (performance bias) | Low risk | "Attending physicians, trainees, nurses, and control subjects were blinded to subject group assignment." Blinded intervention participants to study objectives |
| Blinding of outcome assessment (detection bias) | Low risk | Blinded care providers to study group assignments Outcomes were obtained from documentation in the immunization registry, maintained by the clinic; data are entered in the registry daily |
| Incomplete outcome data (attrition bias) | Unclear risk | Entered vaccinations in registry each day Registry error rate of 8% when reviewing small sample "There may have been underascertainment of immunization status because of underrecording in the registry or because patients obtained vaccinations at a site that was not captured by the registry." |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias Checked 40 charts to assess reliability of immunization registry data; 8% error rate with missing vaccination in registry; < 1% duplicate records |
| Baseline measurement | Low risk | Assessed PCV7 vaccination status at baseline Study groups similar for age, sex, insurance status, and immunization rates for the primary vaccination series |
| Methods | Study design: randomized trial Study aim: evaluate effectiveness of letter reminder and postcard recall intervention on influenza immunization rates among children with high risk conditions | |
| Participants | Inclusion: pediatric patients with high‐risk conditions, record in registry and billing database, and clinic visit to participating practices within 18 months When 2 or more siblings with high‐risk conditions in same household, randomly selected 1 child to participate | |
| Interventions | Intervention: staged reminder letter and postcard recall; letter strongly encouraged parents to have their children vaccinated for influenza; provided telephone number to schedule appointment; sent second reminder 4 weeks later to those not yet vaccinated, emphasizing that child may have a condition that increases risk for influenza infection; sent postcard to those not immunized 4 weeks after second letter, stating there was still time to vaccinate child; mailings used practice letterhead and were addressed to parents of participants; n = 920 | |
| Outcomes | Number and percent receiving influenza vaccination | |
| Notes | Authors mentioned that reminder‐recall intervention may have increased clinician awareness about influenza immunization; this may have increased vaccinations in control group | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Allocated patients within each study practice; used SAS software |
| Allocation concealment (selection bias) | Low risk | Participating practices used a common billing system and registry Participants were "assigned to intervention versus control groups by simple random allocation using SAS software" |
| Blinding of participants and personnel (performance bias) | Low risk | "We did not inform providers about which of their patients had been identified or recalled." |
| Blinding of outcome assessment (detection bias) | Low risk | Blinding not specified; however, providers were not informed about patient study group assignment, and outcome data were obtained using immunization registry and billing data |
| Incomplete outcome data (attrition bias) | Low risk | Obtained immunization data for each study participant using immunization registry and billing data, during March 2003 Telephoned randomly selected group of those not immunized to ask about influenza vaccinations at other locations, during April to May 2003 |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | Authors mentioned that reminder‐recall intervention may have increased clinician awareness about influenza immunization; this may have increased vaccinations in control group During November and December 2002, before study began, comparison of medical record data and registry immunization data revealed 14% of vaccines not entered or incorrectly entered in immunization registry |
| Baseline measurement | Low risk | In year prior to study, entered data about all children less than 72 months of age in study practices into regional immunization registry Compared demographic characteristics of intervention and control group participants; found to be similar for age category, insurance status, and percent up‐to‐date with immunizations by 24 months of age |
| Methods | Study design: randomized trial Study aim: evaluate effectiveness of patient postcard reminders and telephone recall interventions on increasing immunizations among young children | |
| Participants | Inclusion: children with record in immunization registry and not up‐to‐date with immunizations | |
| Interventions | Intervention: sent postcard reminder to parents, indicating child needed immunizations and parents should call clinic to schedule nurse‐only or physicians visit; re‐mailed postcards returned with forwarding address; called parents to obtain forwarding address if card returned without it; conducted telephone recall 1 month after postcard mailing if patient not seen or scheduled to be seen; made up to 4 telephone call attempts; n = 205 | |
| Outcomes | Number and percent up‐to‐date with immunizations; point estimates | |
| Notes | Follow‐up study to previous randomized trial that evaluated immunization reminder and recall; focused on addressing barriers identified in earlier study; no overlap in study participants between 2 trials Quality improvement initiative did not improve accuracy of parent contact information Other socio‐economic status barriers may have contributed to results Clinic had computerized database of immunization records, since May 1998, for all patients seen | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "subjects were randomized by simple random allocation" |
| Allocation concealment (selection bias) | Unclear risk | Participants randomized by "simple random allocation" to intervention and control groups Specific process not described |
| Blinding of participants and personnel (performance bias) | Low risk | Blinded staff and providers to study group assignment |
| Blinding of outcome assessment (detection bias) | Low risk | "Clinic staff and providers were blinded to study group assignment, and group allocation was not identified in the registry." Front office staff access the immunization registry; however, it is not clear who enters immunizations |
| Incomplete outcome data (attrition bias) | Unclear risk | Extent of follow‐up not explicitly specified Obtained immunization status at baseline for all participants by immunization registry and medical record review Asked parents about immunizations received outside of clinic to update records; then obtained medical record releases, faxed to outside clinics, and tracked Unable to reach 90 of 205 families by mail and telephone when conducting assessment of missed immunization opportunities Majority of immunization providers in area were not participating in registry at time of study |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | Immunization registry data found to have 8% error rate and duplicate record rate of less than 1% Brief study period of 2 months to get children up‐to‐date with immunizations may have been insufficient to achieve desired goals |
| Baseline measurement | Low risk | Obtained baseline data by immunization registry and medical record review for all participants Intervention and control groups were similar for age, sex, and prior clinic utilization |
| Methods | Study design: randomized trial Study aim: assess sustained effect of computer‐generated telephone and letter reminders on immunization coverage during first 2 years of life | |
| Participants | Inclusion: children who received first dose of diphtheria‐tetanus‐pertussis (DTP) or poliovirus (PV) vaccines; telephone numbers listed in computerized health department database Age: 60 through 90 days Setting: 4 public health clinics in Denver metropolitan area; tri‐county health jurisdiction; Denver, Colorado (USA) n = 1227 enrolled; 861 reached 24‐month follow‐up point at study end; 735 received full intervention during 22‐month follow‐up period | |
| Interventions | Intervention group 1: computerized telephone messages (autodialer) followed by letters; 1 autodialer reminder message prior to scheduled immunization date and up to 4 recall messages, 1 per week, over 4‐week period after due date; if no response, 1 letter was sent a week after fifth autodialer contact; sent second letter 1 week later, if needed Intervention group 2: computerized telephone messages (autodialer) only; 1 autodialer reminder message prior to scheduled immunization date and up to 4 recall messages, 1 per week, over 4‐week period after due date; made up to 9 attempts for each autodialer call, from 6 pm to 9 pm on weekdays, and noon to 8 pm on Saturdays Intervention group 3: letters only; up to 4 computer‐generated letters; sent first letter 2 days after scheduled immunization was missed; follow‐up letters were sent at 1‐week intervals Conducted all interventions from main office according to schedule Letter and autodialer messages were simple, indicating children were due for immunizations, immunizations are important, they prevent children from getting diseases that can make them very sick, and parents should make appointments or keep existing ones Delivered messages in English and Spanish according to specified preferred language Control: no notification | |
| Outcomes | Received all immunization in series at 24 months of age Group 1 ‐ autodialer and letter: 9.3 percentage points over control group Group 2 ‐ autodialer only: 8.4 percentage points over control group Group 3 ‐ recall letter only: 7.3 percentage points over control group Analysis based on families reached | |
| Notes | Data not entered in RevMan data tables | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Identified participants through the computerized health department database; randomization procedure not described |
| Allocation concealment (selection bias) | Unclear risk | 4 public health clinics in 3 counties had computerized databases that were linked to the main office; interventions were conducted from the main office; Randomized children within households; allocation method not explicitly described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified; clinic staff entered immunization due dates into computerized immunization records |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified; "Data were abstracted from the same computerized databases that were used to make decisions about scheduling of both immunization visits and the interventions associated with those visits." |
| Incomplete outcome data (attrition bias) | Unclear risk | At the end of clinic visits, entered immunization due dates in computerized immunization database; abstracted immunization data from database Of 1227 randomized children, 861 were 24 months of age by study end; followed 735 of 1227; "Study completion rates, however, did not differ by group or by demographic characteristics." Investigators did not attempt to obtain vaccination data at other sites |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes, including intention‐to‐treat and receipt analyses |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Study groups were similar for number of children in household, sex, and Medicaid insurance status Differences between study groups observed for ethnicity and language preference |
| Methods | Study design: randomized trial Study duration: November 2008 to February 2009 | |
| Participants | Inclusion: children with high‐risk chronic conditions living in 3 county local health department jurisdictions; currently or previously enrolled in Medicaid Age: 24 to 60 months Exclusion: children had already received influenza vaccination during fall 2008; ineligible for reminder‐recall notices through Michigan Care Improvement Registry because they opted out, moved, or died Setting: local health departments; 3 Michigan counties (USA) n = 3618 potentially eligible children were identified; after excluding ineligible children, 1372 were mailed reminder letters; 1358 were allocated to control group; total study sample = 2730; 2001 children with valid addresses were included in effectiveness analyses | |
| Interventions | Intervention: letter reminder; generated notices using Michigan Care Improvement Registry, statewide immunization information system with data on at least 95% of children up to 6 years; generated English‐language reminders during first week of November 2008; letters noted importance of annual influenza vaccination, especially for persons with chronic conditions, and encouraged parents to contact local health department or child's clinician; sent letters using first class mail with "return service requested" to help track undeliverable letters; n = 1372 Control: no reminder; n = 1358 | |
| Outcomes | Entered 1 or more seasonal influenza vaccination doses into Michigan Care Improvement Registry during follow‐up period, from November 2008 to February 2009 Intervention: 6.5 percentage point increase over control group Only included participants with valid addressed in analyses | |
| Notes | "The degree to which [children] received reminders from health plans or other providers during the study period is unknown." | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Sorted eligible children by random number within each of 3 local health department jurisdictions; allocated half to reminder and half to control group |
| Allocation concealment (selection bias) | Low risk | Identified children through Michigan Care Improvement Registry, a statewide immunization information system; sorted children by random number; immunization reminders were generated using the registry |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Randomized 3618 potentially eligible children; 2730 (75%) were included in study after excluding ineligible children; after randomization, 687 were excluded because they already received vaccination; 201 were excluded for other reasons, such as opted out of registry or were deceased; included 2001 children with valid addresses in effectiveness analyses, 55% of children randomized; 73% of included children; attrition is balanced between intervention and control groups for each reason |
| Selective reporting (reporting bias) | Unclear risk | Only children with valid addresses were included in the immunization reminder effectiveness analysis |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Identified participants using Michigan Care Improvement Registry; registry data were used to compare study groups for demographic characteristics and vaccination history, including receipt of any influenza vaccination dose during previous season No differences were reported in demographic characteristics between intervention and control groups |
| Methods | Study design: randomized trial Study duration: June 2008 to June 2009 Study aim: evaluate effectiveness of reminder‐recall strategies in increasing vaccination rates among children living in urban area | |
| Participants | Inclusion: not up‐to‐date for at least 1 vaccination for 7‐ or 19‐month recall study arms; turning 12 months of age during August 2008, regardless of vaccination status Age: 7 to 19 months Setting: local health departments in greater Detroit area, including city and surrounding area in Wayne County, Michigan (USA) n = 12,762 eligible; 10,175 analyzed; 2072 in 7‐month recall; 3502 in 12‐month reminder; and 4601 in 19‐month recall | |
| Interventions | Intervention group 1: letter intervention; recall of children not up‐to‐date at 7 months, indicating specific doses needed; n = 2072 Intervention group 2: letter reminder of all children aged 12 months, regardless of vaccination status; noted vaccinations due after first birthday; n = 3502 Intervention group 3: letter intervention; recall of children not up‐to‐date at age 19 months; n = 4601 All letters centralized using the Michigan Care Improvement Registry, a statewide immunization information system Control: no reminder letters; n = 3887, including 1014 for 7‐month recall, 1761 for 12‐month reminder, and 1112 for 19‐month recall | |
| Outcomes | Immunization activity: new dose administered within 60 days of any reminder‐recall cycle Group 1, 7‐month recall: 2 percentage points over control group; 35% versus 33% Group 2, 12‐month reminder: 1 percentage point over control group; 50% versus 49% Group 3, 19‐month recall: 3 percentage points over control group; 18% versus 15% | |
| Notes | — | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Allocated children using "automated group assignment process" |
| Allocation concealment (selection bias) | Low risk | Michigan Care Improvement Registry, a statewide immunization information system, was used to identify eligible children and send reminder and recall interventions; allocated children using "automated group assignment process" |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified; staff at the health departments mailed the reminder‐recall letters |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified; registry was source of intervention delivery and outcome data |
| Incomplete outcome data (attrition bias) | Low risk | State of Michigan law requires that all immunizations administered to persons younger than 20 years be entered in Michigan Care Improvement Registry Reported outcomes for 79.7% of randomized patients |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | "Unknown whether pediatric offices or other local providers independently sent reminder/recall notifications concurrently with this study" |
| Baseline measurement | Low risk | Used registry to identify eligible children Compared demographic characteristics between study groups, stratified by children's age at time of notifications Characteristics of children in 3 study groups similar for local health department jurisdiction and location of prior immunizations; observed differences in Medicaid enrollment between intervention and control group for 7‐month recall groups only |
| Methods | Study design: randomized trial Study aim: evaluate and compare 2 interventions used by school nurses to increase immunizations among children entering school | |
| Participants | Inclusion: children enrolled in schools that were located where child health screening was to be conducted during 1991 | |
| Interventions | Intervention: "active intervention"; telephone call, letter and brochure to parents; school nurses sent letter and brochure to parents of children with missed immunizations, informing parents that children needed immunizations; 1 to 2 months later, school nurse called parents to inquire about vaccination status and encourage parents to have children immunized if not completed; n = 120 Materials generally sent in English; also available in 15 other languages | |
| Outcomes | Number and percent immunized for measles, mumps and DTP | |
| Notes | — | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "For each school, the cards for children who appeared to have missed either the measles or before‐school boosters were randomized into two groups, the passive and active intervention groups." |
| Allocation concealment (selection bias) | Unclear risk | Randomized children to 2 groups; allocation procedure not described; school nurses sent passive and active intervention materials to parents and conducted telephone follow‐up; a research office from the Public Health Unit ascertained immunization status |
| Blinding of participants and personnel (performance bias) | Unclear risk | School nurses delivered the interventions; insufficient information to classify as low risk or high risk |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified; research officers obtained outcomes verbally from parents |
| Incomplete outcome data (attrition bias) | Unclear risk | 34% (41) were lost to follow‐up in passive intervention group, and 24 were initially misclassified at baseline and had actually been immunized; 25.8% (3) were lost to follow‐up in active intervention group, and 40 had been fully immunized at baseline but were misclassified Contacted parents by telephone to determine whether children had received immunizations Insufficient information to permit judgment of low risk or high risk |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | High risk | Not a true control group; control participants received all components of intervention except telephone call |
| Baseline measurement | High risk | Baseline immunization status was obtained by questionnaire for children in kindergarten; misclassification of immunization status was a problem for 24 in passive intervention group and 40 in intervention group |
| Methods | Study design: randomized trial; stratified using 4 criteria, including clinical office, whether woman older than 50 years of age was in household, whether all family members were active in practice, and whether family had health insurance Study aim: evaluate and compare manual health maintenance tracking system with computerized tracking system that generated patient reminders for all patients | |
| Participants | Inclusion: families active in practice, defined as seen in clinic within past 2 years Exclusion: patients living in group homes and those living outside practice area; families that could not be reached by telephone or did not return mailed questionnaire to obtain demographic data for all adult family members; transferred care to another practice; or charts could not be located | |
| Interventions | Intervention: telephone reminders to patients, computer‐generated health maintenance status report on chart and 2‐hour provider instruction session; n = 829 | |
| Outcomes | Provider compliance with 11 health maintenance procedures within protocol, including per cent of participants immunized for tetanus diphtheria Considered providers compliant if: procedure was documented as done, not indicated, offered but patient refused, or it was provided somewhere else Intervention group: 20 percentage point increase over control group | |
| Notes | Randomized families; data not entered in RevMan Study focused on 11 health maintenance procedures, with only one immunization measure: tetanus‐diphtheria immunization Other non‐immunization outcomes studied: tobacco use, blood pressure, weight, serum cholesterol, fecal occult blood test, physician breast exam, mammography, Papanicolaou test, teaching self‐examination, and teaching women to report post‐menopausal bleeding Control families received telephone reminders for health maintenance if requested by provider | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "Randomly assigned" families to intervention or control group within each of 32 strata based on 4 criteria; randomization procedure not described |
| Allocation concealment (selection bias) | Unclear risk | Randomized families to study groups; allocation procedure not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified; however, the statement "the system must allow providers to specify or cancel sending patient reminders as well as specify the month in which reminders will be sent" implies that providers were not blinded; information is not sufficient to assess whether low risk or high risk |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified for conducting chart audits |
| Incomplete outcome data (attrition bias) | High risk | Final chart audit was conducted at end of intervention Outcomes were defined as provider compliance with Td vaccination, as defined above; immunization rates were not reported |
| Selective reporting (reporting bias) | Low risk | Outcome data were presented for all 11 health maintenance procedures |
| Other bias | Unclear risk | Control families received telephone reminders for health maintenance if requested by provider Generated a list of guarantor numbers for each participating practice, randomly; investigators attempted to contact families by telephone or mailed questionnaire to obtain demographic data; families were not included if demographic data not obtained |
| Baseline measurement | Low risk | Manually audited intervention and control charts at baseline Baseline characteristics of intervention and control groups were compared; small differences were observed in health insurance coverage for office visits; other characteristics were similar at baseline |
| Methods | Study design: randomized trial; randomized 9 practices in 3 districts Sequentially enrolled children in each practice until met sample sizes Study duration: not clear; measured vaccination of infants at 10 and 14 weeks; enrolled between February and October 2014 Study aim: evaluate the effect of text message and sticker reminders on vaccination of children | |
| Participants | Inclusion: Kenyan districts with pentavalent 3 vaccine drop outs rates exceeding 10%; brought to selected health facilities in 3 districts for first dose of pentavalent vaccine; enrolled until sample sizes were reached Age: less than 12 months of age; media age 45 days; range of 31 to 99 days Exclusion: districts with high pentavalent vaccine coverage rates, geographically hard to reach, or security concerns; mothers did not have telephone number Setting: 9 practices providing vaccination in 3 districts; Kenya n: 1126 children assessed; 10 excluded; enrolled 1116 | |
| Interventions | Intervention group 1: short text messages reminding caretakers to return children for second and third doses of pentavalent vaccine 2 reminders from automated web‐based system 2 days before and on the day of second and third scheduled pentavalent vaccination due dates; Kiswahili and English; routine health education and advice on vaccinations; n = 372 Intervention group 2: stickers reminding caretakers to return children for second and third doses of pentavalent vaccine; not eligible intervention; n = 372 Control group: "no extra reminder messages"; next appointment date in a well‐child booklet; routine health education and advice on vaccinations; investigator contacted caretaker 2 weeks or more after immunization due date to determine reason for missed vaccinations; n = 372 | |
| Outcomes | Received scheduled pentavalent vaccines at 10 and 14 weeks Intervention group 1, 10 weeks: 98% versus 91% received pentavalent vaccine dose 2; 7 percentage point difference Intervention group 1, 14 weeks: 96% versus 83%; 13 percentage point difference | |
| Notes | Not sure vaccines were only counted vaccines if given on the exact due date | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Random sequence generation not specified |
| Allocation concealment (selection bias) | Unclear risk | Randomization process not specified |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding not specified; randomized practices |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Data collection process not clearly described; data possibly obtained by caretaker questionnaire or clinic records; data not obtained if immunizations obtained at other facilities |
| Selective reporting (reporting bias) | Low risk | Outcomes reported for study questions |
| Other bias | Unclear risk | Not able to determine if other sources of bias because methods not fully described |
| Baseline measurement | Low risk | Study groups were similar for demographic characteristics of caregivers and children |
| Methods | Study design: randomized trial Study duration: 1 February 2004 through 31 May 2006; children monitored through 15 months of age Study aim: evaluate multi‐step reminder‐recall and case management intervention on childhood immunization rates | |
| Participants | Inclusion: newborn infant in which family was planning to receive care at one of 3 participating clinics; infant birth weight greater than 1500 grams Age: infants from birth to 15 months of age Setting: Denver Health Medical Center and 3 of its affiliated community health centers predominantly serving socioeconomically disadvantaged populations, many of which are Hispanic; Denver Health is a vertically integrated community health center system (USA) In 2005, 90% of patients less than 15 months of age and served by these 3 clinics were eligible for Medicaid n = 811 infants; 409 intervention; 402 control | |
| Interventions | Intervention: stepped intervention of case management or patient navigators, telephone reminders, telephone and postcard recall, and home visitation; initially case managers or patient navigators contacted mothers using scripts, in hospital, by phone, or home visit, to identify barriers to care and risks for under‐immunization; mothers were provided with refrigerator magnet with care manager contact information, an immunization schedule, and bag of educational materials; intervention progressed in steps, depending on response from families; n = 409 Step 1: language‐appropriate reminder postcards sent 10 days before each well‐child visit Step 2: mothers received telephone reminder 10 days before each well‐child visit and postcard and telephone recall intervention for each missed well‐child visit or immunization 10 and 21 days after overdue Step 3: infants missing well‐child visits or behind on immunizations received intensive outreach and home visits 30 days after overdue; calls and home visits were made by Master's prepared patient navigators; conducted outreach conduct on evenings, weekdays, and weekends Control: not specified; n = 402 | |
| Outcomes | Outcome 1: primary, continuous number of days under‐immunized in first 15 months of life; ineligible outcome Outcome 2: received all needed childhood immunizations at 15 months of age: 2 pneumococcal; 4 DPT; 3 poliovirus; 1 MMR; 3 H. flu; 3 hepatitis B; 1 varicella; Outcome 3: influenza immunization rates: before and after without comparison group; ineligible study arm Results: Intervention ‐ Outcome 2: 11 percentage points above control group; 44% versus 33% Used intention‐to‐treat analyses | |
| Notes | Intervention intensity similar among 3 study clinics | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Allocated infants to groups using random blocks of 2, 4, or 6 infants, using numbered non‐translucent envelopes Randomization sequence generated and maintained by study personnel |
| Allocation concealment (selection bias) | Low risk | Used newborn nursery log to identify eligible infants; "Research assistants who were responsible for opening the envelopes and assigning the treatment arm were blinded to the randomization sequence"; "randomization sequence was generated by an analyst who was not otherwise involved in the study and it was maintained by the principle investigator, who was not involved in the actual random assignment of patients" |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants or clinicians not specified |
| Blinding of outcome assessment (detection bias) | Low risk | Blinding not specified; however, outcome data obtained from the Denver Health electronic immunization registry, a "system‐wide legal repository for pediatric immunizations" |
| Incomplete outcome data (attrition bias) | Low risk | Used Denver Health electronic immunization registry to obtain immunization outcome data; registry records pediatric immunizations throughout health system and captures an estimated > 97% of immunizations given in health system Only 1 of 409 intervention infants and 3 of 402 control infants excluded from analyses All children monitored through 15 months of age Patients not tracked after leaving Denver Health system |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Unclear risk | Used newborn nursery log to identify potentially eligible infants Obtained demographic and other data from medical chart review; downloaded billing and diagnosis data from Denver Health computer system Study participants were primarily covered by public insurance or uninsured, Hispanic, Spanish‐speaking, and urban families Intervention group had higher proportions of women with maternal alcohol use and tobacco use than women in control group; trends toward more illicit drug use and fewer Hispanic mothers in intervention group |
| Methods | Study design: randomized trial Study aim: evaluate effectiveness of customized family reminder letters on improving preventive services, including immunizations | |
| Participants | Inclusion: patients registered with medical practice for at least 1 year and had made at least 1 visit to the clinic in previous 2 years | |
| Interventions | Letters sent between September 1990 and March 1991 Intervention group 1: computer‐generated customized letters, reminding patients of needed preventive services using plain language in standardized format; mailed packet had cover letter and one page for each family member, describing preventive services that participants were eligible to receive; dates of previously obtained preventive services were listed on individualized letters; n = 613 patients within 204 families Intervention group 2: form letter to patients, which described all recommended preventive procedures for all ages and both sexes; dates of previously received services not included; n = 676 patients within 252 families | |
| Outcomes | Number and percent of overdue vaccines received: adult tetanus, influenza, MMR, Hib, DPT and TOPV; procedures, including immunizations, considered completed if documented as ordered by clinician; influenza vaccination stratified by age over 65 years and persons with chronic disease | |
| Notes | Medical center computerized since 1984 | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Randomly selected patients to participate using computerized patient registration numbers; after individual patients were selected, families of patients were randomized to study arms |
| Allocation concealment (selection bias) | Unclear risk | After individual patients were selected, families of patients were randomized to study arms; specific method not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | "not blinded in that physicians could be aware that a patient was a member of a family in the study if the patient mentioned that the family had received a letter" |
| Blinding of outcome assessment (detection bias) | Unclear risk | Collected outcome data from patient charts and encounter forms; blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Data available for 1971 of 1988 patients Data collected at 2, 4 and 6 months after letters were mailed by reviewing patients' charts and encounter forms Compared patient charts with encounter forms to assess accuracy of physician documentation of preventive services; error rates were measured; in a 5% sample, 3.7% of electronic patient records were missing documentation of 6 preventive services Outcomes were defined based on whether or not service was ordered; unclear whether ordered procedures were completed |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | Physicians could refuse to have letters sent to individuals Procedure was considered done when it was ordered |
| Baseline measurement | High risk | Collected data at baseline; differences in baseline measures were observed between study groups for proportion of procedures up‐to‐date, number of family members, and mean family age |
| Methods | Study design: randomized trial Study aim: assess whether telephone calls made by receptionists at a clinic increase influenza immunization uptake | |
| Participants | Inclusion: registered patients without chronic disease Unit of allocation: household Exclusion: patients with chronic disease, including asthma, diabetes, chronic obstructive pulmonary disease, ischemic heart disease, and renal disease | |
| Interventions | Intervention: telephone call to patient during a 2‐ week period between 25 September 2000 and 6 October 2000; receptionist made up to 2 telephone calls at different times of day to patients; receptionists were provided with information sheets and suggested invitation language; however, they were not trained on how to deliver the intervention; n = 660 individuals within 605 households | |
| Outcomes | Receive of influenza immunization Immunization uptake varied by practice | |
| Notes | Reported differences as "percent" changes rather than percentage point changes; this may be a reporting error Allocated households; adjusted OR reported in paper showed minimal effect of this allocation approach Included data in RevMan data tables Only 60% of intervention households were reached by telephone A national television campaign occurred during September 2000 to promote influenza vaccination Participating practices had conducted influenza immunization recall in past Practices use EMIS computer system for clinical and administrative documentation Measured and reported some costs of intervention | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Used STATA to allocate list of households to study groups within in each practice |
| Allocation concealment (selection bias) | Low risk | Each practice identified registered patients, ages 65 to 74 years; study coordinator used STATA to allocate list of households to study groups within in each practice |
| Blinding of participants and personnel (performance bias) | Low risk | Nurses who worked in immunization clinics were "unaware of the household allocation to control or intervention group"; "immunization is almost exclusively done by appointment at clinics run by practice nurses" Receptions, who made the telephone reminder calls, were not blinded |
| Blinding of outcome assessment (detection bias) | Low risk | Practice nurses recorded immunizations in practice computer system; nurses were unaware of household allocation |
| Incomplete outcome data (attrition bias) | Low risk | Obtained immunization outcome data for all 1261 participants |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Process of obtaining baseline characteristics was not reported; baseline patient characteristics were similar between study groups, and patient characteristics were included in statistical modeling, including age, sex, household size, and practice |
| Methods | Study design: randomized trial Study duration: 2003 to 2004 Practices were recruited during summer 2002; intervention began on different dates starting between 29 September 2003 and 13 October 2003 Intervention ended 22 January 2004 | |
| Participants | Inclusion: all active patients of participating primary care centers; residents of New York; "active" was defined differently for different practices, but included at least 1 visit to practice in past 2 to 5 years Age: 65 years and older Exclusion: patients who had received influenza vaccination before intervention began Setting: 6 of 7 large urban primary care practices that serve large proportion of Rochester, New York's African American and Hispanic older adults agreed to participate (USA) Practices included 2 internal medicine neighborhood health centers, 2 family medicine neighborhood health centers, 1 internal medicine hospital clinic, and 1 internal medicine ‐ pediatric practice n = 3752 patients were randomized; 1748 intervention and 2004 controls | |
| Interventions | Intervention: combination of patient tracking and reminder‐recall, outreach, and provider reminders; n = 1748 Step‐wise practice‐based intervention; patient tracking; provider reminders using bright‐colored sheet with reminder stating "REMEMBER, This patient needs influenza vaccine"; patient recall using letter or card; outreach to patients by telephone; transportation assistance was offered; 1 patient was vaccinated by a home visit Control: standard of care was based on each office routine; 1 practice reported sending some form of notification regarding the influenza vaccination to patients; n = 2004 All: community‐based campaign; enhanced vaccine delivery through non‐traditional venues | |
| Outcomes | Number and percent receiving influenza vaccination during study period Intervention: 42 percentage points over control group | |
| Notes | Sample size calculations were conducted; at least 170 patients per study group were needed to demonstrate at least "15%" difference in vaccination rates with control rates of 50%; enrollment exceeded these requirements 7 control participants (0.35%) were contacted by telephone or mail by mistake | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Randomized patients using last digit of social security number; odd numbers were allocated to intervention group; even numbers were allocated to control group |
| Allocation concealment (selection bias) | High risk | Downloaded patient names and demographic data from individual primary care centers' patient information systems into study site‐specific database; randomized patients using last digit of social security number; odd numbers were allocated to intervention group; even numbers were allocated to control group |
| Blinding of participants and personnel (performance bias) | Unclear risk | "Use of patient reminders/recall precluded blinding of either patients or outreach workers, and use of provider prompts precluded blinding PCC staff"; however, health services "providers tended to be unaware of group assignment for an individual patient except during health‐care visits if the patient chart included a provider prompt" |
| Blinding of outcome assessment (detection bias) | High risk | Outreach workers conducted intervention and reviewed medical records for influenza immunization status |
| Incomplete outcome data (attrition bias) | Low risk | Assessed all participants for vaccination status; none lost to follow‐up Reviewed charts 2 months after study end Performed quality assessment checks with "extremely high accuracy" |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Collected patient demographic data according to processes approved by each primary care center's institutional review board Intervention group had higher proportion covered by Medicare and higher proportion of males than control group Groups similar for race and ethnicity |
| Methods | Study design: randomized trial After randomization, children remained in the study for 6 months Study aim: evaluate effectiveness of registry‐generated patient reminder and recall postcards on childhood immunization rates | |
| Participants | Inclusion: made at least 1 visit to inner city practice network and due or late for DTaP Payer mix: approximately 85% of visits covered by Medicaid | |
| Interventions | Postcards were registry‐generated with photograph of a baby; each postcard had a standard bilingual English or Spanish message indicating need for immunizations and encouraging parents to call the clinic to make an appointment Intervention group 1: continuous reminders; weekly postcards; n = 549 | |
| Outcomes | Up‐to‐date with DTaP; analysis based on intention‐to‐treat | |
| Notes | Network of practices did not previously have reminder systems in place Postcards were returned for 13.6% children 25.6% of children were misclassified as due or late for a DTaP dose and were sent reminders that were not needed One in 6 children not reached because of incorrect addresses One in 6 children not vaccinated because of missed opportunities | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Programmed EzVAC, a provider‐based registry, to identify eligible children every week based on need for DTaP vaccine, randomly sample 12% of those eligible, and then randomly assign those to 1 of 3 study groups |
| Allocation concealment (selection bias) | Low risk | Randomly selected and randomly assigned children to study groups within EzVAC, a computerized system |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Immunizations are entered in EzVAC, a provider‐based registry, at the point of service Blinding was not specified |
| Incomplete outcome data (attrition bias) | Low risk | Used EzVAC registry every week to identify if participants needed repeated reminder Outcomes were measured at 3 and 6 months after randomization Tracked immunizations in EzVAC; for children who were not up‐to‐date in EzVAC, NY Citywide Immunization Registry was checked for out‐of‐network received immunizations (16%) |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | 29 children were recorded as not having received the vaccine because of a vaccine shortage; for these children, investigators simulated the vaccine as being given on date ordered Misclassification rate for DTaP dose was 30% |
| Baseline measurement | Unclear risk | Checked data in EzVAC registry to determine immunization needs at baseline; 25.6% were sent false reminders because they were misclassified as being due or late for a DTaP dose; misclassified children were distributed evenly across study groups |
| Methods | Study design: randomized trial Study aim: evaluate effectiveness of immunization recall for young children served by an urban teaching clinic | |
| Participants | Inclusion: seen for well‐child care or acute illness in clinic Clinic population: 63% covered by Medicaid or a state‐subsidized insurance program; 21% commercial insurance; 16% uninsured; highly transient with at least 50% of families changing addresses or telephone numbers each year | |
| Interventions | Intervention: postcard and attempts to call; provider prompts with child's immunization record attached to front of child's chart Postcard indicated that children needed immunizations and asked parents to call to schedule a visit; provided telephone number; postcards remailed if returned with updated address; up to 4 call attempts were made 2 weeks after postcards were mailed; n = 294 | |
| Outcomes | Received all needed immunizations at 7, 12, and 19 months of follow‐up | |
| Notes | Used computerized database for immunization records | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomization process not described |
| Allocation concealment (selection bias) | Unclear risk | Used computerized immunization database to identify eligible children; randomly assigned children who were not up‐to‐date with immunizations; allocation procedures were not explicitly described |
| Blinding of participants and personnel (performance bias) | Low risk | Blinded clinic providers to study group assignment |
| Blinding of outcome assessment (detection bias) | Low risk | Per clinic policy, nurses enter immunization data directly into the computerized database at the time of administration or shortly after; this process also occurs for historical records received by the clinic; providers were blinded to children’s study group assignment |
| Incomplete outcome data (attrition bias) | Unclear risk | Charts for 2 children in intervention group and 5 in control group were not available for outcome review Authors report incomplete immunization records Unable to contact 28.1% of intervention group participants Inadequate immunization records were reported for approximately 18% of participants |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | At baseline, determined immunization status for 742 children using clinic database and medical records; children who were up‐to‐date with immunizations were excluded |
| Methods | Study design: randomized trial Study aim: achieve universal immunization of 6‐ to 21‐month old children against influenza during the 2003 to 2004 influenza season; evaluate effect of reminder or recall letters on immunization receipt | |
| Participants | Inclusion: children visiting practices during the previous 18 months and had a record in regional immunization registry Exclusion: children with chronic medical condition, died, or there was documentation that family moved to non‐participating practice | |
| Interventions | Intervention group: up to 3 reminder or recall letters were generated by immunization registry; first reminder letter was sent in October 2003 to all intervention participants; second recall letter was sent during November 2003 to those not vaccinated; letters indicated that providers were recommending annual influenza vaccinations for all children 6 to 23 months of age and for children of parents receiving letters; letters also noted how to schedule an appointment; letters for some practices provided information about special walk‐in or influenza vaccination clinics; n = 2595 2 of 5 practices sent third recall letter in December 2003 | |
| Outcomes | Receipt of one or more influenza immunizations during the 2003 ‐ 2004 season | |
| Notes | — | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "Used random allocation of subjects stratified according to practice site" to distribute participants equally to intervention and control groups for each practice; randomization method not described |
| Allocation concealment (selection bias) | Unclear risk | Randomly allocated participants, stratified by practice site, to include equal numbers of control and intervention participants at each site; method of randomization not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | All participating practices used the regional immunization registry and shared a common computerized billing system Both administrative and registry data were used to assess whether an influenza vaccination had been given Staff members are expected to enter immunizations given into registry within 24 hours after administration Quality assessment of 30 charts per practice, comparing registry data to medical record data, showed 97.4% completeness of children in practice that were in the registry Reported error rate of 7.2% in the immunization registry |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | High risk | 3 of 5 practices were experiencing an influenza vaccination shortage in their offices, so did not send third letter to intervention participants Conducted telephone survey August to October 2003 to describe characteristics of study practice populations and assess parents' attitudes and intentions for influenza immunization; contamination may have attenuated observed effect A pandemic was occurring with extensive media coverage |
| Baseline measurement | Low risk | Infants were enrolled, so prior influenza vaccination histories were not generally applicable Intervention and control group participants were similar for age, sex, and insurance coverage |
| Methods | Study design: randomized trial Study aim: assess whether computer‐generated reminder letters improve influenza immunization receipt among children seen at urban teaching clinic | |
| Participants | Inclusion: received primary care at 1 children's clinic; had 2 or more emergency or clinic visits in past year for asthma | |
| Interventions | Intervention: 1 computer‐generated letter to parent and standing order for influenza immunization; n = 43 Letter included: child's name and address; reason for immunization; need for 2 shots for children younger than 9 years of age, at least 1 month apart; request to bring letter to clinic so immunization could be given without an appointment or without seeing physician; signed by clinic's medical director | |
| Outcomes | Number and percent of children immunized with influenza vaccine | |
| Notes | During October 1991, memo sent to care providers, reminding them about influenza vaccination recommendations Nurses could give influenza immunizations without individual physician order Relatively small sample size; power calculations not reported During fall 1991, local media launched campaign to inform public about dangers of influenza and need for vaccination | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Used computer system to randomly assign patients to intervention and control groups |
| Allocation concealment (selection bias) | Low risk | Clinic‐based computer system generated list of eligible patients, randomly assigned them to intervention or control groups, and generated personalized reminder letters for intervention group participants |
| Blinding of participants and personnel (performance bias) | High risk | "Parents were asked to bring the letter to clinic so the immunization could be given without an appointment and without having to see a physician"; this implies lack of blinding of participants and personnel |
| Blinding of outcome assessment (detection bias) | Low risk | Research assistant, blinded to study group assignment, reviewed medical records for influenza immunization status |
| Incomplete outcome data (attrition bias) | Low risk | Research assistant reviewed medical records for each participant to obtain number of influenza vaccinations received for each child |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Generated list of eligible patients from clinic‐based computer system Intervention and control groups similar for sex, age, and number of visits because of asthma in year prior to study Baseline data not reported for prior year influenza vaccination status or overall immunization status |
| Methods | Study design: randomized trial Study aim: evaluate and compare effectiveness of postcard reminders with different messages on improving influenza vaccination rates | |
| Participants | Inclusion: patients at high risk for serious complications from influenza infection based on age over 65 years or diagnosis of chronic heart disease, bronchopulmonary disease, renal disease, or diabetes mellitus Data were collected on 283 participants | |
| Interventions | Intervention group 1: neutral postcard mentioned influenza vaccine now available; listed telephone number for nurse appointments; addressed to "Dear Patient"; n = 68 Intervention group 2: health belief model postcard, emphasizing severity of influenza, susceptibility of at risk persons to influenza, and benefits of vaccination; addressed to "Dear Patient"; n = 70 Intervention group 3: personal postcard; addressed to patient's name and signed by clinician; postcard mentioned that influenza season is approaching and recommended the patient come in for flu shot; it listed telephone number to call and make appointment with nurse; n = 61 | |
| Outcomes | Percent vaccinated for influenza | |
| Notes | Study timeframes unclear | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomization process not described |
| Allocation concealment (selection bias) | Unclear risk | Eligible participants were identified based on diagnostic codes stored in the family medical center’s computer; randomly assigned patients to one of 4 groups; allocation process not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding of outcome assessment not specified; data collection "occurred either when study patients came to the FMC for vaccination or ….when they were called and interviewed by phone." |
| Incomplete outcome data (attrition bias) | Unclear risk | Data collection occurred when intervention patients came to clinic during study period In mid‐December patients were called and interviewed by telephone; control participants were asked if they had received influenza vaccination Obtained vaccination status by self‐report for large proportion of participants because nearly two‐thirds of clinic patients are vaccinated at other varied sites Completed follow‐up on 71.6% of persons initially selected and randomized, and on 92% of persons remaining in study |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | High risk | Baseline demographic data obtained when patients were assigned to study groups Health Belief Model and Personal postcard groups had more patients that had received influenza vaccinations in past year or anytime in past 5 years than control group or neutral postcard group Treatment groups similar for age, sex, prior history of influenza, adverse reaction to vaccine, diagnostic classification, and clinic utilization rates |
| Methods | Study design: controlled before and after Study aim: assess the effect of interventions delivered by community‐based organization on immunization rates | |
| Participants | Inclusion: children in Fulton County; patients of 4 public clinics or residents of one of 9 lower socioeconomic communities 2 clinics served predominantly African American populations, and 2 served predominantly Hispanic populations Allocated clinics to 1 intervention and 1 control clinic for each ethnic‐race category Communities consisted of 6 public housing communities with primarily African American populations, and 3 private housing communities Allocated public housing communities to 3 intervention and 3 control groups Allocated private housing communities to 2 intervention and 1 control group | |
| Interventions | Intervention group 1: "clinic" group; telephone, mail or home visit with family Usually contacts with families were made by telephone or mail, but home visits were made when necessary; n = 2 clinics Intervention group 2: "community"; door‐to‐door campaign to identify under‐vaccinated children, provide immunization education, provide culturally sensitive promotional materials, and introduce them to services; these interventions were followed by a weekly mobile vaccination van or temporary on‐site vaccination stations, free child care, and incentives of food and baby products; ineligible intervention Interventions were applied for 1 year; n = 3 public housing communities and 2 private housing communities | |
| Outcomes | Age‐appropriate vaccination rates and series completion rates Controls: no change in immunization rates | |
| Notes | Data not entered in RevMan data tables Selection of intervention sites was based on the community‐based organization's ties and perceptions of intervention feasibility Organization participated in selection of control sites | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Allocation process was not random |
| Allocation concealment (selection bias) | High risk | Allocation occurred by community and practice; intervention clinics not located in control community; control clinics not located in intervention communities; allocation process was not randomized |
| Blinding of participants and personnel (performance bias) | High risk | Participants and personnel not blinded |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Georgia Department of Public Health district immunization officer visited each county clinic and reviewed health records of all children seen in clinic and who were 21 to 23 months of age at time of officer's visit In 2 intervention clinics, vaccination records were reviewed monthly |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | Selection of intervention sites was based on the community‐based organization's ties and perceptions of intervention feasibility Organization participated in selection of control sites |
| Baseline measurement | Low risk | Reported baseline measures; in 1992, age‐appropriate immunization rate for children 3 to 59 months was 44% overall, and for intervention and control arm participants General characteristics of populations served by clinics and communities were similar between intervention and control groups |
| Methods | Study design: randomized trial Study aim: evaluate effect of large‐scale, registry‐based reminder and recall intervention on childhood immunization rates in inner city population with history of low vaccination rates | |
| Participants | Inclusion: children residing in Fulton County; had received care through Fulton County health department clinics or public hospital health system; and born between 1 July 1995 and 6 August 1996 Children were identified in MATCH immunization registry | |
| Interventions | Intervention group 1: autodialer and postcard; autodialer reminder 7 days before dose was due from health department; repeated every 30 to 60 minutes if no answer or a busy signal; if contacts not successful, postcard was sent at least 5 days before vaccination due date; autodialer recall 6 days after due date if needed dose not in registry; autodialer was repeated on days 11, 17, and 23, if needed, followed by postcard on day 28; Spanish‐language option was available; n = 763 Intervention group 2: outreach; within 7 days after an immunization due date, an outreach worker attempted to reach the family by telephone; sent a postcard if no working telephone; a postcard was sent 7 days later, followed by a home visit 30 days later if a dose was missing; efforts continued monthly until contact was made with the family; n = 760 Intervention group 3: autodialer and outreach; see descriptions for each intervention above; n = 764 | |
| Outcomes | Age‐appropriate vaccination rates | |
| Notes | Power calculations: a sample size of 3050 was reported to provide 80% power to detect 5% differences in immunization rates between study groups Study sample had relatively high vaccination coverage at the start, with most children only needing 1 or 2 visits to complete vaccination series | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Allocated participants using computer‐generated random numbers |
| Allocation concealment (selection bias) | Low risk | Allocated participants using computer‐generated random numbers |
| Blinding of participants and personnel (performance bias) | Unclear risk | "We did not attempt blinding"; reminders and recall interventions encouraged participants to obtain immunizations from health provider; personnel conducting outreach were not health care providers |
| Blinding of outcome assessment (detection bias) | High risk | Blinding not attempted; "all intervention contact attempts and outcomes were recorded in a study database" |
| Incomplete outcome data (attrition bias) | Unclear risk | Entered vaccination data each day, in any Fulton County clinic, into electronic vaccination record; downloaded data weekly to MATCH immunization registry, which has vaccination records from the largest vaccination providers in Atlanta metropolitan area; Authors mention the possibility of registry inaccuracies Did not include non‐registry immunization records |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | Non‐automated recall postcards were sent to some participants in the control group |
| Baseline measurement | Low risk | At baseline, the intervention and control groups were "essentially identical" for demographic and vaccination characteristics |
| Methods | Study design: controlled before and after, with historical comparison and contemporaneous geographic comparison Study duration: 1‐year follow‐up for telephone reminders Study aim: determine causes of the low MMR immunization coverage rates in young children and evaluate effectiveness of telephone reminders and telephone reminders combined with home visits on improving childhood MMR immunization rates | |
| Participants | Inclusion: parents of children behind with MMR immunizations, defined as not receiving 2 MMRs by 2 years of age as of October 2007 to September 2008; born between October 2005 and September 2006 Age: children greater than 2 years of age Setting: Saskatoon Health Region, Saskatchewan; comparison in Regina Qu'Appelle Health Region, Saskatchewan (Canada) n = 911 were behind on at least 1 immunization | |
| Interventions | Intervention group 1: telephone reminder; n = 115 in one subgroup analysis Intervention group 2: telephone reminder and offer to have a public health nurse give vaccinations during a home visit; n = 142 in one subgroup analysis Control for group 1: no telephone reminder; "without enhanced intervention" Control for group 2: telephone reminder | |
| Outcomes | Number and percent received MMR immunization by 24 months of age Group 1: pre‐intervention to post‐intervention increase by 6.6 percentage points in intervention group and 2.7 percentage points in comparison region | |
| Notes | Data not entered in RevMan data tables In Saskatchewan, children are recommended to receive 2 MMR vaccinations by 18 months of age; incomplete coverage is defined as fewer than 2 MMR immunizations by 24 months of age Intervention 1 compared with control group Intervention 2 compared with intervention 1 Historical comparison: used 5‐year average | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Regions were not randomized for eligible intervention; "block randomization through using computer allocation was used to divide the six neighborhoods into two blocks" for ineligible intervention |
| Allocation concealment (selection bias) | High risk | Intervention group was compared with a control health region |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified; this study was conducted in a health region; it is not clear whether clinicians were involved with or aware of the study |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Survey: of 911 children who were behind on at least 1 immunization; 787 (86%) of parents or guardians could not be contacted by telephone; however, 629 of 787 agreed to participate in the survey (69%) Extent of outcome data not clear for telephone reminder |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Saskatchewan Immunization Management System combines vital statistics and health insurance information to identify children that have not received recommended vaccinations for their respective age |
| Methods | Study design: randomized trial Study aim: evaluate effectiveness of computer‐generated recall letters and immunization tracking system on childhood immunization rates | |
| Participants | Inclusion: enrolled children at 2 medical centers, Santa Clara and Santa Theresa medical centers Exclusion: patients with gap in health plan membership between 12 and 19 months of age | |
| Interventions | Intervention: personalized letter and brochure; in English and Spanish; letter indicated that Kaiser's record showed that the child was overdue for an immunization, and parent should call clinic to make appointment for preventive visit; printed on stationery of local medical center; generated and mailed by regional Division of Research to parents; brochure listed recommended immunizations; n = 172 randomized and 153 analyzed | |
| Outcomes | Number and percent of MMR vaccinations recorded in the Kaiser immunization tracking system, or parental report of MMR received outside the system, between 20 and 24 months of age Intervention group: 19 percentage point increase over control group | |
| Notes | No copayments within HMO for immunizations, although there were copayments for office visits for some HMO participants, up to USD 15 High literacy level in study population, based on a previous study in this population | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomized patients using a random number generator |
| Allocation concealment (selection bias) | Low risk | Eligible children were identified each month through a regional computerized immunization tracking system; randomized patients using a random number generator; recall letters were computer‐generated |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified; intervention letters were mailed by the regional Division of Research using letterhead from the individual clinics; the mailing included a "slip" that a parent could take to the injection clinic to obtain immunizations without an appointment |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified; immunization data were obtained from the Kaiser immunization tracking system |
| Incomplete outcome data (attrition bias) | Low risk | Primary analysis included immunizations recorded in Kaiser immunization tracking system Secondary analysis included immunizations recorded in tracking system and immunizations reported by parents in follow‐up survey; 22 families of 160 families whose children had not received MMR vaccine by 24 months of age were not interviewed Kaiser immunization tracking system allows data entry of immunizations received outside of Kaiser Permanente; it is possible that these immunizations are not entered Study results similar between primary analysis with Kaiser immunization tracking system data only and secondary analysis, where parental report data were also included with Kaiser tracking system data |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Children were identified as not having received MMR by 20 months of age in regional computerized immunization tracking system 5 of 321 children were excluded after randomization because they had already received MMR vaccine based on parent report and chart review |
| Methods | Study design: randomized trial; controlled before and after Intervention participants were randomized to 4 groups; controls were not randomized Study aim: assess effectiveness of sending letters to families, delivering automated telephone messages to families, or both, in improving immunization adherence among under‐immunized 20‐month old children in health maintenance organization (HMO) setting | |
| Participants | Inclusion and age: underimmunized 20‐month olds identified by HMO; lived in residence areas of 10 northern California medical centers of Kaiser Permanents Medical Care Program of Northern California n = 867 included in analysis, including 648 randomized to intervention groups and 219 non‐randomized controls; initially 752 were selected and randomized to 4 intervention groups; 67 were excluded because of gap in health insurance coverage | |
| Interventions | Intervention group 1: automated telephone message followed by a letter 1 week later; n = 167 Intervention group 2: automated telephone message; 1‐minute prerecorded message stating that child was overdue for immunizations; it provided telephone numbers for advice or appointment lines at nearest Kaiser clinics; message was personalized with child's first name; system prompted listener to select language for message, either English, Spanish, or Cantonese; n = 165 Intervention group 3: letter; n = 162 Intervention group 4: letter followed by an automated telephone message 1 week later; n = 154 | |
| Outcomes | Primary outcome: receipt of any needed immunization by the 24‐month birthday Odds ratios for combined interventions = 2.1 and 2.5 | |
| Notes | Power: sample size was expected to have 80% power to detect a "16%" difference in immunization outcomes Reported intervention costs | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Randomized patients to 4 intervention groups; selected comparison group of "similar patients" |
| Allocation concealment (selection bias) | Unclear risk | Eligible participants were identified by a computerized immunization tracking system; Randomized patients to 4 intervention groups but not a comparison group because their previous study found letters to be an effective intervention; investigators added "a comparison group of similar patients who turned 20 months old during January 1996"; selection of comparisons not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Interventions were sent from the health maintenance organization's regional office; blinding of participants and personnel not described |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Computerized immunization tracking system may not have complete vaccine information for children enrolled in the health maintenance organization after 42 days of age Tracking system does not consistently include data about immunizations that are given after child leaves health plan 67 of 752 patients were excluded because their follow‐up data may have been incomplete |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Unclear risk | Children were identified using regional immunization tracking system Selected only underimmunized children Characteristics of participants were not described and contrasted between study groups |
| Methods | Study design: randomized trial Study aim: assess effectiveness of computer‐generated telephone reminder and recall messages in increasing immunization visits | |
| Participants | Inclusion: preschool children; if computerized immunization record included telephone number; and if children were due or overdue for immunizations at any time during enrollment Age: less than 2 years Exclusion: more than 1 child in household younger than 2 years, to avoid randomizing multiple children from same household Grouped children into 6 immunization categories, A through F, based on immunizations due (Groups A, C and E) or overdue (Groups B, D and F) | |
| Interventions | Intervention: autodialer; computer‐generated phone reminders; general versus specific reminders; n = 4636 Placed automated calls twice a day for 7 days until made contact Delivered second call during week following first successful contact if an immunization visit was not made | |
| Outcomes | Rates of immunization visits for childhood vaccines Immunization rates were higher for children due for immunization than those overdue | |
| Notes | Telephone numbers listed in computerized immunization database for 94% of children in largest county Contacted 70.3% of intervention households using computer‐generated telephone reminder system It is possible that automated phone messages were received by household members other than parents, such as siblings Measured immunization visits rather than immunizations administered Percentage of intervention households successfully contacted varied by county of residence and ethnicity, with Hispanic and "other" ethnic children having highest percentages of unsuccessful contacts | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Sorted computerized files of eligible children into 6 immunization categories Within each of 4 categories, telephone numbers were sorted and assigned identification numbers sequentially; allocated children with odd identification numbers to intervention group; assigned all other children to non‐intervention group Within 2 remaining categories, telephone numbers were sequentially assigned values of 1, 2, or 3; children in group 1 were allocated to receive a general message, group 2 were to receive a specific message, and group 3 were assigned to the non‐intervention group |
| Allocation concealment (selection bias) | Low risk | Used computerized files of eligible children from 14 county health departments to randomize children to study groups |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified; immunization visits were recorded in each health department’s immunization database; following each autodialer call session, "this information was uploaded and merged with the study file" |
| Blinding of outcome assessment (detection bias) | Low risk | Outcome assessment blinding not specified; outcomes assessed using computerized immunization records |
| Incomplete outcome data (attrition bias) | Unclear risk | Immediately recorded immunization visits in each health department's immunization database when children arrived to receive immunizations Followed study participants for 30 days, beginning on start date of follow‐up or date an immunization was due; for children late for immunizations, start date was first date of successful contact |
| Selective reporting (reporting bias) | Low risk | Reported immunization status for all 6 immunization categories, as described in the methods |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Created county‐specific computerized study files of all eligible children Allocated children after immunization status was determined Measured and compared characteristics of children; groups similar for county and type of residence, ethnicity, sex, and age |
| Methods | Study design: randomized trial Study aim: evaluate effectiveness of simple office interventions in increasing influenza vaccination rates | |
| Participants | Inclusion and age: all active registered patients in practice, 65 years and older Exclusion: patients chronically hospitalized or in nursing homes; persons unable to communicate by telephone or house‐bound | |
| Interventions | Intervention: telephone call to patient and bright‐colored reminder sticker on clinic chart to remind health services team to promote influenza vaccination; n = 120 Telephone calls were made by staff physician, registered nurse, and registered nursing assistant, in approximately equal numbers During calls, informed patients that influenza vaccine was available and they could schedule a regular or nurse visit Made maximum of 3 telephone call attempts to each household Telephone calls were not made if patients made clinic visit before the call was planned | |
| Outcomes | Number and percent receiving influenza vaccine; intention‐to‐treat analysis Intervention group: 24 percentage point increase over control group | |
| Notes | An 8" by 11" advertisement was displayed in the waiting room, saying "Be Keen about Flu Vaccine" Data not entered in RevMan data tables; allocated households No outreach interventions during prior years to promote influenza vaccination | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | "After a random start, patients were alternately assigned to each group" |
| Allocation concealment (selection bias) | High risk | Alternate assignment to groups within one family medicine center; allocated households; related persons in same household were assigned to same group |
| Blinding of participants and personnel (performance bias) | High risk | "Brightly coloured sticker was applied to the charts of the entire study population as a reminder to the health‐care team that the study was under way and that they were expected to promote the flu vaccine"; staff physician, registered nurse, and nursing assistant made telephone calls; callers were provided a call sheet with 5 names each week |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified; "following the immunization period, collaborators reviewed all charts" |
| Incomplete outcome data (attrition bias) | Low risk | Collaborators reviewed all charts following immunization period |
| Selective reporting (reporting bias) | Low risk | Reported influenza immunization outcomes, as described in the methods |
| Other bias | Low risk | Study appears to be free of other sources of bias |
| Baseline measurement | Low risk | Presented influenza immunization rates for participants for 2 years preceding the study; significant differences not observed during 1983 or 1984 Study groups did not vary for sex, mean age, marital status, household composition, number of clinic visits, adverse reactions to medication, and presence of chronic illnesses |
| Methods | Study design: controlled before and after with concurrent control groups Study aim: assess effect of multifaceted influenza vaccination program, in a community setting, that was previously effective in an academic medical center | |
| Participants | Inclusion: patients enrolled in 1 of 4 clinics Age: 65 years and older 2 intervention clinics, 1 suburban and 1 urban, each with an estimated 2800 and 1600 older adults, respectively 2 control clinics selected based on similar locations and comparable numbers of older adults | |
| Interventions | Intervention: letter to patients, standing order for nurses, and reminder sticker on appointment roster; n = 300 Standing order allowed nurses to vaccinate patients without signed physician order Reminder sticker placed on appointment rosters each day for eligible persons; Convenient walk‐in vaccination times made available and publicized in informational mailing Held inservice education session for nurses Described program to physicians at 1‐hour lunch meeting | |
| Outcomes | Number and percent of patients receiving influenza vaccination Intervention clinic 2: 16 percentage point increase in influenza vaccination rate compared to baseline Control clinic 1: 3 percentage point increase in influenza vaccination compared to baseline Control clinic 2: 4 percentage point decrease in influenza vaccination compared to baseline Pre‐intervention to post‐intervention odds ratio: 1.32 | |
| Notes | HMO patients cared for in 19 primary care clinics No special influenza immunization programs in place at clinics prior to study Data not entered in RevMan data tables One intervention clinic, with baseline vaccination rate of 75%, may have experienced a ceiling effect | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Included 2 intervention and 2 control clinics of 19 primary care clinics serving older adults of a health maintenance organization; control clinics were selected based on location and similar numbers of older adults served |
| Allocation concealment (selection bias) | High risk | Allocated clinics to study groups |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Sent postcard survey to randomly selected participants from each of 4 clinics to assess immunization status Pre‐intervention survey response rates were 73% to 89% Post‐intervention survey response rates were 86% to 93% Did not report use of health records or administrative databases to verify immunization outcome data |
| Selective reporting (reporting bias) | Unclear risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | One intervention clinic, with baseline vaccination rate of 75%, may have experienced a ceiling effect |
| Baseline measurement | Unclear risk | Obtained baseline immunization rates; clinic‐specific rates ranged from 51% to 75% Participants in 4 clinics were similar for age and risk‐factors, based on patient survey responses |
| Methods | Study design: randomized trial Study duration: 6‐week intervention phase from 5 October 1994 to 18 November 1994; follow‐up period from 22 November 1994 to 3 April 1995 Study aim: evaluate effectiveness of mailed informational letter in increasing hepatitis B vaccination among college students | |
| Participants | Inclusion: freshman university students; not received hepatitis B vaccine; US citizens Age: less than 20 years Exclusion: international students, because of short study timeframe and time it would take for the letter to reach other countries Setting: University of Rochester, a private university n = 732 | |
| Interventions | Intervention: mailed informational letter was sent to college students and their parents; provided information about hepatitis B vaccine and recommended vaccination; enclosed reminder card with hepatitis B logo and appointment telephone number; n = 366 Control: no informational letter; n = 366 | |
| Outcomes | Number and percent receiving first and second hepatitis B vaccinations Intervention, first hepatitis B: 8.1 percentage points over control group; 11.7% versus 3.6% Intervention, second hepatitis B: 10.1 percentage points over control group; 12% versus 1.9% | |
| Notes | Vaccine charge: USD 66 per series or USD 22 per dose Power and sample size calculations: 365 students per study group were needed, assuming an 18% immunization rate in control group and 28% in intervention group; alpha = 0.05; beta = 0.10 | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | 50% of freshman students and their parents received the intervention letter; "prospective randomized study"; details of randomization not provided |
| Allocation concealment (selection bias) | Unclear risk | Allocation procedure not specified |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Used university health database; no other sources of vaccination data; documentation process not described |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | During September 1994, an intensive campus‐wide educational campaign was conducted to inform students about hepatitis B virus infection and vaccine availability; campaign used articles and announcements in school newspaper, notices on bulletin boards, brochures, and peer‐led education |
| Baseline measurement | Low risk | Used university records to identify students who had not received hepatitis B vaccination; Used health history forms to compare baseline characteristics; study groups did not differ with respect to receipt of care for chronic condition, or history of pelvic infection or viral hepatitis Intervention participants more likely to report exercising than controls; no other differences observed |
| Methods | Study design: randomized trial Recruited participants monthly from September 1998 to January 1999 Obtained immunization status at end of study, during April 1999 | |
| Participants | Inclusion: not received MMR vaccine by 21 months; residents of 1 health authority Setting: 1 health authority, Iechyd Morgannwg Health (United Kingdom) n = 511 children; 255 intervention group; 256 control group Identified children every month during study | |
| Interventions | Intervention: personal reminder letter and "posting leaflet" regarding MMR vaccine; letter was copied to child's general practitioner and health visitor; n = 255 Control: usual practice; no action; n = 256 | |
| Outcomes | Number and percent of participants receiving MMR vaccine between 21 and 24 months of age Intervention: 1.3 percentage points over control group; 7.1% versus 5.8% | |
| Notes | Uptake of first MMR vaccine dose had fallen dramatically after adverse publicity about vaccine Power calculations estimated that 219 participants were needed in intervention and control groups to detect a "10%" difference in proportions immunized, using a 5% significance level, and assuming an intention‐to‐treat analysis | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomized parents of children using computer‐generated random numbers |
| Allocation concealment (selection bias) | Low risk | Each month a list of children was obtained from the computerized child health record system; parents were randomized using computer‐generated random numbers |
| Blinding of participants and personnel (performance bias) | Low risk | "Parents and health professionals were not informed of the trial"; personal reminder letters were "copied to the child’s general practitioner and health visitor." |
| Blinding of outcome assessment (detection bias) | Unclear risk | Immunization status was obtained from the child health record and system; blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Obtained immunization status of 493 children at study end from child health records and system (96.5%) |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Unclear risk | Enrolled children who had not received MMR vaccine Obtained list of eligible children monthly from computerized child health record system Did not report characteristics of participants and prior vaccination status, by study group |
| Methods | Study design: randomized trial Study duration: not specified Study aim: evaluate effectiveness of different types of messages on influenza vaccination | |
| Participants | Inclusion: Medicare beneficiaries without influenza vaccine the previous year based on Medicare reimbursement requests Age: not clear 29 intervention counties and 20 control counties | |
| Interventions | Intervention group 1: reminder letter from peer review organization (PRO); addressed to individuals; on PRO letterhead: allocated into 3 different message groups; message indicated that influenza shot should be received every year; Medicare will pay for vaccination; shot is safe; and shot should be obtained soon Intervention group 1a: reminder letter with gain‐framed insert; letter stated patient was at risk for getting serious case of influenza; insert included picture of woman with positive testimonial; n = 3260 Intervention group 1b: reminder letter with loss‐framed insert; letter stated patient was at risk for getting serious case of influenza; insert included picture of woman with negative testimonial, indicating she had not received flu shot and spent several days in bed, sick with the flu; n = 3262 Intervention group 1c: brief reminder from North Dakota peer review organization; n = 3258 Intervention group 2: action letter; county health officers sent one‐page letters with explicit action instructions; n = 6057 Control; group 3: no letters; n = 20 counties; 7896 participants | |
| Outcomes | Number and percent receiving influenza vaccination Group 1a, reminder letter only: 4.9 percentage point increase over control group Group 1b, reminder letter with gain‐framed insert: 3.9 percentage point increase over control group Group 1c, reminder letter with loss‐framed insert: 4.9 percentage point increase over control group Group 2, action letter: 8.6 percentage point increase over control group | |
| Notes | All interventions were letter reminders, therefore they were grouped for analysis | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomized 29 counties to the intervention group and 20 counties to the control group; the randomization process was not described |
| Allocation concealment (selection bias) | Unclear risk | Randomized counties to 3 groups Randomized patients within reminder letter group to 3 subgroups |
| Blinding of participants and personnel (performance bias) | Unclear risk | County health officers were asked to mail a single letter from their own offices |
| Blinding of outcome assessment (detection bias) | Low risk | Blinding not specified; however, used claims to determine immunization status |
| Incomplete outcome data (attrition bias) | Low risk | Determined vaccination rates by analyzing Medicare claims for 6 months following intervention In 20 control counties, tracked randomly selected participants for behavior; did not include returned letters in numbers Participant loss was estimated at 6% |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Selected participants that had not received influenza vaccination during previous year, based on Medicare claims files |
| Methods | Study design: randomized trial Study duration: 2 months; 23 October 1984 to 31 December 1984 Study aim: compared 3 ways to remind patients about influenza vaccination | |
| Participants | Inclusion: patients registered in 4 practices Age: at least 65 years, for influenza vaccination study arm Exclusion: patients in an institution | |
| Interventions | Intervention group 1: patient reminder in person by physician; not an eligible intervention Intervention group 2: patient reminder by telephone; called by their nurse within 10 days after the start of the study; made up to 5 attempts to contact each family Intervention group 3: patient reminder letter; single letter sent on October 23, encouraging patients to receive vaccination; printed and addressed by computer; signed by patient's physician and practice nurse; letter recommended influenza vaccination, mentioned availability, and encouraged patient to call clinic and schedule vaccination appointment Control group 1: no intervention control group Control group 2: non‐participating controls; 2 practices that opted not to join the study | |
| Outcomes | Number and percent receiving influenza vaccination Patients in 2 non‐participating practices had lowest vaccination percentages | |
| Notes | All patients attending medical center had been registered on computerized record system since 1976; updated system data to prepare for the study Data not included in RevMan data tables Only 2 of 239 letters were returned as undeliverable | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomization process not described |
| Allocation concealment (selection bias) | Unclear risk | Allocated families; grouped family members at same address Patient information included in computerized record system Allocation procedure not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Letters were printed and addressed by the computer; blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Blinding not specified; vaccinations given at the family medicine center were recorded in the computer and used for analysis in the database |
| Incomplete outcome data (attrition bias) | Unclear risk | Recorded vaccinations given at family medical center in computer database Difficult to assess follow‐up of patients who did not come to clinic; investigators called random samples of patients from each study group to estimate underreporting of vaccination, 8 weeks after study ended; 97 were contacted |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | "Because the physicians in the randomized trial would be asked to remind some but not all of their patients, they might tend to remind every patient they saw and thereby inflate the rates of vaccination…"; authors analyzed vaccination data for 2 additional non‐randomized controls to attempt to assess the extent of possible bias Of 97 patients contacted by phone, 15 indicated they received vaccine, including 8 at the center; 7 of 8 (87.5%) were confirmed as having received the vaccine by reviewing physician consultation notes |
| Baseline measurement | Low risk | Determined vaccination status prior to study Prior year immunization data not reported Compared groups for family size, age, and sex; differences in characteristics were not detected |
| Methods | Study design: randomized trial Study duration: September 2010 to February 2012; 2 consecutive influenza seasons Study aim: evaluate effectiveness of text message reminders on increasing influenza vaccination among ambulatory pregnant women, especially those unsure about or unwilling to receive the vaccine | |
| Participants | Inclusion: obstetrics patients less than 28 weeks of gestation; have cell phone with text messaging capabilities Age: 14 to 50 years Exclusion: received influenza vaccination that season, prior to the study; wanted to receive vaccination the day of potential study enrollment; contraindications, such as egg allergy or prior adverse reaction; or previously participated in study Setting: women recruited at routine obstetrics visits at Magee‐Women's Hospital outpatient clinic; academic medical center (USA) n = 216 enrolled women; 158 included in pre‐protocol analysis | |
| Interventions | Intervention: 12 weekly text messages encouraging general pregnancy health plus influenza vaccination; texts mentioned benefits and safety of influenza vaccination during pregnancy; n = 104 Control: 12 weekly text messages encouraging general pregnancy health General texts covered topics such as prenatal vitamins, nutritional foods, and seat belt use during pregnancy; n = 100 | |
| Outcomes | Number and percent receiving influenza vaccination Intervention: 2 percentage points over control group; 33% versus 31%; not statistically significant | |
| Notes | Offered influenza vaccine to patients at prenatal visits; offered at no cost to clinic patients Power calculations: sample size of 70 women per study group was estimated to have 80% power to detect vaccination rate change from 55% at baseline to at least 70% | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | "The randomization sequence was generated and group assignments were placed in sequentially numbered, sealed, opaque envelopes by a researcher" |
| Allocation concealment (selection bias) | Low risk | Randomized participants "to two study arms with equal frequency using a permuted block design with random block sizes of two, four and six"; using sequentially numbered, sealed, opaque envelopes; the researcher managing the randomization was "uninvolved in participant recruitment or clinical care" |
| Blinding of participants and personnel (performance bias) | Low risk | Health services "providers were blind to the groups to which participants were randomized" |
| Blinding of outcome assessment (detection bias) | Low risk | "Record review was conducted after exit surveys were completed by a researcher (M.H.M.) unaware of participants’ random allocation" |
| Incomplete outcome data (attrition bias) | Unclear risk | Data were available for 140 of 216 enrolled women (64.8%); 18 women in the control and 28 in the intervention were "nonevaluable" because they did not receive text messages, pregnancy was terminated early, or they were lost to follow‐up One researcher reviewed medical records to verify vaccine receipt after exit surveys were completed Electronic health record automatically updated vaccination date when administered, through unspecified mechanism |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Women were potentially eligible if they had not received influenza vaccination during current season, based on self‐report and documentation in electronic health record Participants completed anonymous surveys before and after intervention to determine sociodemographic characteristics, beliefs about prevention, and attitudes about text messaging Groups similar at baseline for age, race, education, marital status, household income category, and insurance status Pre‐intervention surveys were self‐administered at enrollment; post‐intervention surveys were conducted by telephone approximately 12 weeks after enrollment by research staff |
| Methods | Study design: randomized trial Study aim: evaluate whether 1 or 2 sequentially mailed reminder letters would improve receipt of influenza immunization among high‐risk patients | |
| Participants | Inclusion: high risk patients seen between February and September 1990 | |
| Interventions | Intervention group 1: 1 reminder letter to patients; n = 135 Intervention group 2: 2 reminder letters to patients; n = 138 Reminder letters were written at fifth grade reading level; described need for influenza vaccination, mentioned vaccine does not cause influenza, possibility of minor side effects, and vaccine could be obtained free of charge without an appointment | |
| Outcomes | Number and percent received influenza vaccination Group 2, 2 letters: 8.5 percentage point decrease over control group | |
| Notes | Immunizations obtained at scheduled appointments, annual health fair that promotes health for older adults, and on a walk‐in basis at the clinic | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomization methods not described |
| Allocation concealment (selection bias) | Unclear risk | Eligible patients were identified by searching a computerized clinical tracking system using date of birth and diagnosis codes recorded by primary care providers; randomized patients to 1 of 3 groups; method not described |
| Blinding of participants and personnel (performance bias) | Low risk | Health center providers blinded to study group assignment |
| Blinding of outcome assessment (detection bias) | Low risk | Outcome assessment blinding not specified; however, only immunizations given at the health center were analyzed; and providers were blinded to study group assignment |
| Incomplete outcome data (attrition bias) | Unclear risk | Immunization data only obtained from health center, not other sites |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Unclear risk | Baseline data not reported |
| Methods | Study design: randomized trial | |
| Participants | Inclusion: high risk elderly members of health maintenance organization (HMO); discharged alive from hospital between October 1983 and September 1984 Discharge diagnoses: cardiovascular, pulmonary, renal, metabolic or nutritional, neurological, or malignant diseases | |
| Interventions | Intervention: personalized persuasive letter sent to patients; letter emphasized importance of influenza vaccination for older adults at high risk for influenza and complications, benefits of vaccination, and how and where to obtain the vaccine; n = 1105 | |
| Outcomes | Percent of eligible persons receiving influenza vaccination Intervention: 8.8 percentage point increase over control group Pneumococcal vaccinations also measured; but not targeted by intervention | |
| Notes | Immunizations covered by HMO health plan; can be obtained by members at affiliated immunization clinic without an appointment Power calculations not specified | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomized patients using "pseudo‐random digit" of individual membership identification number |
| Allocation concealment (selection bias) | Unclear risk | Participants were identified by computerized inpatient records using age, discharge status, and discharge diagnoses; randomized patients using "pseudo‐random digit" of individual membership identification number |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Obtained influenza vaccination data by retrospective review of medical records at end of study period; measured from October 1984 through May 1985 Did not specify proportion of outcomes obtained |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Unclear risk | Intervention and control groups similar for age and chronic condition distribution; intervention group has a somewhat higher proportion of males Baseline immunization rates for influenza or other vaccinations not specified |
| Methods | Study design: randomized trial Study aim: evaluate effect of postal reminder and vaccination fee on influenza vaccination rates | |
| Participants | Patient inclusion: 45 patients, selected consecutively per practice; persons being treated for chronic conditions of pulmonary or cardiovascular systems, persons with acquired or congenital immunodeficiencies, other chronic diseases identified by physician as being high‐risk, and residents of nursing homes Practitioner inclusion: planned to select 15 practitioners; did not send mailed reminders in previous years; serving at least 45 elderly patients in specified risk group | |
| Interventions | Intervention group 1: postal invitation and free vaccine; invitation letter, personalized with patient's name and general practitioner's signature; n = 195 Intervention group 2: postal invitation and usual charge; n = 195 Letters personalized with patient's name and clinician's signature | |
| Outcomes | Number and percent receiving influenza vaccine Combined intervention more effective than postal letter alone | |
| Notes | Only solo practitioners were invited to participate; characteristics of participating providers similar to other general practitioners in Denmark for age and number of patients; few female clinicians participated in study Among 51 general practitioners who did not want to participate, 1 used reminders in past, 1 considered randomization to be unethical, and 49 either considered study workload to be too heavy or did not provide a reason Financial incentives, such as providing vaccine free versus a charge, were not eligible interventions | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomization process not specified |
| Allocation concealment (selection bias) | Unclear risk | Selected 45 patients from each practice, consecutively with a random starting point; then randomized patients within each practice to 3 study groups; method of allocation not specified |
| Blinding of participants and personnel (performance bias) | Low risk | "Randomization was blinded for the GPs" |
| Blinding of outcome assessment (detection bias) | Unclear risk | General practitioners blinded to randomization; practitioners apparently knew which patients were randomized; the date of vaccination was "registered"; data collection process not described; outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | For all patients vaccinated, documentation included: indication for vaccination, date of birth, sex, vaccination date, and whether patient was vaccinated during previous year; Possible data misclassifications were checked with practitioners |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Unclear risk | Recorded vaccination status for previous year for all vaccinated patients; differences between groups not specified 83% of control group participants that received vaccinations during study period were vaccinated in prior year |
| Methods | Study design: randomized trial Study duration: September 2012 to August 2013 Study aim: evaluate effectiveness of bi‐directional text messages in increasing immunization rates among adolescents | |
| Participants | Inclusion: adolescents needing recommended adolescent vaccination or well child check; seen at participating practice at least once in previous 2 years; parents had cell phone number Age: 11 to 17 years Exclusion: sibling participating in study Setting: 5 urban‐suburban private pediatric and 2 safety‐net practices in Colorado (USA) n = 4587; 2228 intervention and 2359 control | |
| Interventions | Intervention: up to 3 (abstract) or 4 (page e1222) brief text messages with script, sent to parents, indicating that patient is due for either vaccination, checkup, or both; reply options: request to be called by clinic to schedule an appointment, plan to call the clinic, or stop texts; n = 2228 Control: usual care; no reminders; n = 2359 Practices did not use reminders during the study other than texts to intervention participants | |
| Outcomes | Outcome 1: receipt of all needed vaccinations, including Tdap, MCV4 and HPV Outcome 2: receipt of any vaccination Intervention, outcome 1: 4.1 percentage points over control group; 15.0% versus 11.9% Intervention, outcome 2: 5.2 percentage points over control group; 15.0% versus 20.2% Intention‐to‐treat approach used for primary analysis | |
| Notes | Intervention was developed with focus group input from adolescents, parents, and care providers, from 7 practices | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomized patients within each practice using random number generation with SAS 9.3 |
| Allocation concealment (selection bias) | Low risk | Randomized patients within each practice using random number generation with SAS 9.3; providers were blinded to group allocation Selected practices purposefully to enroll a diverse cross section of patients |
| Blinding of participants and personnel (performance bias) | Low risk | Care providers blinded to group assignment Blinded intervention parents and adolescents to which sibling was enrolled in study when household had multiple potentially eligible adolescents; non‐study siblings also received intervention, but were not included in analyses |
| Blinding of outcome assessment (detection bias) | Low risk | Outcome assessment blinding not specified; however, care providers were blinded, and investigators used administrative data from practices' electronic billing systems and the immunization information system |
| Incomplete outcome data (attrition bias) | Low risk | Included practices participate in the Colorado Immunization Information System, which was used in most primary care practices, school‐based health centers, public health departments, and some pharmacies All outcomes were assessed 6 months after last text message; no patients lost to follow‐up 1877 of 2228 received the text messages in the intervention group |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias Data in Colorado Immunization Information System, for new interfaces, are reviewed for quality and validity |
| Baseline measurement | Low risk | Used practice administrative data to determine study eligibility; merged electronic billing systems data with Colorado Immunization Information System data Study groups similar for age, sex, immunization status, or other primary outcomes at baseline |
| Methods | Study design: randomized trial | |
| Participants | Inclusion: mothers and newborns delivered by Family Practice residents Exclusion: child with serious neonatal illness, such as extreme prematurity, that may require different immunization schedule; living outside county | |
| Interventions | Intervention: reminder letter to parents 2 months post delivery, 10‐ to 15‐minute parent education session about immunizations on first day postpartum, delivered by nurse or physician, and one page handout summarizing key points from immunization discussion; n = 116 | |
| Outcomes | Percent immunized for DTP and oral polio (OPV), first, second, and third doses 2‐ and 4‐month vaccinations considered on time if occurred within 3 months and 5 months after delivery, respectively | |
| Notes | Authors mentioned concerns about dual system for indigent care in the area and restricted hours of immunization clinic | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | "postpartum mothers were assigned to either the intervention or the control group according to delivery date"; Intervention: Sunday, Tuesday, Thursday; Control: Saturday, Monday, Wednesday |
| Allocation concealment (selection bias) | High risk | Used date of birth of infant; infants and their mothers allocated to intervention or control group based on delivery date |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Followed children for 2‐ and 4‐month DPT and OPV vaccinations; reviewed immunization records for completion of first 3 DPT and OPV immunizations at 1 year of age Contacted several physicians' offices to determine if vaccinations were obtained at private physicians' practices Immunizations are costly, so authors do not believe many immunizations are obtained from private practitioners; however, records of 1 clinician that administered vaccinations through the Medicaid Early and Periodic Screening, Diagnosis and Treatment (EPSDT) program were reviewed for study patients, and immunization data were included |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Study began at birth, so no prior immunization data Groups were similar for age, race, previous number of children, and prenatal care |
| Methods | Study design: randomized trial | |
| Participants | Inclusion: active patients at family medicine center; at least 1 family member had clinic visit within previous 2 years | |
| Interventions | Intervention group 1: 2 computer‐generated personalized reminder letters to patients describing needed preventive services and requesting they make physician appointment to receive them; letters printed on letterhead stationery and signed by patient's primary physician Sent first letter during August 1998; sent second letter in January or February 1989, unless first letter was returned without forwarding address; n = 1925 patients; 12 physicians Intervention group 2: computer‐generated reminder letters to patients and computer‐generated physician reminders; generated 1‐page physician reminders the night before scheduled appointments; nursing staff attached them to medical record the morning of scheduled visit; form used by clinicians to check off actions taken for each preventive service; n = 1908 patients; 13 physicians Intervention group 3: computer‐generated physician reminders only; this group is not an intervention in our review; n = 1988 patients; 14 physicians Control: educational sessions for residents, quarterly audits and flow sheet on chart; n = 1576 patients; 10 physicians All groups received educational and administrative interventions; resident physicians attended educational sessions about health promotion and targeted preventive services; performed quarterly audits to identify percentage of patients up‐to‐date with the 5 preventive services, per physician practice; health maintenance flow sheet was placed in medical record for all adult patients | |
| Outcomes | Percent of persons receiving tetanus vaccine Other outcomes tracked: serum cholesterol measurement; fecal occult blood testing; mammography; Papanicolaou smears | |
| Notes | Allocated providers; data not included in RevMan | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | "Patients and their physicians were randomly assigned by practice group into one of 4 study groups" |
| Allocation concealment (selection bias) | Unclear risk | Conducted study at 1 family medicine center; used computerized database to identify patients; did not specify randomization process |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Research assistants collected physician reminder checklist forms each day; data on tetanus immunizations received outside the clinic could be entered in the computer by clinic nurses |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | Allowed physicians to withhold letters from individual patients Compared accuracy of computerized database with 500 patients' medical records; Kappa value for tetanus vaccines was 0.67 |
| Baseline measurement | High risk | Reviewed computerized medical records to assess whether patients were up to date with 5 preventive services, including tetanus vaccine At baseline, most patients had made at least 1 prior visit to the clinic Study groups differed for race distribution, insurance coverage, and visit frequency |
| Methods | Study design: randomized trial Study aim: assess effectiveness of postcard reminder on influenza vaccination | |
| Participants | Inclusion: all nonresidential patients of the practice Age: at least 65 years Exclusion: received influenza vaccine by 1 April 1996, left the practice, allergic to egg protein, known by the practice to object to influenza vaccination, severe or terminal illness, dementia or unstable psychiatric conditions, or in nursing home; patients in nursing homes were not included in the registry from which patients were identified | |
| Interventions | Intervention: single large postcard reminder with large print; sent on 1 April 1996 in a hand‐addressed envelope, encouraging patients to visit the practice for influenza vaccination before month end; stressed seriousness of influenza and provided availability and cost information; had practice logo; Flesch readability score of 68, needing a minimum IQ of 90 to understand it; n = 154; 96 women and 58 men Controls may have been exposed to mass media campaign | |
| Outcomes | Number and percent receiving influenza vaccination Intervention more effective for men | |
| Notes | Data not entered in RevMan | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Used computer‐generated random number facility to allocate patients to study groups |
| Allocation concealment (selection bias) | Low risk | Identified participants from a computerized age, sex, and disease register at the practice Used computer‐generated random number facility to allocate patients to study groups Allocated both members of married couples to same group |
| Blinding of participants and personnel (performance bias) | Low risk | General practitioners blind to randomization |
| Blinding of outcome assessment (detection bias) | Low risk | Record reviewer was blind to study group allocation |
| Incomplete outcome data (attrition bias) | Low risk | Medical records were reviewed for influenza vaccination 4 months after intervention postcard was sent Vaccination was considered not given if influenza vaccination prescription was given to patient but vaccination was not recorded in medical record |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Collected and analyzed baseline data; influenza vaccination rates similar between study groups during 1995 |
| Methods | Study design: randomized trial Study duration: July 2013 to March 2014 Study aim: evaluate effectiveness of centralized text message reminder on increasing receipt of first HPV vaccination dose among low‐income adolescents | |
| Participants | Inclusion: no prior HPV vaccinations; enrolled in Monroe Plan, a single health maintenance organization; patients having primary care provider at one of 39 primary care practices; phone number listed in the insurer's database; eligible as of 1 July 2013 Age: 11 to 16 years Exclusion: sibling of participating adolescent; transferred out of participating practice, or no longer insured by managed care organization during study period Setting: managed care organization; 39 primary care practices, 29 pediatric and 10 family medicine; each practice served more than 175 adolescents enrolled in the managed care organization (USA) n = 3812 publicly insured adolescents | |
| Interventions | Intervention: sent up to 4 text message reminders to parents; generated by programmer at managed care organization, using third party vendor; initial text message allowed parents to opt out of text reminders; first reminder text indicated the adolescent was due for HPV vaccination, and were asked to call to schedule clinic appointment; n = 1893 Control: received initial message regarding health, with message that the parent could opt out; this was followed by different general adolescent health topic messages, such as eat breakfast, each time reminders were sent to intervention group parents; n = 1919 | |
| Outcomes | Primary outcome: received first HPV vaccine dose Secondary outcomes: received second and third HPV vaccine doses Intervention group, first dose, persons with cell phone: 2.1 percentage points over control group; 14.4% versus 12.3% Intervention, second dose, persons with cell phone: 0.9 percentage point over control group; 6.1% versus 5.2% Intervention, third dose, persons with cell phone: 0.6 percentage point over control group; 2.0% versus 1.4% | |
| Notes | We requested and obtained detailed numerator and denominator data from first author Almost half of parents did not have a working telephone number or a phone capable of receiving text messages, including 760 control and 730 intervention participants; 278 controls and 205 in intervention group opted out of text messages | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Stata was used to generate a randomization table |
| Allocation concealment (selection bias) | Low risk | Study was based centrally at a large not‐for‐profit managed care organization; randomized adolescents within each practice; used Stata to generate randomization table |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified; managed care organization programmer reviewed vaccination data to identify need for text messages using billing and registry data |
| Incomplete outcome data (attrition bias) | Low risk | New York law requires documentation of immunizations in state immunization registry for persons less than 19 years Examined results for all participants |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Obtained baseline HPV vaccination status by reviewing billing data or New York state's immunization registry Managed care organization programmer reviewed immunization data Intervention and control participants were similar for age distribution, Medicaid and State Children's Health Insurance Program (SCHIP) coverage, practice specialty, and urban or suburban versus rural residence None had received HPV vaccination prior to enrollment |
| Methods | Study design: randomized trial Study duration: patients recruited from April 2012 to December 2013; follow‐up to April 2014 Study aim: assess effect of phone or text message reminders to parents of adolescents on receipt of human papillomavirus vaccinations | |
| Participants | Inclusion: parents of adolescents who received HPV vaccine and filled out consent form at first or second dose Age: 11 to 17 years at enrollment Exclusion: completed HPV vaccine series or did not get first HPV vaccine dose; sibling of participant; incomplete forms; no longer interested; language barrier Setting: 3 urban primary care practices in Rochester, NY, USA n = 749 randomized | |
| Interventions | Intervention group 1: autodialer; maximum of 3 successful reminders for each dose due; sent to parents 1 week apart using Televox communication system; up to 6 attempts; message indicates adolescent due for next HPV vaccination and to call to schedule appointment; n = 178 Intervention group 2: text messages; maximum of 3 successful reminders for each dose due; sent to parents 1 week apart using Televox communication system; shorter version than autodialer message; reminders continued through April 2014 if needed; n = 191 Control group 1, for autodialer: not described; n = 180 Control group 2, for text: not described; n = 200 | |
| Outcomes | Outcome 1 ‐‐ primary: time from enrollment to receipt of second and third doses of HPV vaccine; intent‐to‐treat analysis Outcome 2 ‐‐ secondary: HPV vaccination; doses 1, 2, and 3 Outcome 2, autodialer: 48% versus 40%; 8 percentage point difference Outcome 2, text message: 49% versus 31%; 18 percentage point difference | |
| Notes | Sparse methodological details | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Participants consented to participate and selected preferred reminder method, and were randomized in a blocked format to reminder or usual care |
| Allocation concealment (selection bias) | Low risk | Analyst managing the randomization was blinded to individual group assignment |
| Blinding of participants and personnel (performance bias) | Unclear risk | Analyst was blinded; but blinding not described for clinical, participant or other study personnel |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding and data sources not described |
| Incomplete outcome data (attrition bias) | Unclear risk | Data collection procedures and follow‐up were not described |
| Selective reporting (reporting bias) | Low risk | Reported outcomes for study questions |
| Other bias | Unclear risk | Insufficient information to assess other bias |
| Baseline measurement | Low risk | Phone and text groups similar for sex, practice, insurance, and ethnicity; adolescents in text arm slightly older than phone arm; whites more likely to choose text arm compared with blacks |
| Methods | Study design: randomized trial Study duration: September 2009 to 30 April 2010 Study aim: assess efficacy of educational program and personalized letter on improving influenza vaccination rates among patients 60 years and older | |
| Participants | Inclusion: patients of participating practices Age: at least 60 years on first day of 2009 influenza vaccination season Setting: practices of 13 family physicians; Centro de Slud Rafalafena, a health center in Castellon, Comunidad Valenciana (Spain) n = 2402 adults | |
| Interventions | Intervention: Education Program Group (EPG); personalized letter was sent once to participants by surface mail during the first few days of September 2009, a few weeks before the official influenza vaccination campaign began; written in Spanish; included information about clinical manifestations of influenza and possible complications, vaccine efficacy, and recommendations from Centers for Disease Control and Prevention and local authorities of Comunidad Valenciana; addressed common vaccine concerns; written in plain language; n = 1201 Control: no program group; n = 1201 | |
| Outcomes | Number and percent of participants receiving influenza vaccination Intervention group: 5.4 percentage point increase over control group | |
| Notes | No letters were returned as undeliverable No participants were excluded because of egg allergy or previous diagnosis of Guillain‐Barre Syndrome Power calculations determined that 1187 participants were needed per study group to detect at least a "5%" difference in influenza vaccination rates, with significance level of 0.05 and power of 80%; sample size was achieved | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Used computer random number generator to randomly assign participants |
| Allocation concealment (selection bias) | Low risk | Patients were included or excluded using Abucasis II, an Internet application used for clinical follow‐up of all patients in the Agencia Valenciana de Salud Used computer random number generator to randomly assign participants |
| Blinding of participants and personnel (performance bias) | Low risk | Blinded health services workers |
| Blinding of outcome assessment (detection bias) | Low risk | Personal identification information was replaced with codes and use throughout all study phases Obtained vaccination data from Internet application, Abucasis II |
| Incomplete outcome data (attrition bias) | Low risk | Collected data for all participants, including 2009 influenza vaccination coverage All data available for 2241 of 2402 patients (93%) |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Collected data for all participants: sex, age, nationality, race, labor status, primary care physician, district or town of residence, 2008 influenza vaccination status Groups were similar for sex, age, employment status, city of residence, and 2008 influenza vaccination rates |
| Methods | Study design: randomized trial; 2 by 2 factorial design Study aim: measure effect of multi‐modal tracking and outreach intervention on improving vaccination coverage among children | |
| Participants | Inclusion: all children in 9 practices, born between 1 March 1993 and 28 February 1994 Exclusion: children who changed to nonparticipating provider or moved from Monroe County, New York were excluded from analyses Practices included: 2 pediatric urban group practices; 2 family medicine neighborhood health centers; 1 pediatric neighborhood health center; 1 hospital‐based clinic; 3 rural health centers | |
| Interventions | Intervention group 1: tracking with outreach; lay outreach workers, recruited from respective practice neighborhoods, were assigned to at least 1 practice; workers reviewed medical records to determine immunization status, and worked with parents of underimmunized children by sending postcards and making telephone calls; they made home visits to non‐responding parents; n = 630 Caseload: approximately 300 per outreach worker Intervention group 2: provider prompts; not an eligible intervention; n = 744 Intervention group 3: tracking, outreach, and provider prompts; received group 1 interventions, and distinct marker and "missed opportunity card" were placed on charts for children needing immunizations; n = 648 | |
| Outcomes | Number and percent completing age‐appropriate vaccination series, including DTP, OPV, MMR, and Hib | |
| Notes | Used 1‐month grace period to determine series completion outcomes Allocated patients; siblings not split between study groups; data not entered in RevMan Baseline immunization rates in area were relatively high, similar to national rates | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Allocated children to study groups using computer program |
| Allocation concealment (selection bias) | Low risk | Computer billing or encounter files were used to identify names and identifiers for participants Allocated children to study groups using computer program; outreach workers were provided with lists of intervention participants to conduct outreach and track |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified; outreach workers did not document their interventions in the medical charts |
| Blinding of outcome assessment (detection bias) | Low risk | Collected data by chart abstraction; chart reviewers were blind to study group assignment |
| Incomplete outcome data (attrition bias) | Low risk | Independent research information group collected outcome data by conducting medical chart abstraction Study completion ranged from 88% to 94% within study groups Monroe County Health Department generally provided less than 1% of immunizations; health department provided written documentation to primary care providers or administered immunizations |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias Performed quality control checks with dual independent review of 10% of charts; only provider‐validated immunization histories were accepted |
| Baseline measurement | Low risk | Independent research group at the University of Rochester collected baseline data by medical chart abstraction; this group was not involved with conducting interventions Study groups were similar for age, sex, insurance type, and baseline immunization status; study groups differed for racial composition; race was not recorded in charts for almost half the participants |
| Methods | Study design: randomized trial Study aim: compare effectiveness of patient telephone or letter reminders and physician reminders in improving rates of preventive services, including influenza and tetanus vaccinations | |
| Participants | Inclusion: patients active in practice, based on response to letters sent to patients in 1984 Exclusion: in hospital or institution Setting: Ottawa Civic Hospital Family Medicine Centre (Canada) Clinical practice was organized into 6 teams, each team served approximately 1200 patients and comprised 1 physician, 1 nurse, and 3 to 5 residents; patients visited their team during regular office appointments | |
| Interventions | Intervention group 1: telephone reminder to patient; practice nurse attempted to call family, trying up to 5 times; nurse informed patient about needed procedures and attempted to arrange to have them performed; n = 1104 families; 1468 people Intervention group 2: sent computer‐generated reminder letter to patient and families; signed by physician and nurse; described needed procedures and importance of having them performed; sent second reminder to nonresponders after 21 days; 1168 families; 1541 people Intervention group 3: computer‐generated physician reminder included on routinely printed encounter form before any office visit to inform physician of outstanding preventive services; ineligible intervention; 1122 families; 1471 people | |
| Outcomes | Percent of procedures performed | |
| Notes | Allocated families; data not entered in RevMan 67 participants in the telephone group were not contacted because no phone, hearing impairment, or did not understand English or French; of remaining 1037 in phone group, 66% were contacted 164 letters were returned as undeliverable (14%) | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Used a standard randomization computer program to allocate families to study groups |
| Allocation concealment (selection bias) | Low risk | All patients of family medicine center registered in computer database since 1976; used a standard randomization computer program to allocate families to study groups For the active reminder groups, the computer printed a list of names and telephone numbers of persons needing the interventions each 2‐week study period |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified; obtained outcome data from computer database and asked patients about immunizations and other procedures obtained at other sites |
| Incomplete outcome data (attrition bias) | Unclear risk | Obtained outcome data from computer database for analysis Asked patients about procedures completed at other facilities and if they could be verified; procedures were recorded as completed if the patient said yes |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Unclear risk | Reviewed all patients' computerized records to identify whether procedures were completed prior to study period within appropriate timeframe Similar distribution of sex between study groups; not clear about other characteristics |
| Methods | Study design: randomized trial Followed‐up 1 year after reminder intervention; some patients had almost 2 years of follow‐up Study aim: evaluate effect of 3 computerized reminders on tetanus immunization rates | |
| Participants | Inclusion: clinic patients Exclusion: in hospital or institution Each practice consists of a team of 1 staff physician, 1 nurse, and 3 or 4 residents | |
| Interventions | Intervention group 1: physician reminder and in‐person patient reminder by physician; computer‐generated reminder was printed on encounter form used for billing; to ask patient about tetanus vaccination; ineligible intervention n =1399 Intervention group 2: telephone patient reminder; practice nurse attempted to contact family by phone, making up to 5 calls per family during office hours; n = 1390 Intervention group 3: computer‐generated patient reminder letter was sent to the family; signed by physician and nurse; inquired about tetanus vaccination and recommended booster every 10 years; enclosed prepaid envelope so patients could send a reply; n = 1471 Control group 1: no reminder; n = 1329 | |
| Outcomes | Percent of patients vaccinated during study period with tetanus booster or clear statement of receipt in past 10 years Analyses completed with 1 randomly selected person from each family; analyses repeated using data for all patients in sample | |
| Notes | Data for 1 patient per family were entered in RevMan | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomized patients, as family groups, to study groups using standard randomization computer program |
| Allocation concealment (selection bias) | Low risk | Since 1976, all clinic patients have been registered in computer database; randomized families to study groups using standard randomization computer program; each family was given a unique identifier; each 2‐week study period, the computer printed a list of patients and telephone numbers to receive telephone calls and letters |
| Blinding of participants and personnel (performance bias) | High risk | "Each family member was given a unique identifier, and the group allocation was transcribed onto the file of each family member" |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Each patient was followed up for at least 1 year |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Unclear risk | At baseline, reviewed charts of 5589 patients; any tetanus vaccination information was added to computerized database Authors noted incomplete baseline records Patients in reminder groups were asked to provide year of last tetanus vaccination Study groups were similar for age, sex, and family size; family size was different between groups when data from non‐participating practices were included |
| Methods | Study design: randomized trial, allocated participants by week Vaccine series completion was assessed through 16 June 2000 Study aim: evaluate effectiveness of telephone reminder‐recall intervention on increasing rates of hepatitis B vaccinations | |
| Participants | Inclusion: male patients who reported susceptibility to hepatitis A or B; had accepted first dose of hepatitis A or B vaccine before enrollment; provided telephone number for nurse to call and leave a message with reminders about due or overdue doses; only men who have sex with men were included in analyses | |
| Interventions | Intervention: telephone reminders; receive reminder 1 week before vaccination dose was due, and recalls at 2 and 6 weeks after dose was due; at least 1 other call attempt a different time of day, if patients not reached; n = 279 | |
| Outcomes | Number and percent receiving second hepatitis B vaccine dose | |
| Notes | Hepatitis A and B vaccines were provided free‐of‐charge to clinic 16.1% of intervention patients did not receive full intervention for second hepatitis B vaccine dose Vaccinations were free to patients | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Allocated participants by week enrolled |
| Allocation concealment (selection bias) | High risk | Allocated participants by week enrolled |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified; "Vaccine‐eligible clients were asked their willingness to be enrolled in an evaluation of a strategy to enhance completion of the vaccination series." |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Clinic employees recorded vaccination‐related information on vaccination record forms, including dates vaccinations were received at clinic, serious vaccine‐related adverse reactions, and reasons for dropping out of vaccine program 524 of 541 patients who accepted first vaccine dose included in evaluation |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | To be eligible for the study, "clients had to accept a first dose of at least one of the vaccines…" Written information about hepatitis A and B infections, and availability, safety, and efficacy of the vaccines was available in clinic waiting room At the beginning of clinic visits, patients were informed by the interviewer about the availability of free hepatitis A and B vaccines Clinicians were asked to discuss availability of and recommend hepatitis vaccines to eligible patients Nurses explained to patients about vaccinations number and schedule |
| Baseline measurement | Unclear risk | Collected demographic data from each patient at each clinic visit: age, race, ethnicity, highest level of educational attainment; data were displayed for all clients, vaccine‐eligible clients, and clients who accepted the vaccine, but not stratified by study group Vaccine eligibility was based on self‐report of previous hepatitis B infection or vaccination |
| Methods | Study design: randomized trial | |
| Participants | Inclusion of patients: patients of 16 general practitioners Inclusion of practitioners: capacity to generate list of names and addresses of all patients over 65 years; normally provide influenza vaccination to patients; work at least 80% full time equivalent; do not currently have postal influenza vaccination reminder in place Clinician n = 16 participated of 31 contacted; 8 not eligible; 7 eligible but not interested in participating | |
| Interventions | Intervention group 1: personalized letter to patients, recommending visit to general practitioner to receive influenza vaccination; n = 931 Intervention group 2: personalized letter to patients, recommending visit to general practitioner to receive the influenza vaccination at no charge; letters signed by principal investigator; n = 930 | |
| Outcomes | Number and percent receiving influenza vaccination | |
| Notes | Vaccination sales in New Zealand suggested that no more than 20% of older adults were vaccinated for influenza each year; Typical cost of influenza vaccination was NZD 20 | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Random sequence generation not specified |
| Allocation concealment (selection bias) | Unclear risk | General practitioners were randomly selected from a list of those currently active in the region Each practitioner generated a list of up to 210 patients over 65 years; randomized patients; process not specified |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | List of patients was used to document receipt of influenza vaccination, number of influenza vaccines given in each group and each general practitioner Full results were available for 15 of 16 participating general practitioners; data not available for control group and letter and free vaccine group for one practitioner; "the major potential source of bias in this study is incomplete recording of the administration of vaccine to people enrolled"; "Because people receiving free vaccine were required to hand in their individually signed letter, all of which were returned to the principle investigator, administration of flu vaccine to group 3 was readily verified." |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Unclear risk | Baseline measurement and data not reported |
| Methods | Study design: randomized trial | |
| Participants | Inclusion: continuing care patients of the General Internal Medicine Clinic, listed in computer file | |
| Interventions | Intervention: patient reminder letter and seminar on pneumococcal vaccination to clinic staff; letters sent in October 1982, encouraging patients to receive pneumococcal vaccination or update clinic records; n = 163 | |
| Outcomes | Intervention group, pneumococcal vaccine: 20 percentage point increase over control group | |
| Notes | — | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomization process not described |
| Allocation concealment (selection bias) | Unclear risk | Used computer file to generate list of patients; randomized them to intervention and control group; process not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Charts were examined for changes in vaccination status; outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | In October 1983, reviewed charts of intervention and control patients for changes in vaccination status Data reported for 80 of 92 (87%) randomized control group and 163 of 173 (94.2%) intervention group patients |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Reviewed all charts of study group patients for vaccination status between July and September 1982 Study groups similar for age, sex, provider, prior year influenza vaccination status, and number of patients dropped from study |
| Methods | Study design: randomized trial nested within a larger study Study aim: evaluate effect of centralized computerized immunization register and reminder postcards on childhood immunization rates | |
| Participants | Inclusion: infants entered in health department's computer system; intervention group participants were all infants born between 20 April 1985 and 31 December 1985; control group participants were all infants born between 1 January 1985 and 20 April 1985 | |
| Interventions | Intervention: reminder card sent to parent early during any month in which vaccinations were due; monthly printout was sent to general practitioner with names of children due for immunizations; n = 709 Non‐randomized controls: 613 | |
| Outcomes | Receipt of childhood immunizations: percent immunized at 6 weeks: 18.2% point increase, and at 3 and 5 months | |
| Notes | Established centralized computerized immunization registry in New Zealand to address concerns about immunization data, such as using payment records that are grouped and not linked to patient‐level information, and overestimated vaccination levels using parent questionnaires, administered by public health nurses, which monitor children's immunization at the time of school entry | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Allocated patients by even and odd dates of birth |
| Allocation concealment (selection bias) | High risk | Allocated patients to larger study based on date of birth, then further allocated into patient reminder study by even and odd dates of birth |
| Blinding of participants and personnel (performance bias) | Unclear risk | Practitioners received a monthly printout of names of all infants in the study with infants' dates of birth; blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Completed printouts, listing patients needing vaccinations, by placing check mark in appropriate immunization column for each infant; wrote in unlisted infants that received vaccinations; lists were submitted monthly as claims; payments were made, and infants' computer files were updated with immunizations given Immunizations among infants who moved in and out of the area after birth were not recorded |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Infants were registered in a computer database as soon as possible after birth, with infant's date of birth, mother's name and address, and general practitioner's name |
| Methods | Study design: randomized trial | |
| Participants | Inclusion: high risk patients Exclusion: patients 65 years and older without other risk factors | |
| Interventions | Intervention: reminder postcard sent during 2 weeks before influenza vaccine was available, indicating that physician had determined they were at high risk for flu complications, and strongly urging them to come to clinic for immunization; n = 519 | |
| Outcomes | Percent of persons receiving influenza vaccine Postcard was observed to be effective for sex and rank subcategories, and most age subcategories, except those less than 21 years and 21 to 40 years of age | |
| Notes | Allocated families, patients analyzed; data not entered in RevMan Registered information about all practice patients in computer: name, demographic data, and diagnoses Influenza vaccines were available to all eligible patients on walk‐in basis, without an appointment, and free of charge | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Allocated families to study groups using table of random numbers |
| Allocation concealment (selection bias) | Low risk | Last 2 digits of military sponsor's social security number were used for all members of a family to group them in allocation process; then families were allocated using table of random numbers |
| Blinding of participants and personnel (performance bias) | Low risk | Physicians in the department "were aware that a study was in progress and that some of their patients might receive postcards about influenza immunization" Offered vaccination to all eligible patients on a walk‐in basis, without appointment, and free of charge |
| Blinding of outcome assessment (detection bias) | Unclear risk | "The physician or a nurse completed the standard department computer form for each patient receiving influenza immunization"; outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Physician or nurse completed standard computer form for each patient that received an influenza vaccination |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Unclear risk | Collected data on all identified patients at high risk for influenza complications: age, sex, rank of sponsor, and whether the patient received the influenza vaccine during the 6‐month study period Study groups similar for age distribution; intervention group had more female participants and officers than control group |
| Methods | Study design: randomized trial; study subset randomized to intervention and control groups Study duration: 8‐month study; drew sample 1 August 2013 through 15 November 2013; study began 19 August 2013; claims reported by 1 April 2014 Study aim: evaluate feasibility of implementing multi‐level intervention to increase HPV vaccination among adolescents | |
| Participants | Inclusion: enrolled in Medicaid or CHIP in June 2013; no HPV vaccine claims; residential zip code in North Central Florida; Gainesville, FL or surrounding primary care service area; at least 1 regular office visit between 1 July 2011 and 1 August 2013 Age: 11 to 17 years; mean = 13.7 years Exclusion: previously received HPV vaccine based on Medicaid or CHIP claims data Setting: clinics in Gainesville, Florida and surrounding service areas (USA) n: 5663 in non‐health information technology (HIT) groups | |
| Interventions | Intervention group 1: 2 postcards sent to parents, one at study start and one 2 months later; sex‐specific; used learner verification framework; behavioral experts developed and refined postcards using iterative approach with focus groups of parents of Florida Medicaid and CHIP‐enrolled adolescents; 6‐ by 8‐inch full color postcards with images of adolescents and parents; English and Spanish; described vaccine benefits, costs, side effects, and safety; urged parents to discuss vaccination with the adolescent’s health services provider; n = 2839 Intervention group 2: HIT system; not eligible intervention; n = 1774 Control, non‐HIT: no patient reminder or recall; n = 2824 | |
| Outcomes | Initiation of human papillomavirus vaccine series Intervention group 1: 5.6% versus 4.6%; 1 percentage point higher | |
| Notes | Randomly selected 200 parents of girls and 200 parents of boys to receive 3‐page survey to assess the acceptability of postcards; enclosed USD 5 cash and hand‐stamped return envelope; survey sent on 1 November 2013; follow‐up survey sent 28 January 2014 to non‐respondents | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomly assigned half of the boys and half of the girls in the HIT and comparison arms to receive postcard intervention; method of randomization not described |
| Allocation concealment (selection bias) | Unclear risk | As above |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not described |
| Blinding of outcome assessment (detection bias) | Unclear risk | Used Medicaid and Children's Health Insurance Program (CHIP) claims and patient surveys to assess vaccinations; blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | Obtained immunization data from Medicaid and CHIP claims; however, 36% of adolescents without vaccination claims reported initiating HPV vaccination series |
| Selective reporting (reporting bias) | Low risk | Reported results for all groups and subgroups, answering study questions |
| Other bias | Unclear risk | Study methods not sufficiently detailed to assess other potential sources of bias |
| Baseline measurement | Unclear risk | Baseline vaccination rates initially differed between HIT and non‐HIT providers; investigators modified geographic areas to create better balance; did not present demographic characteristics, stratified by study group |
| Methods | Study design: randomized trial Study aim: evaluate effect of computer‐generated telephone reminders on improving on‐time vaccinations among children attending public clinics | |
| Participants | Inclusion: previously vaccinated at 2 public health clinics in southwest Fulton County; listed in clinics' file of current patients; due to receive DTP, OPV, or MMR during 6‐week study enrollment period; Atlanta, Georgia (USA) Participating clinics provide care for poor, minority populations | |
| Interventions | Intervention: autodialer, 1 per patient; calls made by telecomputer with pre‐programmed standard message using a normal human voice; message indicated the health department was reminding family that the child was due for an immunization or shot, bring child to health center any day during current week, immunizations are important to protect child's health from specific diseases, and immunizations are required for day care or school; calls made at beginning of the day before child was due for an immunization; up to 9 attempts were made, not counting wrong numbers; some attempts were made during evenings; n = 112 | |
| Outcomes | Childhood vaccines: number and percent of children receiving vaccinations on time Girls were slightly more likely to be vaccinated on time than boys Blacks, Hispanics, and children attending the larger clinic were somewhat more likely to have been vaccinated on time than whites, non‐Hispanics, and those attending the smaller clinic Younger children were more likely to receive vaccinations on time compared with older children | |
| Notes | 67.3% of intervention homes were reached with autodialer system Estimated intervention costs | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomization methods not described |
| Allocation concealment (selection bias) | Unclear risk | Reviewed clinic files to identify eligible children; randomized children; specific allocation method not described; intervention was delivered by telecomputer |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Followed children for 1 month beginning the date they became due to receive an immunization Abstracted data from clinic records at end of study Of 229 who met eligibility criteria, 6 were lost to follow‐up, and 1 was deferred from needing additional immunizations; randomized remaining 222 to study groups |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Unclear risk | Reviewed immunization records of children less than 2 years of age; abstracted information from patients' charts: date of birth, sex, race, ethnicity, date and type of previous immunizations, telephone number, and other services received at health center Children in control group were slightly younger, more likely to be female, and more likely to attend the larger clinic, than intervention participants Control participants were less likely to participate in other services offered by clinics than intervention participants |
| Methods | Study design: controlled before and after for adolescent study; "randomized trial"; 2 intervention sites and 4 control sites Selected random sample and compared to age‐ and sex‐matched controls Study duration: January 2009 to June 2009 Study aim: assess feasibility and efficacy of text message reminder‐recall on return of adolescents to their medical home for routine vaccinations | |
| Participants | Inclusion: parents or guardians of 11‐ to 18‐year olds with any visits at participating study site within previous 12 months; patients in need of meningococcal (MCV‐4), tetanus‐diphtheria‐acellular pertussis (Tdap) or both; and cell phone number was listed in registration system Age: adolescents, 11 to 18 years Exclusion: parents of adolescents who had received a different tetanus‐containing vaccine within previous 2 years n = 361; 195 parents randomly selected for the intervention group; included 166 controls, matched for age and sex 1656 patients at intervention sites and 1460 at control sites needed Tdap or MCV; 625 of these had a cell phone listed in registration system | |
| Interventions | Intervention: Text 4 Health ‐ Adolescents; parents received text messages at weeks 1, 2, 3, 6, and 7, to notify them of child's need for vaccination(s); messages were stopped if vaccination was recorded in electronic system; n = 2 practice sites and 195 patients Text messages included patient's first name, clinic name, a list of when immunizations could be obtained at the clinic, and how to discontinue additional messages Messages were sent in English or Spanish, based on listed preference Control: standard of care; did not include immunization reminders; n = 4 practice sites and 166 patients | |
| Outcomes | Primary outcome: receipt of 1 or both of 2 routinely recommended adolescent vaccines, meningococcal (MCV‐4) and tetanus‐diphtheria‐acellular pertussis (Tdap), at 4, 12, and 24 weeks after study assignment Secondary outcome: receipt of any vaccine, including MCV, Tdap, HPV, influenza, or others Intervention, for primary outcome at 12 weeks: 12.8 percentage points over control group; 26.7% versus 13.9% Intervention, for primary outcome at 4 weeks: 11.2 percentage points over control group; 15.4% versus 4.2% | |
| Notes | EzVac text messaging platform was developed and integrated into the hospital's immunization information system; system is linked to hospital registration and computerized order entry systems; immunization data synchronized with New York City's immunization registry Reported power calculations; sample size of 150 participants, 80% power, and alpha of 5% was conservatively powered to detect a 15 percentage point difference between intervention and control groups; this outcome was achieved by 24 weeks of follow‐up Authors report limitation of not knowing how persons without cell phone numbers in their system would differ from those with cell phone numbers in the system, other than demographic data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | "Two intervention and four control sites were assigned to provide comparable baseline populations…" |
| Allocation concealment (selection bias) | High risk | Each week a computer algorithm was used to automatically randomly select sample of eligible parents from intervention sites; selected control participants from control practices, matching for age and sex with intervention participants |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Collected immunization data from EzVac; outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | New York City Citywide immunization registry captures more than 85% of the immunizations administered in the city and an estimated 93% of free immunizations distributed through Vaccines for Children program |
| Selective reporting (reporting bias) | Unclear risk | "We did not include the human papillomavirus (HPV) and influenza vaccines because their uptake may have reflected unique parental attitudes and beliefs; in addition, the intervention extended beyond the influenza season." |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Baseline characteristics similar between study groups for sex, race and ethnicity, insurance status, primary language, and adolescent and childhood immunizations |
| Methods | Study design: randomized trial Study duration: February 2008 to August 2009 Study aim: assess effect of letter and autodialer reminder‐recall interventions on adolescent immunization rates | |
| Participants | Inclusion: adolescents seen at their practice at least once in 2 years before the study; needed one or more targeted adolescent vaccinations, including tetanus‐diphtheria‐acellular pertussis (Tdap), meningococcal conjugate (MCV4), or first dose of human papillomavirus (HPV1) vaccine for females Age: 11 to 18 years Setting: 4 suburban private pediatric practices in metropolitan Denver, Colorado (USA) n = 1600; 400 adolescents from each of 4 practices | |
| Interventions | Intervention: up to 2 letters separated by 2 autodialer telephone calls; all families were sent a first letter and autodialer call; adolescents who needed a targeted vaccination 1 month later received a second autodialer call; a second letter was sent 2 months after initial reminder‐recall if adolescent still needed immunizations Letters were printed on practice letterhead Letters and autodialer indicated that adolescents were due for at least 1 vaccine and briefly described each vaccination; n = 800 Control: usual care; did not include reminder‐recall; n = 800 | |
| Outcomes | Outcome 1: received at least 1 targeted vaccine, or more than 1 vaccine, 6 months after the intervention; described inconsistently between the abstract and page e1438 Outcome 2: received all targeted vaccines, 6 months after intervention Intervention ‐ Outcome 1: 12.5 percentage points above control group Intervention ‐ Outcome 2: 11 percentage points above control group Intention‐to‐treat used for all analyses | |
| Notes | 751 of 800 adolescents in intervention group received at least 1 recall autodialer call and letter If more than 1 adolescent in a family met the inclusion criteria, 1 was randomly selected to have data included; siblings received same intervention type as the enrolled sibling Power calculations were reported; with 200 adolescents per practice, study would have 80% power to detect a 7 percentage point difference in immunization rates between study groups | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomized patients within each practice using random number generation (SAS 9.1) |
| Allocation concealment (selection bias) | Low risk | Practices participate in the Colorado Immunization Information System, a statewide immunization registry and share a common billing system; used data combined from these systems to determine study eligibility; randomized within each practice using random number generator |
| Blinding of participants and personnel (performance bias) | Low risk | "Providers were blinded to group allocation" |
| Blinding of outcome assessment (detection bias) | Low risk | Outcome assessment blinding not specified; however, providers were blinded and outcome data were obtained from registry and billing data |
| Incomplete outcome data (attrition bias) | Low risk | Obtained outcome data from Colorado Immunization Registry; supplemented with billing data Final cohort sizes were 799 intervention and 797 control |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Administrative data from practices' electronic billing systems were merged with the Colorado Immunization Registry data to determine eligibility at baseline based on immunization records Intervention and control groups similar for age, sex, and insurance status Baseline vaccination rates varied by practice; not clear whether baseline rates varied by study group |
| Methods | Study design: randomized trial | |
| Participants | Inclusion: moderate to severe asthma; had acute asthma attack with administration of bronchodilators in prior year, or use of medications on a chronic basis 70% of visits are covered by Medicaid | |
| Interventions | Intervention: sent 1 computer‐generated letter to parents during October 1990; explaining that influenza season was approaching, child may develop influenza complications, influenza shot is highly protective and has minimal side effects, and asking parents to schedule a visit for influenza shot; written in sixth grade reading level; n = 63 | |
| Outcomes | Number and percent of patients receiving at least 1 influenza vaccination | |
| Notes | Initiated computerized database in 1987; database included 5400 patients by November 1989 Instructed all clinic nurses and doctors about importance of influenza vaccination for children with asthma; reminded them about clinic vaccination policy Placed checklist in front of medical charts of all eligible patients Did not report cost of intervention data | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Randomized children; allocation method not described |
| Allocation concealment (selection bias) | Unclear risk | Computer database was used to identify children with diagnosis of asthma; conducted chart review for those children to assess study eligibility; randomized children; method not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | January 1991: reminder group parents that received letter were contacted to determine parent characteristics February 1991: influenza vaccination status was determined by medical chart review |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Children in intervention and control groups were similar for age, sex, Medicaid enrollment, taking chronic asthma medications, and emergency department visit for asthma within 6 months of study enrollment Conducted cross‐validation study the next year by sending computer‐generated reminders to all patients with active asthma to assess whether influenza immunization rates would be similar; vaccination rates were similar between this group (27%) and prior year intervention group (29%) |
| Methods | Study design: randomized trial Study aim: assess the effect of telephone reminder‐recall interventions on adolescent immunization rates in urban practices | |
| Participants | Inclusion: adolescents with 1 or more visits at participating clinics Exclusion: siblings, randomly selecting one adolescent per family; no practice visits within 24 months; residing outside the county; no telephone number in database Rochester has high rates of childhood poverty | |
| Interventions | Intervention: autodialer; automated telephone message reminder system; number of calls varied per participant, based on need for immunizations or well child visits and responses to reminder calls Calls attempted 6 of 7 days a week, during day or early evening; made in English using a voice recording Interventions managed by central office; n = 1496; 132 considered inactive but included | |
| Outcomes | Intervention group, hepatitis B: 4.2 percentage point increase over control group; 62.0% versus 57.8%; difference is reported as 2.2 in Table 2 Intervention group, tetanus diphtheria (Td): 2.1 percentage point increase over control group; 52% versus 49.9% Intervention group, average values: 3.2 percentage point increase over control group Used intention‐to‐treat analysis | |
| Notes | Power calculations reported; more than 750 adolescents per practice were required to detect a "10%" improvement in vaccination rates; 3006 were eligible for randomization 62.8% did not respond to reminders; 3.4% were no longer clinic patients; 9.8% wanted calls discontinued Conducted medical record review to identify participant telephone numbers at beginning of study; investigators experienced difficulty with obtaining accurate numbers | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Allocated adolescents into study groups using random‐number generator |
| Allocation concealment (selection bias) | Low risk | Eligible adolescents identified through practice billing databases; stratified adolescents into 2 groups based on age, 11 to 12 years and 13 to 14 years, then randomly allocated into groups using random‐number generator; research personnel located at a central office |
| Blinding of participants and personnel (performance bias) | Low risk | Health services personnel unaware of group allocation for study participants |
| Blinding of outcome assessment (detection bias) | Low risk | Obtained primary outcome measures by blinded medical record review at study end using abstraction form |
| Incomplete outcome data (attrition bias) | Low risk | Tracked adolescents and need for reminders using database; research assistant verified weekly upcoming appointments, immunizations, and telephone number changes Conducted medical chart review at study end; records not found for 132 intervention and 168 control participants Some adolescents may have received vaccinations in other locations |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Conducted quality assessment checks for 5% of participant records; 98% reliability |
| Baseline measurement | Low risk | At baseline, intervention and control groups were similar for hepatitis B, Td coverage and well child visit rates Collected demographic data from billing files Intervention and control groups similar for age group, sex, practice distribution, insurance, race, and ethnicity |
| Methods | Study design: randomized trial Study duration: conducted intervention from 1 October 2007 to 31 December 2008; assessed outcomes for 3 months after intervention Study aim: evaluate effect of tiered patient immunization navigator intervention, which included telephone calls and letters, on improving immunizations among adolescents living in urban areas | |
| Participants | Inclusion: adolescents enrolled in participating practices Age: 11 to 15 years; birth dates from 1 July 1992 to 30 June 1997 Setting: 8 largest urban primary care practices serving adolescents in Rochester, New York; included 2 federally qualified community health centers, 2 pediatric hospital‐based clinics, 1 family medicine teaching clinic, 1 hospital‐associated medicine‐pediatrics practice, and 2 urban private practices (USA) n = 7546 from 6682 families; 5910 families had one adolescent Almost 80% of adolescents in Rochester live below poverty line | |
| Interventions | Intervention: tiered intervention; population‐based approach with progressively more intensive intervention, based on need; n = 3707 Step 1: track participants in tracking system Step 2: reminders and recall for vaccination or preventive care visit with 1 month grace period; 2 telephone calls at least 1 week apart, made by navigators; offered transportation assistance, if needed; 2 letters 2 weeks apart after calls or if telephone numbers not available Step 3: navigators made home visit to assess barriers to seeking care, promote prevention, and encourage appointments Control: standard of care; n = 3839 | |
| Outcomes | Immunization rates for 3 individual vaccine types, and all 3 vaccines combined; meningococcus, pertussis, and HPV for girls Differences in immunization rates between intervention and control groups ranged between "12% to 16%," depending on vaccine Intervention ‐ MCV4: 14.3 percentage points over control group; 63.9% versus 49.6% Intervention ‐ Tdap: 12.1 percentage points over control group; 65.5% versus 53.4% Intervention ‐ first HPV: 15.6 percentage points over control group; 58.5% versus 42.9% Intervention ‐ second HPV: 15.8 percentage points over control group; 52.0% versus 36.2% Intervention ‐ third HPV: 12.4 percentage points over control group; 36.5% versus 24.1% Intervention ‐ all 3 vaccines: 12.3 percentage points over control group; 44.7% versus 32.4% | |
| Notes | Data not entered in RevMan data tables; allocated families, grouping siblings; stratified by practice, age and sex Intervention complexity makes it difficult to determine effectiveness of each component Study occurred before practices used state's immunization registry for adolescent immunizations All participating practices routinely used telephone or letter reminders for families with upcoming scheduled visits; reminders did not include active immunization reminders or recall | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Randomly assigned each family to study groups using commercially available software program, stratifying on practice, age and sex |
| Allocation concealment (selection bias) | Low risk | Identified families with age‐eligible adolescents from major insurance company databases and billing systems; randomly selected referent adolescent; randomly assigned families using computer program |
| Blinding of participants and personnel (performance bias) | Low risk | Health services providers not aware of study group assignment |
| Blinding of outcome assessment (detection bias) | Unclear risk | Trained patient immunization navigators delivered the intervention and used a web‐based database to track adolescents and document immunizations, preventive care visits, and tasks performed |
| Incomplete outcome data (attrition bias) | Low risk | Created web‐based database for navigators to track adolescents and document immunizations, preventive care visits, and tasks Reviewed medical records after intervention period and used abstraction form to obtain all adolescent immunization dates Searched New York State immunization registry for additional immunizations given |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Assessed reliability of medical record review abstraction using 5% sample; Kappa >= 0.89 for interrater reliability |
| Baseline measurement | Low risk | Obtained vaccination history from medical record review Identified eligible participants using lists from 2 major insurance plan databases for 8 practices and from billing systems Searched New York State immunization registry for additional immunizations given at baseline Intervention and control groups similar for demographic characteristics and baseline immunization and preventive care visit rates |
| Methods | Study design: randomized trial Study duration: 1 year; 11 December 2009 to 12 December 2010 Study aim: evaluate effect of managed care‐based letter and autodialer reminders and recall on immunization rates among low‐income adolescents | |
| Participants | Inclusion: adolescents enrolled in Monroe Plan on 31 December 2009; primary care provider participating in study Age: 10.5 through 17 years; mean age at study start was 14.4 years Setting: 15 counties in upstate New York state; 37 participating primary care practices that each served at least 30 eligible adolescents, enrolled in Monroe Plan for Medical Care, a large not‐for‐profit managed care organization that serves more than 72,000 publicly insured children enrolled in Medicaid or New York State Children's Health Insurance Program Practices: 22 pediatric; 13 family medicine; 2 internal medicine; 1 other dropped out n = 7404 adolescents from 5559 families were randomized into 3 study groups; 3289 lacked a telephone or geocodable address; 4115 youths remained in the study | |
| Interventions | Intervention group 1: mailed letter asking parents to call primary care practice to schedule appointment; listed telephone number; centralized reminder and recall; letters written in English and Spanish; 2‐sided; written at less than seventh grade reading level; specified age of child, but not name, managed care organization, primary care practice, and recommended services; letters sent at 10‐week intervals for Tdap, MCV4, and first HPV dose; sent letters at 5‐week intervals for HPV‐2 and HPV‐3, with maximum of 8 reminders per vaccine dose; n = 1396 Intervention group 2: autodialer telephone reminders in English or Spanish; centralized reminder‐recall; same content and frequency as letters; n = 1423 Managed care organization developed automatic algorithm that reviewed vaccination status every 5 weeks, triggering reminders, starting at 10.8 years of age Control: standard of care; some practices used visit or immunization reminders or recall; n = 1296 | |
| Outcomes | Immunizations rates for routine adolescent vaccines: meningococcus, pertussis, HPV Outcome 1: received all needed immunizations among adolescents missing any vaccinations at study beginning Outcome 2: immunization rates at study end, among all eligible adolescents at study start Group 1, letter, outcome 1: 8 percentage points over control group; 21% versus 13% Group 2, autodialer, outcome 1: 4 percentage points over control group; 17% versus 13% Group 1, letter, outcome 2: 6 percentage points over control; 56% versus 50% Group 2, autodialer, outcome 2: 3 percentage points over control; 53% versus 50% | |
| Notes | Data not entered in RevMan; allocated siblings to same study group; 73% of families had 1 adolescent Survey of participating practices revealed 12 of 24 respondents used telephone or mailed reminders for adolescents with scheduled preventive care visits; 6 of 24 used telephone or mailed reminders for patients behind on vaccines Managed care organization lacked telephone numbers for 41% of those initially randomized to study groups | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Used Stata to randomly assign referent adolescent per family and age‐eligible siblings to study groups |
| Allocation concealment (selection bias) | Low risk | Study was based at a managed care organization serving 72,404 children and adolescents; used managed care database to select practices serving at least 30 adolescents; randomized families and adolescents within each participating practice using computer program (Stata); allocated siblings to same group based on address and geo‐coding software Stratified based on practice, age in years, and sex |
| Blinding of participants and personnel (performance bias) | Low risk | Health services providers not aware of study group assignments |
| Blinding of outcome assessment (detection bias) | Low risk | Outcome assessment blinding not specified; however, immunization outcomes obtained from claims data and immunization registry data; and health services providers not aware of study group assignments |
| Incomplete outcome data (attrition bias) | Low risk | Used managed care organization claims files to obtain vaccination data; merged data with New York immunization registry data |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Unclear risk | Survey of participating practices revealed 12 of 24 respondents used telephone or mailed reminders for adolescents with scheduled preventive care visits; 6 of 24 used telephone or mailed reminders for patients behind on vaccines; randomized within practices to minimize so the effect of these interventions would be similar across study groups Many adolescents did attend clinic visits |
| Baseline measurement | Low risk | Determined eligibility for vaccines based on 2010 Advisory Committee on Immunization Practices (ACIP) guidelines Obtained demographic data from managed care organization's enrollment files and vaccination data from claims files Intervention and control groups similar, at baseline, for demographic characteristics, immunization status, and preventive visit rates |
| Methods | Study design: randomized trial | |
| Participants | Inclusion: received first or second DTP from main health department clinic Exclusion: siblings of included participants Main clinic and study site in Everett; 2 other clinics in southern and eastern sections of county; immunizations also available at well‐child clinics throughout county, 1 day each month | |
| Interventions | Intervention: sent 1 to 2 postcard reminders to parent or guardian listed in immunization record; sent first postcard to children overdue for immunizations 1 month after due date; sent second postcard a month later if immunization not received; n = 182 followed | |
| Outcomes | Number and percent immunized for DTP | |
| Notes | Washington State Immunization Program required use of manual or computerized follow‐up and recall system in all local health departments that obtain state‐purchased vaccines; each county had flexibility to develop their own systems and methods | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | High risk | Sequence generated by week children received vaccination at time of enrollment |
| Allocation concealment (selection bias) | High risk | Allocated children to the intervention or control group systematically using alternate weeks |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Followed 393 children, including 182 intervention and 211 control, for 5 months from enrollment or until the next DTP was administered At study end, parents of children in both groups without evidence of returning to clinic for immunizations were sent letter explaining study and enclosed questionnaire to determine if immunizations obtained at different location and name of site; made telephone calls to parents not returning questionnaire Immunization status obtained for 87.9% of children in intervention group and 84.4% in control group |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Used birth certificates to collect demographic and other data about child and family Study groups similar for family size at birth, socioeconomic status, race, age of child, age of parents, and month prenatal care began |
| Methods | Study design: randomized trial Study duration: 10 weeks follow‐up; reviewed medical records between October 1998 to December 1998 to determine baseline immunization levels Study aim: evaluate whether mailed and telephone recall systems are effective for increasing immunization among young children in Medicaid managed care practice | |
| Participants | Inclusion: children enrolled in Rite Care, Rhode Island's Medicaid managed care program; continuously enrolled in participating primary care clinics during July, August, and September 1998; underimmunized, defined as overdue for diphtheria and tetanus toxoids, pertussis, polio, Haemophilus influenzae type b, measles‐mumps‐rubella, or hepatitis B vaccines Age: less than 6 years as of 30 September 1998 Setting: primary care clinics at Hasbro Children's Hospital ‐ Rhode Island Hospital, university‐affiliated teaching hospital; Rite Care, Rhode Island's Medicaid managed care program, Providence, Rhode Island (USA) Clinics serve more than 10% of 50,000 children enrolled in Rite Care n = 264; control = 71; telephone group = 60; mail reminder = 63; sequential mail and telephone = 70 | |
| Interventions | Intervention group 1: telephone reminder; telephone calls made to families by clinic receptions who spoke English and Spanish; informed parents that children were overdue for immunizations and requested they make appointments with primary care provider during the call, if possible; made at least 3 call attempts per family, morning, afternoon, and early evening; n = 60 Intervention group 2: mail reminder; sent letter to family, indicating child was overdue for immunizations; parents were encouraged to call clinic to schedule appointment with primary care provider; n = 63 Intervention group 3: sequential mail and telephone reminder; mailed letter, following by telephone call one week later, if appointment not in scheduling system; n = 70 Control: no intervention; n = 71 | |
| Outcomes | Outcome 1: children received all needed immunizations at end of 10‐week follow‐up period Outcome 2: children received immunizations during study period Group 1, telephone, outcome 1: 10.5 percentage points over control group; 13.3% versus 2.8% Group 2, letter, outcome 1: 11.5 percentage points over control group; 14.3% versus 2.8% Group 3, letter and telephone, outcome 1: 14.3 percentage points over control group; 17.1% versus 2.8% Group 1, telephone, outcome 2: 11.5 percentage points over control group; 16.7% versus 4.2% Group 2, letter, outcome 2: 14.8 percentage points over control group; 19.0% versus 4.2% Group 3, letter and telephone, outcome 2: 21.5 percentage points over control group; 25.7% versus 4.2% | |
| Notes | Participating clinics did not have immunization outreach program before this study 53% in phone reminder group not contacted; 30.2% of letters returned | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Used computer‐generated random numbers to randomize children |
| Allocation concealment (selection bias) | Low risk | Children were enrolled in Rhode Island’s Medicaid managed care program; determined eligibility and randomized children using computerized immunization tracking system, which operated in a commercially available database |
| Blinding of participants and personnel (performance bias) | Unclear risk | English and Spanish‐speaking receptions made calls for the telephone reminder group Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified for medical record reviews, conducted to obtain baseline data, and to determine nurse‐only visits and newly received or documented immunizations after 10‐week follow‐up Assessed immunization status using these data and immunization tracking system |
| Incomplete outcome data (attrition bias) | Low risk | Used immunization tracking system to obtain immunization outcome data Reviewed medical records to determine nurse‐only visits, and newly received or newly documented vaccinations |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes to answer study questions |
| Other bias | Low risk | Study seems to be free of other sources of bias |
| Baseline measurement | Low risk | Used computerized immunization tracking system to identify immunization status at baseline Groups similar at baseline for age, sex, and vaccine‐specific rates |
| Methods | Study design: randomized trial Study duration: 6‐month follow‐up period; telephone calls made to persons with chronic conditions during June 2004 and older adults during July 2004 Study aim: evaluate effect of telephone reminder on pneumococcal vaccination rates, and compare effectiveness among primarily non‐Hispanic black versus non‐Hispanic white patient populations | |
| Participants | Inclusion: all patients at 5 participating managed care network general medicine clinics; unvaccinated based on administrative database; chronic disease specified as diabetes mellitus, chronic heart failure, or coronary artery disease in database Age: 18 years and older for chronic disease group; older than 65 years for older adult group Exclusion: patients vaccinated or indicated, by postcard, they had received vaccine at different site within 3 months after mailed reminder Setting: 5 managed care network general medicine clinics; Atlanta, Georgia (USA) n = 6106; 3711 with chronic disease; 2395 older adults | |
| Interventions | Intervention: telephone reminder; nurses made calls, asked patients about pneumococcal vaccination and explained vaccine is recommended and is covered benefit of health plan with copayment; asked patients if they wanted to receive vaccine; could schedule vaccination appointment during call; n = 3043, including 1845 with chronic diseases and 1198 older than 65 years During spring 2004, before intervention, practice sent letter to intervention and control participants, to introduce study and indicate a nurse would call in next few weeks Control: usual care; did not receive introductory study letter; n = 3063, including 1866 with chronic diseases and 1197 older than 65 years Sent mailed reminders to both groups during March and April 2004, encouraging patients to schedule clinic visit to receive pneumococcal vaccination, or return enclosed postcard if received at different setting | |
| Outcomes | Number and percent of persons receiving pneumococcal vaccination Intervention ‐ chronic disease group: 10 percentage points over control group; 16% versus 6% Intervention ‐ greater than 65 years: 9 percentage points over control group; 17% versus 8% Intervention ‐ combined: 9.2 percentage points over control group; 16.1% versus 6.9% Intention‐to‐treat analysis | |
| Notes | Posted preventive services reminders in all medical offices | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Allocated patients to intervention and control groups using random number generator; one‐to‐one ratio |
| Allocation concealment (selection bias) | Low risk | Identified eligible participants using administrative database for 5 clinics; allocated patients to intervention and control groups using random number generator; one‐to‐one ratio |
| Blinding of participants and personnel (performance bias) | Low risk | "The study was blinded; randomization assignment was not known to the patient’s primary care physician or home medical office." |
| Blinding of outcome assessment (detection bias) | Low risk | "Used Current Procedural Technology code for vaccination as the outcome"; "The study was blinded" |
| Incomplete outcome data (attrition bias) | Low risk | Obtained primary outcome by identifying participants with CPT code 90732 in administrative database |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes to answer study questions |
| Other bias | Unclear risk | Sent mailed reminders to intervention and control patients during spring 2004 At baseline, "large proportion" of intervention participants reported receipt of pneumococcal vaccination previously, but not documented in their records; these patients were included in study; similar data not available for controls |
| Baseline measurement | Low risk | Identified vaccination status at baseline and enrolled participants without pneumococcal vaccination Compared study groups for age, length of HMO enrollment, sex, and chronic disease distribution; observed minor differences between intervention and control group |
| Methods | Study design: randomized trial | |
| Participants | Inclusion: inner‐city children within 10 zip code areas; born to African American woman; enrolled infants during the first few weeks of life Area children predominantly African American and from low income families | |
| Interventions | Intervention: case management with phone calls and health passport Case managers conducted in‐depth assessments in home before children were 6 months of age; home visits scheduled 2 weeks before next immunization due date; discussed immunization schedules and misconceptions about contraindications for vaccinations; made telephone calls or home visits after scheduled well‐child visits to assess compliance with care; assisted families with overcoming barriers, such as transportation or lapses in Medicaid coverage Visit frequency varied based on parent compliance with care Home visits occurred at approximately 3.5 and 5.5 months of age for children receiving timely visits and immunizations; n = 209 | |
| Outcomes | Number and percent up‐to‐date with childhood immunizations at 1 year of age | |
| Notes | — | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | RAND survey employees randomized mother‐infant pairs in blocks of 4, prior to baseline interview |
| Allocation concealment (selection bias) | Low risk | Obtained lists of names and addresses from the county vital statistics branch to identify births in 10 target zip code areas; randomization was conducted centrally by RAND survey employees |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Low risk | Survey staff made one contact with parent when infants were 4 to 5 months of age to update addresses and telephone numbers Collected information from both groups by face‐to‐face interview at end of intervention period; included recalled information only if parents had complete dates and specific immunization provided 71% of mothers provided written records with valid immunization information at exit interview Immunization data also obtained by abstracting provider charts for 299 (82%) children Provider records incomplete or unavailable for 40 children Combined data, then omitted redundant information; immunization data available for 89% of final sample |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes to answer study questions |
| Other bias | Low risk | Compared immunizations recorded in provider records and hand‐held records with recall data, if provided; 7.4% of group recalled at least 1 immunization not recorded in provider or hand‐held records Validated recall data with provider records for 66 of 106 respondents Recall data for 17 children (4.7%) met data inclusion criteria |
| Baseline measurement | Low risk | Collected baseline data from both groups by face‐to‐face interviews Intervention and control groups were similar for child's birth order, mother's educational level, maternal age, knowledge of immunization schedule, level of support available from family, maternal work in past year, life difficulties score, and receipt of prenatal care Control group less likely to be living with a partner than intervention group |
| Methods | Study design: randomized trial | |
| Participants | Inclusion: 25% sample from Ohio's live, legitimate resident births during March 1978, classified as "high risk"; High risk: had at least 1 parent with less than high school education, regardless of family size; or only 1 parent with some college education and family consisted of 4 or more children, including enrolled child Controls: selected 10% sample from infants identified as "high risk" | |
| Interventions | Intervention: reminder letter to parents of high risk children, timed to be received 1 October 1978; n = 253 randomized; 179 responded to questionnaire (69.2%) | |
| Outcomes | Outcome 1, number and percent of children receiving childhood vaccines: 16 percentage point increase over control group Outcome 2, number and per cent of children receiving all needed vaccinations: 12 percentage point increase over control group | |
| Notes | Based on survey of 2‐year old immunization levels, parental education and family size were found to be risk factors for failing to complete immunization series by 2 years of age | |
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Unclear risk | Selected 10 percent sample from a list of all live births from 1 month and 1 state, classified as high risk children, to serve as controls; parents of other high risk children received the intervention; randomization method not described |
| Allocation concealment (selection bias) | Unclear risk | Used Ohio Department of Health birth certificate data to identify eligible children and classify them, by computer, as high or low risk; randomization procedures not described |
| Blinding of participants and personnel (performance bias) | Unclear risk | Blinding of participants and personnel not specified |
| Blinding of outcome assessment (detection bias) | Unclear risk | Outcome assessment blinding not specified |
| Incomplete outcome data (attrition bias) | Unclear risk | November 1978: sent questionnaire to intervention and control parents to inquire about immunization actions taken during October 1978 Contacted parents by telephone if did not respond to mailed questionnaire Obtained questionnaire responses from 70% of participants, 70.5% of control and 69.6% of intervention parents |
| Selective reporting (reporting bias) | Low risk | Study purpose and methods are described; published data included all expected outcomes to answer study questions |
| Other bias | Low risk | Compared random sample of mailed questionnaire and telephone responses with provider records; "No inaccuracies were detected"; size of this sample not clear |
| Baseline measurement | Unclear risk | Based on the questionnaire and telephone data, groups similar for DTP and polio immunization rates before letter was sent; did not report comparisons for other characteristics |
Abbreviations:
ACIP: Advisory Committee on Immunization Practices
CBA: controlled before and after study
CHIP: Children's Health Insurance Program
CI: confidence interval
DTP or DTaP: diphtheria tetanus pertussis vaccine
GP: general practitioner
H. flu: Haemophilus influenzae
Hib: Haemophilus influenzae type B vaccine
HIT: health information technology
HMO: health maintenance organization
MCV: meningococcal vaccine
MMR: measles, mumps, rubella vaccine
OPV: oral poliovirus vaccine
OR: odds ratio
PCV: pneumococcal vaccine
PV: poliovirus
Td: tetanus diphtheria
Tdap: tetanus diphtheria acellular pertussis vaccine
TOPV: trivalent oral polio vaccine
VAR: varicella vaccine
Characteristics of excluded studies [ordered by study ID]
Ir a:
| Study | Reason for exclusion |
| Article was retracted (USA) | |
| Multi‐modal intervention, consisting of lecture, email reminders, and recruitment of key staff member in each clinic who personally approached each staff member to ask them to get influenza vaccine (Israel) | |
| Primary intervention was text messages; however, intervention and comparison group used appointment card and financial incentives for enrollment; intervention group also received financial incentive for completing post‐intervention interview (USA) | |
| Influenza vaccination collected through self‐report by telephone and mailed survey; 4 study groups included: 1 postcard; 2 postcards; 1 postcard and employer toolkit; 2 postcards and employer toolkit; no true control group (USA) | |
| Study design: not randomized trial, CBA, or ITS (USA) | |
| Study design: cross‐sectional; no controls (USA) | |
| Self‐selected participants; first 24 consecutive participants received intensive educational intervention; next 45 participants received educational intervention and text message follow‐up; HPV vaccination data obtained by self‐report by telephone (USA) | |
| Obtained immunization data by self‐report through telephone survey; compared postcard reminder to mailed informational brochure; not a true control group (USA) | |
| Compared reminder letter with invitation letter for home visit health check by nurse with immunization offered in home; no true control group; Melton Mowbray, Leicestershire (England) | |
| Not a study | |
| 4 potential interventions, varied based on contact information; no control (USA) | |
| Primary intervention was use of community volunteers to conduct outreach, generally home visits, with some telephone follow‐up and initial letters to introduce study (USA) | |
| Study design: not randomized trial, CBA, or ITS (USA) | |
| Study design: survey (Australia) | |
| Intervention: mailed "marketing" piece Primary outcomes: inpatient hospitalizations and emergency department visits | |
| Sent intervention letters in bulk mail, those not delivered were not identified or tracked; presented immunization outcome data as numbers per 10,000 persons rather than presenting numerators and denominators; influenza vaccination was measured using insurance claims data only (USA) | |
| Sticker intervention did not meet intervention type inclusion criteria. (Ethiopia) | |
| Immunization data obtained by parent report from approximately 43% of participants; MMR vaccination rates may have been influenced by other vaccination campaigns that occurred in relation to large, university‐based measles outbreak (Canada) | |
| Only 3 time periods in a time series; Great Fall, Montana (USA) | |
| Intervention included standing order policies (USA) | |
| Intervention included audit and feedback, provider education, and other interventions (USA) | |
| Study design unclear, possible ITS; cannot determine effects of patient reminder because a package of interventions was tested; no true baseline data (USA) | |
| Not randomized trial, CBA, or ITS; retrospective cohort study design (USA) | |
| Study results not presented; letter intervention not clearly described, possibly a handout when patients went to a pharmacy | |
| Intervention was provider reminders through chart‐based prompts (USA) | |
| Outcome was measles, not vaccination (England and Wales) | |
| Health workers in all communities were expected to provide some type of reminder; no true control group (Guatemala) | |
| Not CBA, randomized trial or ITS (USA) | |
| Offered participants free vaccines on a flexible schedule (USA) | |
| Intervention included educational posters and clinical reminder in electronic health record | |
| Quasi‐experimental design; possibly before and after with historical comparison; convenience samples (USA?) | |
| Tested multiple interventions to improve influenza vaccinations, such as use of vaccine cart, vaccine days, free vaccinations, and education; not specifically patient reminder or recall (USA) | |
| Compared 3 different types of personal telephone calls (Italy) | |
| Assessed whether a required signed written consent affected influenza vaccination acceptance among older adults; sent letters to study and control participants (Canada) | |
| Both study groups received text messages (China) | |
| Intervention is not fully clear; included tracking effort and clinician notification (USA) | |
| Multiple interventions including mailed reminders, clinician training, quality improvement, clinician financial incentives to send reminders, school‐based intervention with phone calls (USA) | |
| Sent postcard to intervention group; comparison group received standard educational intervention, not clearly described (USA) | |
| Study design: not randomized trial, CBA, or ITS (USA) | |
| Text messages and letter versus letter; no true control group; outcomes were timeliness of vaccination (USA) | |
| Intervention involved clinician education and surveys | |
| Intervention was postcard‐sized handout distributed at a clinic (Australia) | |
| Study design: not randomized trial, CBA, or ITS (UK) | |
| Church‐based intervention; one group received immunization education; other group involved on‐site immunizations (USA) | |
| Intervention included point‐of‐service provider reminders (USA) | |
| Intervention was computerized provider reminder system in emergency department (USA) | |
| Intervention was public health nurse visit to work sites that included distribution of promotional materials and information about where to obtain free vaccines (UK) | |
| Outcome: kept immunization appointments (USA) | |
| Intervention consisted of several activities, including guidelines for managers, training in supportive supervision, monitoring and evaluation, and funding for immunization‐related activities (Republic of Georgia) | |
| Probably retrospective cohort study; if reminders were not received, participants were classified as controls; controls were not‐comparable (USA) | |
| Outcome vaccination data were collected by nurse from parents, possibly by self‐report; usual care comparison received written reminders in immunization card; study protocol was reviewed with all participants (Guatemala) | |
| Comparison group received general intervention, including exposure to posters, newsletters, t‐shirts, buttons, departmental meetings, access to expanded hours at influenza vaccination stations; no true control group (USA) | |
| Compared 3 different telephone recall interventions contrasting different people making calls (Italy) | |
| Intervention was audiovisual message about tetanus booster vaccination in clinic waiting rooms (Belgium) | |
| Participants were swapped from intervention and control group after randomization if they did not have cell phones (Nigeria) | |
| Intervention was influenza vaccine provider clinical alerts (USA) | |
| Intervention included provider prompts and tested the Immunization Action Coalition's "Do I need any vaccinations today?" (USA) | |
| Methods described in separate report (Canada) | |
| Intervention: provider reminders (Australia) | |
| Cost and cost‐effectiveness study (USA) | |
| Intervention was academic detailing (USA) | |
| Sent letters and postcards to intervention groups; health information group received message "Health is the prize when you immunize"; Law Message group received message "If your kids don't get their shots on time ‐‐ it's a crime"; interventions not specifically reminders or recall (USA) | |
| Randomized study groups to 2 different immunization schedules; used various interventions to remind parents about vaccination, including home visits, telephone calls, and postcards; may have provided interventions to both study groups (USA) | |
| Uncontrolled before and after study with family reminders, provider reminders, education and other interventions (USA) | |
| Intervention is in person discussion between patient and pharmacist and provision of vaccine record to patient; before and after study design (Germany) | |
| Adolescents in one county received school‐based influenza vaccination education and free vaccination at school‐based vaccine clinic; in provider‐based county, adolescents received education and free vaccination by local health provider; and controls received neither intervention; non‐randomized study (USA) | |
| Study design: not randomized trial, CBA, or ITS (USA) | |
| Study design: not randomized trial, CBA, or ITS (Canada) | |
| Intervention included patient and provider reminders; before and after study design (USA) | |
| Systematic review; interventions delivered by lay health workers | |
| Randomly assigned participants to telephone interview group or comparison group; some telephone reminders occurred; not clearly reminder or recall study; tested effect of parental survey of attitudes on immunization rates (USA) | |
| Intervention consisted of door‐to‐door outreach by emergency medical technicians to determine immunization status and encourage participants to get well‐care visits and immunizations; before and after study design (USA) | |
| Randomized practices to multi‐component intervention group or usual care control group; intervention consisted of brief introduction letter, enclosed immunization information, and follow‐up telephone calls; intervention was only delivered to 42% of eligible children; some practices were doing recall before the study (New Zealand) | |
| Tested Put Prevention Into Practice intervention utilizing office‐based interventions; before and after study design (USA) | |
| Editorial | |
| Randomized trial with 3 study groups: nurse case management plus tracking and incentives; standard management, incentives, and tracking; standard management and incentives (USA) | |
| Compared clinic‐based vaccination with school‐based vaccination (Canada) | |
| Study of mammography with discussions of immunizations (Canada) | |
| Study design: not randomized trial, CBA, or ITS; retrospective questionnaire of one‐third of all 4758 general practitioners (The Netherlands) | |
| Intervention includes intensive reminder and recall with audit and feedback, incentives, and other interventions; does not specifically test effect for patient reminder or recall intervention (USA) | |
| Study design: compared 2 interventions; no real control group | |
| Compared postcard with message based on health belief model to postcard with neutral message (Australia) | |
| Intervention included health education, reminders, registry, newsletter, and refrigerator memo with contact information; insufficient data points in time series (USA) | |
| Comparison practices used own call or recall system; not a true comparison (Scotland) | |
| Comparison practices implemented usual seasonal influenza vaccination campaign, such as posters, and letters to patients (UK) | |
| Uncontrolled before and after study with recall cards combined with posters in clinic examination rooms (USA) | |
| Intervention participants enrolled in Women's Infant and Children (WIC) food and nutrition program were given vouchers once per month instead of every 3 months, until child was up‐to‐date; intervention was supplemented with telephone calls and mailings when voucher intervention did not work (USA) | |
| Usual care control group included influenza vaccine clinical decision support in the electronic health record and "automated phone call reminders for appointments and general information about influenza vaccination procedures provided in the clinic." (USA) | |
| Usual care included automated telephone appointment reminders (USA) | |
| Study design: controlled study without baseline data (Finland) | |
| Study design: survey (USA) | |
| Study design: longitudinal study without control group (Canada) | |
| Postcard and telephone reminders focused on appointments and were blinded to immunization status; unclear whether reminders focused on immunizations (USA) | |
| Intervention was brochure given to patients at visits (USA) | |
| Study design: possibly CBA; randomly selected intervention participants from rural area to receive immunization reminder letter; this geographic area also received multi‐faceted community‐wide pneumococcal immunization campaign, including television and newspaper advertisements, posters, and brochures; additional participants were selected from a geographic area that did not receive media campaign; compared pneumococcal vaccination rates between patient reminder letter group and education campaign plus letter group (USA) | |
| Randomized to "usual" text message or "enhanced" text message; no true control group (USA) | |
| Control participants were sent mailing with list of resources that offered free vaccines (USA) | |
| Intervention was postcard reminder followed by telephone reminder; comparison group also received postcards; no true control group (USA) | |
| Intervention not patient reminders; study design not randomized trial, CBA, or ITS (USA) | |
| Involved 3 recall methods within school setting: sent pass to students in class to go to clinic; phone call was made to classroom; and clinic staff member went to classroom to take student to clinic (USA) | |
| Compared centralized versus practice‐based reminder and recall; no true control group (USA) | |
| Practice‐based versus population‐based recall (USA) | |
| Compared centralized reminder and recall intervention with practice‐based reminder and recall interventions (USA) | |
| Text messages were either delivered alone, with autodialer, or with email; results not provided separately (USA) | |
| Centralized reminder and recall versus practice‐based reminder and recall; no true control group (USA) | |
| Study design: not randomized trial, CBA, or ITS (USA) | |
| Parents self‐selected to receive text messages; parents who opted not to receive test messages served as comparison group; also used historical comparison group (USA) | |
| Parents self‐selected into intervention group to receive text message reminders; controls had opted out of text message interventions (USA) | |
| Study design: cohort study (New Zealand) | |
| Study design: pre‐test post‐test (USA) | |
| Obtained immunization outcomes by self‐report; control group received some interventions; multiple interventions (USA) | |
| Study design: cross‐sectional (USA) | |
| Study design: not randomized trial, CBA, or ITS (USA) | |
| Study design: survey; used registry for intervention (Canada) | |
| Study design: not randomized trial, CBA, or ITS (USA) | |
| Study design: compared 2 reminders; no true control group (Australia) | |
| Telephone reminder focused on well‐child visits (USA) | |
| Patient reminders may have been integrated into broader intervention, consisting of continuing medical education and office systems; immunization outcomes cannot be clearly associated with patient reminders (USA) | |
| Study design: not randomized trial, CBA, or ITS (Hong Kong) | |
| Sustainability of previous study (Canada) | |
| Study design: not randomized trial, CBA, or ITS; multiple interventions and outcomes; complete data not presented (USA) | |
| Sent reminder letters to all participants; some were influenza vaccine reminder letters; some with either a prompt to write date when planning to get vaccine or date and time to get vaccine; midwestern utility firm | |
| Some outcomes were assessed by self‐report (USA) | |
| Study design: not randomized trial, CBA, or ITS (USA) | |
| Probably a retrospective or prospective cohort study design (USA) | |
| In one intervention group, sent questionnaire to parents to obtain details about immunization status; data source for analysis not clearly described (Wales) | |
| Primary control group comprised people who declined participation; secondary control group comprised people who "were not contacted by phone" but met eligibility criteria (USA) | |
| Intervention was immunization reminders to parents when child was hospitalized, reminding parents to contact care provider for immunization appointment after hospitalization; measured immunization status by self‐report (Switzerland) | |
| Intervention included vaccine planning, staff education, paycheck notices reminding employees where to obtain vaccines, vaccination access at work, contact with unimmunized staff, data tracking, and performance feedback; time series lacked sufficient data points (USA) | |
| Study design: not randomized trial, ITS, or CBA; study of computer intervention (England and Wales) | |
| Study design: not randomized trial, CBA, or ITS (USA) | |
| Study design: cross‐sectional; not randomized trial, CBA, or ITS (USA) | |
| Multiple interventions, including annual publicity mailing, walk‐in clinics, and nurse standing orders; time series did not include sufficient baseline data collection or clear intervention points; vaccination status obtained by survey (USA) | |
| All participants received routine reminders from health services providers; financial incentives were given to encourage study participation (USA) | |
| Report; not a study | |
| Each of 5 intervention sites delivered combination of patient‐, provider‐, and system‐oriented strategies; specific interventions not clearly described for each site (USA) | |
| Interventions were menu of patient reminders, provider reminders, provider education, and systems‐based interventions; combination interventions did not clearly fit into our comparison categories (USA) | |
| Control group received emails and other advertising; collected data from employer surveys; one intervention group received financial incentive (USA) | |
| Study groups included: invitation letter to receive influenza vaccination; letter and "leaflet"; and letter and invitation to visit at home; no true control group (United Kingdom) | |
| Interventions included nurse case management with incentives and tracking, standard care with incentives and tracking, and standard care with incentives (USA) | |
| Study design: ITS with fewer than 2 data points (USA) | |
| Focus on adherence to lifestyle and other medical recommendations; not clear if immunizations will be an outcome in this proposed trial (Spain) | |
| Financial incentive for questionnaire; mailing of information to intervention and comparison group; no true control group; multi‐level intervention (USA) | |
| Intervention: face‐to‐face discussion at clinic, review of written information, and mailing of packet with reminder letter and information; comparison: given flier about HPV vaccination at clinic visit; used self‐report for some outcomes (USA) | |
| Mailed reminders were sent from 2 control practices; participants self‐selected intervention types, including text, email, phone, Facebook, or standard mail; excluded patients who did not want to be contacted with reminders (USA) | |
| Study design: not clear; polio campaign may distort findings (Finland) | |
| Compared 3 different brochures to enhance influenza vaccination (Thailand) | |
| Study aim: validate computer tracking system (USA) | |
| Intervention involved outreach, telephone call attempts, and mailed brochures about 5 refused preventive services; may be refusal conversion study rather than reminder or recall; possibly CBA study design (USA) | |
| Study design: post‐hoc analysis of clustered randomized trial; tracked "inactive" infants; not focused specifically on patient reminders (USA) | |
| Intervention is "Standards for Pediatric Immunization Practice" rather than patient reminders | |
| Intervention was audit and feedback with supplemental outreach to intervention group providers (USA) | |
| Study design: not randomized trial, CBA, ITS; no control group (probably New Zealand) | |
| Study design: "prospective controlled trial"; possibly prospective cohort study; no true control group; not patient reminders (USA) | |
| Participants received financial incentive and opportunity to receive Apple iPad; text message and email reminder data not separated; data collect by survey (USA) | |
| Combination intervention, including SMS texting regarding availability of influenza vaccination; cannot clearly distinguish effect of patient reminder | |
| Study design: not randomized trial, CBA, or true ITS; not enough data points (USA) | |
| Compared text message appointment reminder with traditional appointment reminder on visit attendance and immunization series completion | |
| Study aim: cost analysis (England and Wales) | |
| Study design: randomized trial of 2 interventions; compared telephone and mail reminder with mail only reminders; no true control group (Canada) | |
| Not a study; results of symposium (USA) | |
| 3 intervention groups; no true control group (USA) | |
| Obtained vaccination data from self‐report; Indiana (USA) | |
| Study design: not randomized trial, ITS, or CBA; compared 2 interventions (Canada) | |
| Intervention and control groups received automated telephone reminder for influenza vaccination; intervention group also received text message reminder (USA) | |
| Intervention was text message reminders for influenza vaccination; both groups received telephone reminders; no true control group (USA) | |
| Compared written reminder, basic text reminder, and educational text reminder; no true control group (USA) | |
| Electronic abstract only (USA) | |
| Launched full immunization campaign during study period, including press releases, special immunization clinics at pharmacies and stores, and health education kits mailed to physicians; interventions were reminder letters for influenza vaccination only and pneumococcal and influenza vaccination; Medicare beneficiaries (USA) | |
| Discussion of large number of preventive practices over 20 years, but study details not reported (USA) | |
| Financial incentive; all participants had scheduled visits; invitational letters and educational materials sent to all 1 to 2 weeks before the visit (USA) | |
| Study design: post‐test; mailed cues (USA) | |
| Intervention: patient carried cards; not true patient reminder (USA) | |
| Intervention: patient carried cards; no real control group (USA) | |
| Study design: "non‐equivalent control group design"; pre‐test, post‐test; interventions include organizational changes, such as mail prompt, stocking of vaccine, and others; not able to measure effect of mail prompt (The Netherlands) | |
| Study design: not randomized trial, CBA, or ITS; no control group; not patient reminder or recall intervention study (USA) | |
| Study design: not randomized trial, CBA, or ITS (Spain) | |
| Study design: pre‐test post‐test (USA) | |
| Comparison group included interventions; provided financial incentives as conditional cash transfers ("mMoney") or airtime (Kenya) | |
| Compared 2 postcard reminders; sent postcard reminder with educational message about influenza vaccination safety for persons with asthma to intervention group; comparison postcard did not include educational message; no true control group (USA) | |
| Multiple interventions (USA) | |
| Obtained vaccination data from mailed surveys and telephone follow‐up Veterans Affairs centers (USA) | |
| Combined mailed reminder letters with other interventions; time series lacked sufficient number of measures; no control group (USA) | |
| Reported data on community outreach intervention (USA) | |
| Published abstract; manuscript unpublished (USA) | |
| Intervention: combination of patient reminder within electronic personal health record and provider reminders, with both groups receiving provider reminder; convenience sample of patients who agreed to participate in study (USA) | |
| Combination of population‐based and other interventions; not clear who received various types of interventions; study design not clear (USA) | |
| Outcome: number of immunization visits and number of immunizations (USA) | |
| Obtained outcome data by telephone interview (Canada) | |
| Study design: not randomized trial, CBA, or ITS; no true control group; interventions varied by practice; possibility of patient reminders being included (USA) |
CBA: controlled before and after study
HPV: human papillomavirus
ITS: interrupted time series
MMR: measles, mumps, rubella
Data and analyses
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 55 | 138625 | Risk Ratio (M‐H, Random, 95% CI) | 1.28 [1.23, 1.35] |
| Analysis 1.1  Comparison 1 Patient reminders (summary), Outcome 1 Immunized. | ||||
| 1.1 Childhood immunizations | 23 | 31099 | Risk Ratio (M‐H, Random, 95% CI) | 1.22 [1.15, 1.29] |
| 1.2 Childhood influenza immunizations | 5 | 9265 | Risk Ratio (M‐H, Random, 95% CI) | 1.51 [1.14, 1.99] |
| 1.3 Adult immunizations ‐ other | 4 | 8065 | Risk Ratio (M‐H, Random, 95% CI) | 2.08 [0.91, 4.78] |
| 1.4 Adult influenza immunizations | 15 | 59328 | Risk Ratio (M‐H, Random, 95% CI) | 1.29 [1.17, 1.43] |
| 1.5 Adolescent immunizations | 10 | 30868 | Risk Ratio (M‐H, Random, 95% CI) | 1.29 [1.17, 1.42] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 7 | 9120 | Risk Ratio (M‐H, Random, 95% CI) | 1.75 [1.20, 2.54] |
| Analysis 2.1  Comparison 2 Patient telephone reminder or recall, Outcome 1 Immunized. | ||||
| 1.1 Childhood immunizations | 2 | 234 | Risk Ratio (M‐H, Random, 95% CI) | 2.27 [1.12, 4.63] |
| 1.2 Adult immunizations ‐ other | 2 | 6630 | Risk Ratio (M‐H, Random, 95% CI) | 1.58 [0.55, 4.57] |
| 1.3 Adult influenza immunizations | 2 | 1838 | Risk Ratio (M‐H, Random, 95% CI) | 1.53 [0.73, 3.20] |
| 1.4 Adolescent immunizations | 1 | 418 | Risk Ratio (M‐H, Random, 95% CI) | 2.04 [1.05, 3.95] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 27 | 81100 | Risk Ratio (M‐H, Random, 95% CI) | 1.29 [1.21, 1.38] |
| Analysis 3.1  Comparison 3 Patient letter reminder or recall, Outcome 1 Immunized. | ||||
| 1.1 Childhood immunizations | 9 | 13009 | Risk Ratio (M‐H, Random, 95% CI) | 1.16 [1.06, 1.27] |
| 1.2 Childhood influenza immunizations | 5 | 9265 | Risk Ratio (M‐H, Random, 95% CI) | 1.51 [1.14, 1.99] |
| 1.3 Adult immunizations ‐ other | 2 | 1435 | Risk Ratio (M‐H, Random, 95% CI) | 3.13 [1.44, 6.84] |
| 1.4 Adult influenza immunizations | 11 | 44454 | Risk Ratio (M‐H, Random, 95% CI) | 1.35 [1.19, 1.52] |
| 1.5 Adolescent immunizations | 2 | 12937 | Risk Ratio (M‐H, Random, 95% CI) | 1.91 [0.71, 5.11] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 8 | 27734 | Risk Ratio (M‐H, Random, 95% CI) | 1.18 [1.08, 1.30] |
| Analysis 4.1  Comparison 4 Patient postcard reminder or recall, Outcome 1 Immunized. | ||||
| 1.1 Childhood immunizations | 4 | 2806 | Risk Ratio (M‐H, Random, 95% CI) | 1.24 [1.05, 1.46] |
| 1.2 Adult influenza immunizations | 3 | 19265 | Risk Ratio (M‐H, Random, 95% CI) | 1.15 [0.95, 1.39] |
| 1.3 Adolescent immunizations | 1 | 5663 | Risk Ratio (M‐H, Random, 95% CI) | 1.23 [0.98, 1.54] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 6 | 7772 | Risk Ratio (M‐H, Random, 95% CI) | 1.29 [1.15, 1.44] |
| Analysis 5.1  Comparison 5 Patient text message reminder or recall, Outcome 1 Immunized. | ||||
| 1.1 Childhood immunizations | 1 | 304 | Risk Ratio (M‐H, Random, 95% CI) | 1.22 [1.11, 1.33] |
| 1.2 Adult influenza immunizations | 1 | 204 | Risk Ratio (M‐H, Random, 95% CI) | 1.05 [0.71, 1.58] |
| 1.3 Adolescent immunizations | 4 | 7264 | Risk Ratio (M‐H, Random, 95% CI) | 1.38 [1.16, 1.64] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 5 | 11947 | Risk Ratio (M‐H, Random, 95% CI) | 1.17 [1.03, 1.32] |
| Analysis 6.1  Comparison 6 Patient autodialer message reminder or recall, Outcome 1 Immunized. | ||||
| 1.1 Childhood immunizations | 3 | 8583 | Risk Ratio (M‐H, Random, 95% CI) | 1.27 [1.19, 1.35] |
| 1.2 Adolescent immunizations | 2 | 3364 | Risk Ratio (M‐H, Random, 95% CI) | 1.08 [0.99, 1.17] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 8 | 6506 | Risk Ratio (M‐H, Random, 95% CI) | 1.28 [1.14, 1.45] |
| Analysis 7.1  Comparison 7 Combination patient mail and telephone reminder or recall, Outcome 1 Immunized. | ||||
| 1.1 Childhood immunizations | 7 | 4910 | Risk Ratio (M‐H, Random, 95% CI) | 1.27 [1.09, 1.48] |
| 1.2 Adolescent immunizations | 1 | 1596 | Risk Ratio (M‐H, Random, 95% CI) | 1.36 [1.20, 1.53] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 3 | 2701 | Risk Ratio (M‐H, Random, 95% CI) | 1.22 [1.10, 1.35] |
| Analysis 8.1  Comparison 8 Combination patient reminder or recall with outreach, Outcome 1 Immunized. | ||||
| 1.1 Childhood immunizations | 3 | 2701 | Risk Ratio (M‐H, Random, 95% CI) | 1.22 [1.10, 1.35] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 2 | 4120 | Risk Ratio (M‐H, Random, 95% CI) | 2.91 [2.67, 3.19] |
| Analysis 9.1  Comparison 9 Combination patient reminder or recall with provider reminder, Outcome 1 Immunized. | ||||
| 1.1 Adult immunizations ‐ other | 1 | 264 | Risk Ratio (M‐H, Random, 95% CI) | 4.07 [1.13, 14.70] |
| 1.2 Adult influenza immunizations | 2 | 3856 | Risk Ratio (M‐H, Random, 95% CI) | 2.91 [2.66, 3.18] |
Study flow diagram.
'Risk of bias' summary: review authors' judgements about each risk of bias item for each included study.
Funnel plot of comparison: Patient reminder or recall summary measure versus control
Comparison 1 Patient reminders (summary), Outcome 1 Immunized.
Comparison 2 Patient telephone reminder or recall, Outcome 1 Immunized.
Comparison 3 Patient letter reminder or recall, Outcome 1 Immunized.
Comparison 4 Patient postcard reminder or recall, Outcome 1 Immunized.
Comparison 5 Patient text message reminder or recall, Outcome 1 Immunized.
Comparison 6 Patient autodialer message reminder or recall, Outcome 1 Immunized.
Comparison 7 Combination patient mail and telephone reminder or recall, Outcome 1 Immunized.
Comparison 8 Combination patient reminder or recall with outreach, Outcome 1 Immunized.
Comparison 9 Combination patient reminder or recall with provider reminder, Outcome 1 Immunized.
| Patient reminder or recall interventions compared with no patient reminder or recall for receipt of immunizations | ||||||
| Patient or population: children, adolescents, and adults with a need for routine immunizations, excluding travel immunizations Settings: patient telephone reminder or recall interventions are typically received in the home; the interventions originate from outpatient departments of hospitals, community‐based clinical settings, local and state public health departments, and other clinical settings Intervention: patient reminder or recall interventions Comparison: no‐intervention control groups, standard practice activities that did not include immunization‐focused patient reminder or recall interventions, media‐based activities aimed at promoting immunizations, and simple practice‐based immunization awareness campaigns | ||||||
| Intervention type | Outcome: received immunizations | |||||
| Illustrative comparative risks* (95% CI) | Relative effect | No of participants | Certainty of the evidence | Comments | ||
| Assumed risk | Corresponding risk | |||||
| Without intervention | With intervention | |||||
| Patient reminder or recall summary measure | 290 per 1000 | 371 per 1000 (357 to 392) | RR 1.28a (1.23 to 1.35) | 138,625 | Moderateb | — |
| Patient telephone reminder or recall | 164 per 1000 | 287 per 1000 (197 to 417) | RR 1.75 (1.20 to 2.54) | 9120 (7) | Moderatec | — |
| Patient letter reminder or recall | 320 per 1000 | 412 per 1000 (387 to 442) | RR 1.29 (1.21 to 1.38) | 81,100 (27) | Moderated | — |
| Patient postcard reminder or recall | 327 per 1000 | 386 per 1000 (353 to 425) | RR 1.18 (1.08 to 1.30) | 27,734 (8) | Highe | — |
| Patient text message reminder or recall | 161 per 1000 | 208 per 1000 (185 to 232) | RR 1.29 (1.15 to 1.44) | 7772 (6) | High | — |
| Patient autodialer message reminder or recall | 365 per 1000 | 427 per 1000 (376 to 482) | RR 1.17 (1.03 to 1.32) | 11,947 (5) | High | — |
| Combination of patient mail and telephone reminder or recall | 277 per 1000 | 354 per 1000 (316 to 402) | RR 1.28 (1.14 to 1.45) | 6506 (8) | Moderatef | — |
| Combination of patient reminder or recall with outreach intervention | 360 per 1000 | 439 per 1000 (396 to 486) | RR 1.22 (1.10 to 1.35) | 2701 (3) | High | — |
| Combination of patient reminder or recall with provider reminder intervention | 202 per 1000 | 588 per 1000 (540 to 644) | RR 2.91 (2.67 to 3.19) | 4120 (2) | Moderateg | — |
| *The basis for the assumed risk, e.g. the median control group risk across studies, is provided in footnotes. The corresponding risk, and its 95% confidence interval, is based on the assumed risk in the comparison group and the relative effect of the intervention, and its 95% CI. | ||||||
| GRADE Working Group grades of evidence | ||||||
| aIt is important to note that this review is the third update of the initial review that was published in 2002; the results for each update have been relatively stable and consistent with the original review. bWe downgraded the certainty of the evidence by 1 point. GRADE was reduced by 0.5 points because of a small degree of inconsistency in outcomes. Generally, most included studies reported relatively small positive risk ratios, with several negative outliers and several with stronger positive effects; the patient reminder recall interventions also varied. We downgraded precision slightly (‐0.5) because the confidence intervals were wide for several included studies. cWe downgraded the certainty of the evidence by 1 point. GRADE was reduced by 0.5 points because of a small degree of inconsistency in outcomes; the interventions were relatively homogeneous. We downgraded precision slightly (‐0.5) because the confidence intervals were wide for a few included studies. dWe downgraded the certainty of the evidence by 1 point because of a small degree of inconsistency in outcomes (0.5 point); the interventions were relatively homogeneous. We downgraded precision slightly (‐0.5) because the confidence intervals were wide for several included studies. eWe downgraded the certainty of the evidence by 0.5 points because of a high risk of bias for one or two of eight criteria for 15 studies. fWe downgraded the certainty of the evidence by 1 point. GRADE was reduced by 0.5 points because of a small degree of inconsistency in outcomes, with one outlier; the interventions were more varied than the single intervention types. We downgraded precision slightly (‐0.5) because the confidence interval was wide for one outlier. gWe downgraded the certainty of the evidence by 1.5 points. GRADE was reduced by 0.5 points because of a moderate risk of bias in one of three comparisons within two studies. We downgraded precision by 1 point because of two wide confidence intervals in three comparisons. | ||||||
| Patient reminder or recall intervention for receipt of immunization, by type of immunization | ||||||
| Patient or population: children, adolescents, and adults with a need for routine immunizations, excluding travel immunizations Settings: patient reminder or recall interventions are typically received in the home; the interventions originate from outpatient departments of hospitals, community‐based clinical settings, local and state public health departments, and other clinical settings Interventions: patient reminder or recall interventions, including telephone calls, autodialer calls, letters, postcards, text messages, combination of mail or telephone, or combination of patient reminder or recall with outreach; this summary measure excludes patient reminder or recall interventions combined with provider reminders Comparison: no‐intervention control groups, standard practice activities that did not include immunization‐focused patient reminder or recall interventions, media‐based activities aimed at promoting immunizations, and simple practice‐based immunization awareness campaigns | ||||||
| Outcomes | Illustrative comparative risks* (95% CI) | Relative effect | No of participants | Certainty of the evidence | Comments | |
| Assumed risk | Corresponding risk | |||||
| Without intervention | With intervention | |||||
| Childhood immunizations | 333 per 1000 | 406 per 1000 (383 to 430) | RR 1.22 (1.15 to 1.29) | 31,099 (23) | Higha | — |
| Childhood influenza immunizations | 431 per 1000 | 651 per 1000 (491 to 857) | RR 1.51 (1.14 to 1.99) | 9265 (5) | Moderateb | — |
| Adult immunizations ‐ other than influenza or travel ('Other adult') | 109 per 1000 | 227 per 1000 (99 to 521) | RR 2.08 (0.91 to 4.78) | 8065 (4) | Lowc | — |
| Adult influenza immunizations | 292 per 1000 | 376 per 1000 (342 to 418) | RR 1.29 (1.17 to 1.43) | 59,328 (15) | Moderated | — |
| Adolescent immunizations | 244 per 1000 | 314 per 1000 (285 to 346) | RR 1.29 (1.17 to 1.42) | 30,868 (10) | Highe | — |
| *The basis for the assumed risk, e.g. the median control group risk across studies, is provided in footnotes. The corresponding risk, and its 95% confidence interval, is based on the assumed risk in the comparison group and the relative effect of the intervention, and its 95% CI. | ||||||
| GRADE Working Group grades of evidence | ||||||
| aWe did not downgrade the certainty of the evidence: no serious risk of bias, serious inconsistency, serious indirectness, or serious imprecision was identified among the 23 studies; however, one study was an outlier (RR 5.33). bWe downgraded the certainty of the evidence by 1.5 points because of some imprecision (‐1) and inconsistency (‐0.5). One of five studies had a wide confidence interval and effect sizes ranged from 1.08 to 4.6. cWe downgraded the certainty of the evidence by 2 points because of lack of agreement between studies (‐1) and some imprecision (‐1). Effect sizes ranged from 1.08 to 3.61 and two of five studies had wide confidence intervals. dWe downgraded the certainty of the evidence by 1.5 points because of some inconsistency in results (‐0.5) and some imprecision (‐1). Effect sizes ranged from 0.91 to 3.11 and one of 15 studies had a wide confidence interval. eWe did not downgrade the certainty of the evidence: no serious risk of bias, serious inconsistency, serious indirectness, or serious imprecision was identified among the 10 studies. | ||||||
| Group or subgroup | RR (CI) for full set of included studies | RR (CI) after deleting studies with 'high' risk of bias for random sequence generation, allocation concealment, and/or incomplete outcomes | RR (CI) after deleting studies with primary outcome of received all needed vaccinations |
| Summary measure | 1.28 (1.23 to 1.35) | 1.29 (1.23 to 1.36) | 1.32 (1.25 to 1.39) |
| Child | 1.22 (1.15 to 1.29) | 1.19 (1.12 to 1.27) | 1.24 (1.15 to 1.34) |
| Influenza – child | 1.51 (1.14 to 1.99) | 1.51 (1.14 to 1.99) | 1.37 (1.05 to 1.77) |
| Adult – other | 2.08 (0.91 to 4.78) | 2.35 (2.02 to 2.74) | 2.08 (0.91 to 4.78) |
| Influenza – adult | 1.29 (1.17 to 1.43) | 1.33 (1.20 to 1.48) | 1.29 (1.17 to 1.43) |
| Adolescent | 1.29 (1.17 to 1.42) | 1.26 (1.15 to 1.39) | 1.33 (1.20 to 1.48) |
| CI: confidence interval | |||
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 55 | 138625 | Risk Ratio (M‐H, Random, 95% CI) | 1.28 [1.23, 1.35] |
| 1.1 Childhood immunizations | 23 | 31099 | Risk Ratio (M‐H, Random, 95% CI) | 1.22 [1.15, 1.29] |
| 1.2 Childhood influenza immunizations | 5 | 9265 | Risk Ratio (M‐H, Random, 95% CI) | 1.51 [1.14, 1.99] |
| 1.3 Adult immunizations ‐ other | 4 | 8065 | Risk Ratio (M‐H, Random, 95% CI) | 2.08 [0.91, 4.78] |
| 1.4 Adult influenza immunizations | 15 | 59328 | Risk Ratio (M‐H, Random, 95% CI) | 1.29 [1.17, 1.43] |
| 1.5 Adolescent immunizations | 10 | 30868 | Risk Ratio (M‐H, Random, 95% CI) | 1.29 [1.17, 1.42] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 7 | 9120 | Risk Ratio (M‐H, Random, 95% CI) | 1.75 [1.20, 2.54] |
| 1.1 Childhood immunizations | 2 | 234 | Risk Ratio (M‐H, Random, 95% CI) | 2.27 [1.12, 4.63] |
| 1.2 Adult immunizations ‐ other | 2 | 6630 | Risk Ratio (M‐H, Random, 95% CI) | 1.58 [0.55, 4.57] |
| 1.3 Adult influenza immunizations | 2 | 1838 | Risk Ratio (M‐H, Random, 95% CI) | 1.53 [0.73, 3.20] |
| 1.4 Adolescent immunizations | 1 | 418 | Risk Ratio (M‐H, Random, 95% CI) | 2.04 [1.05, 3.95] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 27 | 81100 | Risk Ratio (M‐H, Random, 95% CI) | 1.29 [1.21, 1.38] |
| 1.1 Childhood immunizations | 9 | 13009 | Risk Ratio (M‐H, Random, 95% CI) | 1.16 [1.06, 1.27] |
| 1.2 Childhood influenza immunizations | 5 | 9265 | Risk Ratio (M‐H, Random, 95% CI) | 1.51 [1.14, 1.99] |
| 1.3 Adult immunizations ‐ other | 2 | 1435 | Risk Ratio (M‐H, Random, 95% CI) | 3.13 [1.44, 6.84] |
| 1.4 Adult influenza immunizations | 11 | 44454 | Risk Ratio (M‐H, Random, 95% CI) | 1.35 [1.19, 1.52] |
| 1.5 Adolescent immunizations | 2 | 12937 | Risk Ratio (M‐H, Random, 95% CI) | 1.91 [0.71, 5.11] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 8 | 27734 | Risk Ratio (M‐H, Random, 95% CI) | 1.18 [1.08, 1.30] |
| 1.1 Childhood immunizations | 4 | 2806 | Risk Ratio (M‐H, Random, 95% CI) | 1.24 [1.05, 1.46] |
| 1.2 Adult influenza immunizations | 3 | 19265 | Risk Ratio (M‐H, Random, 95% CI) | 1.15 [0.95, 1.39] |
| 1.3 Adolescent immunizations | 1 | 5663 | Risk Ratio (M‐H, Random, 95% CI) | 1.23 [0.98, 1.54] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 6 | 7772 | Risk Ratio (M‐H, Random, 95% CI) | 1.29 [1.15, 1.44] |
| 1.1 Childhood immunizations | 1 | 304 | Risk Ratio (M‐H, Random, 95% CI) | 1.22 [1.11, 1.33] |
| 1.2 Adult influenza immunizations | 1 | 204 | Risk Ratio (M‐H, Random, 95% CI) | 1.05 [0.71, 1.58] |
| 1.3 Adolescent immunizations | 4 | 7264 | Risk Ratio (M‐H, Random, 95% CI) | 1.38 [1.16, 1.64] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 5 | 11947 | Risk Ratio (M‐H, Random, 95% CI) | 1.17 [1.03, 1.32] |
| 1.1 Childhood immunizations | 3 | 8583 | Risk Ratio (M‐H, Random, 95% CI) | 1.27 [1.19, 1.35] |
| 1.2 Adolescent immunizations | 2 | 3364 | Risk Ratio (M‐H, Random, 95% CI) | 1.08 [0.99, 1.17] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 8 | 6506 | Risk Ratio (M‐H, Random, 95% CI) | 1.28 [1.14, 1.45] |
| 1.1 Childhood immunizations | 7 | 4910 | Risk Ratio (M‐H, Random, 95% CI) | 1.27 [1.09, 1.48] |
| 1.2 Adolescent immunizations | 1 | 1596 | Risk Ratio (M‐H, Random, 95% CI) | 1.36 [1.20, 1.53] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 3 | 2701 | Risk Ratio (M‐H, Random, 95% CI) | 1.22 [1.10, 1.35] |
| 1.1 Childhood immunizations | 3 | 2701 | Risk Ratio (M‐H, Random, 95% CI) | 1.22 [1.10, 1.35] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Immunized Show forest plot | 2 | 4120 | Risk Ratio (M‐H, Random, 95% CI) | 2.91 [2.67, 3.19] |
| 1.1 Adult immunizations ‐ other | 1 | 264 | Risk Ratio (M‐H, Random, 95% CI) | 4.07 [1.13, 14.70] |
| 1.2 Adult influenza immunizations | 2 | 3856 | Risk Ratio (M‐H, Random, 95% CI) | 2.91 [2.66, 3.18] |
