Implementation Science at UW
Research Methods in Implementation Science
A broad and inclusive definition of the field defines implementation science as:
A systematic, scientific approach to ask and answer questions about how we get “what works” to people who need it, for as long as they need it, with greater speed, fidelity, efficiency, quality and relevant coverage.
This broad landscape of inquiry allows for the application of at least fourteen research methods from a wide range of disciplines. These methods are used in order to understand and improve the determinants, processes, and outcomes of implementation and, ultimately, scale-up and sustainability to achieve population-level health benefits.
While this selection of research methods is not exhaustive, together they provide a set of tools that are used to assess and improve implementation and scale-up of health and human-centered interventions. Scroll down for additional resources for each method.
These methods allow exploration of the continuum consisting of three categories of outcomes:
- proximal implementation outcomes – acceptability, appropriateness, adoption, costs, feasibility, fidelity, penetration, and sustainability
- intermediate service delivery outcomes – efficiency, equity, timeliness, and patient-centeredness
- distal health outcomes
Open Access articles will be marked with ✪
Please note some journals will require subscriptions to access a linked article.
Systems Analysis and Improvement Approach
Operations Research
Qualitative Health Systems Research
Definition: The Systems Analysis and Improvement Approach (SAIA) is an evidence-based implementation strategy that combines system engineering tools into a five-step, facility-level package to give clinic staff and managers a system-wide view of their cascade performance, identify priority areas for improvement, discern modifiable opportunities for improvement, and test workflow modifications. The process is iterative, which means health care teams can continue to use the package to further improve care and respond to new bottlenecks that arise. Visit www.saia-strategy.com to learn more about this method and about projects using SAIA currently.
Methodology:
- ✪ 1Systems analysis and improvement to optimize pMTCT (SAIA): A cluster randomized trial (Implementation Science, 2014)
- Evaluation of a Systems Analysis and Improvement Approach to Optimize Prevention of Mother-To-Child Transmission of HIV Using the Consolidated Framework for Implementation Research (Journal of Acquired Immune Deficiency Syndromes, 2016)
- Cascade Analysis: An Adaptable Implementation Strategy Across HIV and Non-HIV Delivery Platforms (Journal of Acquired Immune Deficiency Syndromes, 2019)
- Optimizing treatment cascades for task-shared mental healthcare in Mozambique: Preliminary effectiveness of the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH) (Health Policy & Planning, 2020)
Examples of use:
- ✪ The prevention of mother-to-child transmission of HIV cascade analysis tool: supporting health managers to improve facility-level service delivery (BMC Research Notes, 2014)
- Implementation and Operational Research: Impact of a Systems Engineering Intervention on PMTCT Service Delivery in Côte d'Ivoire, Kenya, Mozambique (Journal of Acquired Immune Deficiency Syndromes, 2016)
- ✪ Scaling-up the Systems Analysis and Improvement Approach for prevention of mother-to-child HIV transmission in Mozambique (SAIA-SCALE): a stepped-wedge cluster randomized trial (Implementation Science, 2019)
Definition: The use of qualitative or quantitative models to facilitate decision-making in complex implementation, particularly relating to structure, prospective evaluation, and reconfiguration.
Key Questions:
- What are the implementation problems exhibited by a particular project?
- What are the innovative solutions to deal with implementation problems?
- What policies or service delivery models can improve effectiveness or efficiency?
- What is the optimal allocation of resources for the program?
Methodology:
- ✪ Operational research as implementation science: definitions, challenges and research priorities (Implementation Science, 2016)
- Experiences in implementation and publication of operations research interventions: gaps and a way forward (Journal of the International AIDS Society, 2016)
Learn More:
- Advancing the strategic use of HIV operations research to strengthen local policies and programmes: The Research to Prevention Project (Journal of the International AIDS Society, 2015)
- ✪ Shifting management of a community volunteer system for improved child health outcomes: Results from an operations research study in Burundi (BMC Health Services Research, 2015)
- ✪ Efficiency of the implementation of cardiovascular risk management in primary care practices: an observational study (Implementation Science, 2016)
Definition: Research that uses methods of observation and interviewing to evaluate health systems through the lens of those who experience them, and to explain factors that shape outcomes, dimensions of care, as well as the social and political determinants of health. For an excellent overview of qualitative methods in implementation research, see Qualitative methods in implementation research: An introduction (Psychiatry Research, 2019).
Methodology:
- ✪ Qualitative Data Analysis for Health Services Research: Developing Taxonomy, Themes, and Theory (Health Services Research, 2007)
- ✪ Qualitative Methods in Implementation Science (The National Cancer Institute, 2018)
Examples of use:
- Anchoring contextual analysis in health policy and systems research: A narrative review of contextual factors influencing health committees in low and middle income countries (Social Science & Medicine, 2015)
- ✪ Identification of gaps for implementation science in the HIV prevention, care and treatment cascade: A qualitative study in 19 districts in Uganda (BMC Research Notes, 2016)
- ✪ Health policy and systems research and analysis in Nigeria: Examining health policymakers’ and researchers’ capacity assets, needs and perspectives in south-east Nigeria (Health Research Policy and Systems, 2016)
- ✪ Synergies, strengths and challenges: findings on community capability from a systematic health systems research literature review (BMC Health Services Research, 2016)
- ✪ Unraveling PBF effects beyond impact evaluation: results from a qualitative study in Cameroon (BMJ Global Health, 2017)
Rapid Evidence Synthesis
Rapid Ethnographic Assessment
Causal Pathway Diagrams
Prioritizing Barriers
Definition: Used to summarize and synthesize research literature and can be applied to synthesizing evidence on known determinants for implementing evidence-based interventions. To learn more about rapid evidence synthesis in implementation science, visit www.ImpSciMethods.org.
Methodology:
- ✪ Rapid Evidence Synthesis: A toolkit for finding and summarizing evidence on short timelines
- ✪ Rapid evidence synthesis to enable innovation and adoption in health and social care (Systematic Reviews, 2022)
Examples of use:
- ✪ Study protocol: Novel Methods for Implementing Measurement-Based Care with youth in Low-Resource Environments (NIMBLE) (Implementation Science Communications, 2023)
- ✪ Implementation of a Rapid Evidence Assessment Infrastructure during the Coronavirus Disease 2019 (COVID-19) Pandemic to Develop Policies, Clinical Pathways, Stimulate Academic Research, and Create Educational Opportunities (The Journal of Pediatrics, 2020)
Definition: A method used to efficiently gather ethnographic data about determinants by seeking to understand the people, tasks, and environments involved from stakeholder perspectives. To learn more about rapid ethnographic assessment in implementation science, visit www.ImpSciMethods.org.
Methodology:
- ✪ Rapid Ethnographic Assessment: A toolkit for understanding implementation context on short timelines
Examples of Use:
- ✪ Designing an implementation strategy to increase health-related social needs screening: Applying the PRISM framework in a resource-limited clinical setting (Translational Behavioral Medicine, 2023)
- ✪ A Rapid Ethnographic Assessment of the Impact of the COVID-19 Pandemic on Mental Health Services Delivery in an Acute Care Medical Emergency Department and Trauma Center (Administration and Policy in Mental Health and Mental Health Services Research , 2021)
Definition: A graphical tool that enables visualization of how implementation strategies bring about implementation outcomes and the conditions under which they work. To learn more about causal pathway diagrams in implementation science, visit www.ImpSciMethods.org.
Methodology:
Definition: Enables teams to determine which implementation barriers are most important to address to support evidence-based intervention implementation. To learn more about prioritizing barriers in implementation science, visit www.ImpSciMethods.org.
Methodology:
Organizational Assessment
Economic Evaluation
Stakeholder & Policy Analysis
Definition: Assessments that measure an organization's capacity, performance, culture, climate, readiness for implementation or change, external context, and/or social networks.
Methodology:
- ✪ The role of organizational research in implementing evidence-based practice: QUERI Series (Implementation Science, 2008)
- ✪ Organizational- and system-level characteristics that influence implementation of shared decision-making and strategies to address them — a scoping review (Implementation Science, 2018)
Examples of use:
- ✪ The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study (Implementation Science, 2012)
- ✪ A complementary marriage of perspectives: understanding organizational social context using mixed methods (Implementation Science, 2014)
- ✪ Towards evidence-based palliative care in nursing homes in Sweden: a qualitative study informed by the organizational readiness to change theory (Implementation Science, 2018)
- ✪ Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship (Implementation Science, 2018)
Definition: Monetary-focused assessments evaluating system efficiency and allocation of resources to implementations, interventions, or services.
Methodology:
- Methodological reviews of economic evaluations in health care: what do they target? (The European Journal of Health Economics, 2014)
- ✪ Economic evaluation of implementation strategies in health care (Implementation Science, 2014)
- Economic evaluation in implementation science: Making the business case for implementation strategies (Psychiatry Research, 2019)
- ✪ Use of health economic evaluation in the implementation and improvement science fields—a systematic literature review (Implementation Science, 2019)
- Including Economic Evaluations in Implementation Science (Journal of General Internal Medicine , 2020)
Examples of use:
- ✪ Economic Evaluation of Active Implementation versus Guideline Dissemination for Evidence-Based Care of Acute Low-Back Pain in a General Practice Setting (PLOS One, 2013)
- ✪ Economic evaluation of Community Level Interventions for Pre-eclampsia (CLIP) in South Asian and African countries: a study protocol (Implementation Science, 2015)
- ✪ A Cost-Effectiveness Tool to Guide the Prioritization of Interventions for Rheumatic Fever and Rheumatic Heart Disease Control in African Nations (PLOS Neglected Tropical Diseases, 2016)
- ✪ Intervention Now to Eliminate Repeat Unintended Pregnancy in Teenagers (INTERUPT): A systematic review of intervention effectiveness and cost-effectiveness, and qualitative and realist synthesis of implementation factors and user engagement (BMC Medicine, 2017)
- ✪ Economic Evaluations of Stepped Models of Care for Depression and Anxiety and Associated Implementation Strategies: A Review of Empiric Studies (International Journal of Integrated Care, 2019)
Webinar Recordings:
Definition: The intentional integration of stakeholder perspectives and feedback in the analysis of policy advisability, execution and impact.
Methodology
- A Framework for Enhancing the Value of Research for Dissemination and Implementation (American Journal of Public Health, 2014)
- ✪ Addressing Health Equity Through Action on the Social Determinants of Health: A Global Review of Policy Outcome Evaluation Methods (International Journal of Health Policy and Management, 2018)
- ✪ How to engage stakeholders in research: design principles to support improvement (Health Research Policy and Systems, 2018)
- ✪ Engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process: a scoping review (Implementation Science, 2018)
Examples of use:
- ✪ The micropolitics of implementation; a qualitative study exploring the impact of power, authority, and influence when implementing change in healthcare teams (BMC Health Services Research, 2020)
- ✪ Evidence-informed policy formulation and implementation: A comparative case study of two national policies for improving health and social care in Sweden (Implementation Science, 2015)
- ✪ HIV policy implementation in two health and demographic surveillance sites in Uganda: Findings from a national policy review, health facility surveys and key informant interviews (Implementation Science, 2017)
- Implementation science in global health settings: Collaborating with governmental & community partners in Uganda (Psychiatry Research, 2019)
Webinar Recordings:
- 💻 Policy Implementation Science: A Critical Component of Evidence Translation (2020)
- 💻 Improving the Use of Scientific Evidence in Policy Formulation (2020)
- 💻 Implementation Science: Context & Equity in Cancer Research (2020)
Podcasts:
- 🎧 Policymaking Is Not a Science (Yet) [Ep. 405] (Freakonomics, 2020)
Surveillance & Data Systems
Social Marketing
Dissemination Research
Impact Evaluation
Definition: Routine surveillance data from control and experimental groups can be used illustrate the performance or impact of new policies and programs in an environment.
Methodology:
- ✪ Public Health Surveillance Systems: Recent Advances in Their Use and Evaluation (Annual Review of Public Health, 2016)
Examples of use:
- ✪ Data-driven quality improvement in low-and middle-income country health systems: Lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia (BMC Health Services Research, 2017)
- ✪ Using routine health information systems for well-designed health evaluations in low- and middle-income countries (Health Policy and Planning, 2016)
- Implementation Science and the Pregnancy Risk Assessment Monitoring System (Journal of Women's Health, 2014)
- ✪ Surveillance Systems to Track and Evaluate Obesity Prevention Efforts (Annual Review of Public Health, 2017)
- ✪ Harnessing the Power of Data to Guide Local Action and End Tuberculosis (The Journal of Infectious Diseases, 2017)
- ✪ HIV Clustering in Mississippi: Spatial Epidemiological Study to Inform Implementation Science in the Deep South (JMIR Public Health and Surveillance, 2018)
Definition: The application of marketing principals to policy selection, as well as to implementation or intervention planning and operational delivery, done in a reflexive and critical manner.
Methodology & Reviews:
- ✪ Social Marketing in Public Health (Annual Review of Public Health, 2005)
- ✪ A Framework for Disseminating Evidence-Based Health Promotion Practices (Preventing Chronic Disease, 2012)
- ✪ The effectiveness of social marketing in global health: A systematic review (Health Policy and Planning, 2017)
- ✪ Social marketing interventions for the prevention and control of neglected tropical diseases: A systematic review (PLoS Neglected Tropical Diseases, 2020)
- ✪ Health marketing and behavioral change: A review of the literature (Journal of Medicine and Life, 2018)
- ✪ Role of social marketing in promoting primary care to succeed in current era (Journal of Family Medicine and Primary Care, 2019)
Examples of use:
- Direct‐to‐Consumer Marketing: A Complementary Approach to Traditional Dissemination and Implementation Efforts for Mental Health and Substance Abuse Interventions (Clinical Psychology Science and Practice, 2015)
- Selling Innovations Like Soap: The Interactive Systems Framework and Social Marketing (American Journal of Community Psychology, 2017)
- Using system dynamics modeling to evaluate a community-based social marketing framework (Journal of Social Marketing, 2019)
- ✪ Effectiveness of social marketing in improving knowledge, attitudes and practice of consumption of vitamin A-fortified oil in Tanzania (Public Health Nutrition, 2019)
Definition: The targeted distribution of information and intervention materials to a specific public health or clinical practice audience.
Methodology:
- ✪ Disseminating research findings: what should researchers do? A systematic scoping review of conceptual frameworks (Implementation Science, 2010)
- The Science, and Art, of Program Dissemination: Strategies, Successes, and Challenges (New Directions for Child and Adolescent Development, 2015)
- ✪ Strategies for disseminating recommendations or guidelines to patients: a systematic review (Implementation Science, 2016)
- WEBINAR: Designing for Dissemination: How might we increase the reach and impact of our discoveries? (Recording of ACCORDS Grand Rounds, presented by Ross Brownson, June 2018)
Examples of use:
- ✪ Developing a national dissemination plan for collaborative care for depression: QUERI Series (Implementation Science, 2008)
- ✪ Does dissemination extend beyond publication: a survey of a cross section of public funded research in the UK (Implementation Science, 2010)
- ✪ Dissemination of evidence-based cancer control interventions among Catholic faith-based organizations: results from the CRUZA randomized trial (Implementation Science, 2016)
Definition: An evaluation of how the intervention or implementation affects relevant outcomes, intended or otherwise, and typically includes evidence of how outcomes would or would not differ in the absence of the intervention or implementation.
Key Questions:
- What would have happened had the intervention not taken place?
- What was the impact of the intervention on beneficiaries?
- How does the outcome among beneficiaries compare to the outcome among individuals who were not involved in the program?
Examples of Use:
- ✪ Impact of pharmacy worker training and deployment on access to essential medicines and health outcomes in Malawi: protocol for a cluster quasi-experimental evaluation (Implementation Science, 2014)
- Interventions to Drive Uptake of Voluntary Medical Male Circumcision—A Collection of Impact Evaluation Evidence (Journal of Acquired Immune Deficiency Syndromes, 2016)
- The European Union’s Ambient and Assisted Living Joint Programme: An evaluation of its impact on population health and well-being (Health Informatics Journal, 2016)
- ✪ Evaluation of the Impact of a Mass Media Campaign on Periodontal Knowledge among Iranian Adults: A Three-Month Follow-Up (PLOS One, 2017)
- Counting what really counts? Assessing the political impact of science (The Journal of Technology Transfer, 2017)