IMPACT STUDY METHODOLOGY
IMPACT STUDY METHODOLOGY
An Impact Study measures the current poverty-fighting effectiveness of an organization across four dimensions. For each dimension, the effectiveness is determined based on the strength of available evidence compared to Slingshotās outcomes-driven methodology. Impact results provide an objective, consistent way to measure impact, track progress, and identify opportunities for greater impact.
DESCRIPTION
INFLUENCE ON POVERTY-FIGHTING
Estimates the benefits created by an organization for the people it serves relative to the cost of creating those benefits
Enhances the allocation of resources by understanding the poverty-fighting benefits created by programs and interventions
Assesses the magnitude of an organizationās poverty-fighting change in the community beyond its core programs
Validates the most impactful types of partnerships and policy changes an organization is positioned to pursue
Determines the extent to which an organization adheres to best practices for its programs and staff performance management
Improves the magnitude of poverty-fighting outcomes achieved by using evidence-based interventions that have the greatest efficacy
Evaluates the effectiveness with which an organization collects, analyzes, and uses data to improve decision making as it relates to fighting poverty
Provides the foundation for an evidence-based continuous improvement process that enables greater poverty-fighting impact
Overview of Benefit-Cost Ratio Dimension:
Benefit-cost ratio estimates the benefits created by an organization for the people it serves relative to the cost of creating those benefits.
Slingshot has developed a proprietary algorithm for monetizing poverty-fighting benefits that can be applied across various types of interventions. This information enhances the allocation of resources by understanding the poverty-fighting benefits created by programs and interventions.
Purposes of a program map:
- Outlines the range of outputs for the program
- Informs the appropriate baseline population based on the primary sources of program participants
- Identifies the various interventions participants receive through PFO programming
Purposes of an impact tree:
- Identifies the potential poverty-fighting benefits created by the program interventions
- Slingshot defines poverty-fighting benefits as improvements in future earnings or health
- Informs the data and research required to monetize the program benefits (outcomes)
Purposes:
- Collect data and perform research to validate which benefits can be monetized
- Monetize benefits for participants and their immediate families; societal benefits not included
- Aggregate the value of all the monetized benefits and divide it by the organizationās total costs
Overview of Systems-Level Change dimension:
The systems-level change dimension assesses the magnitude of an organizationās poverty-fighting change in the community beyond its core programs. Slingshot considers changes in two broad categories: pathways and policy.
Slingshot evaluates the evidence associated with the breadth and intensity of the changes and an organizationās role in creating each change. This information validates the most impactful types of partnerships and policy changes an organization can pursue.
- Occur when an organization directly influences one or more organizations, often via partnerships or collaboration, in ways that increase poverty-fighting impact
- Alter the local poverty-fighting ecosystem in a way that primarily benefits the participants of the involved organizations
- Examples include:
-
- Identifying and avoiding duplicate efforts with other nonprofits
- Building partnerships that create access to complimentary services for participants
- Establishing a standardized referral process from another nonprofit
-
Ā
- Occur when an organization increases poverty-fighting impact by directly influencing evidence-based changes in economic, social, institutional, or political systems
- Alter the broader local poverty-fighting ecosystem in a way that benefits a large proportion of the applicable population
- Examples include:
- Changing a policy to remove a systemic barrier for people experiencing poverty
- Attracting new, systemic funding streams for poverty-fighting interventions
- Improving a regulation in ways that result in greater poverty alleviation
Ā
Overview of Use of Best Practices Dimension:
The best practices dimension determines the extent to which an organization adheres to best practices for its programs and seeks to improve its performance.
Slingshot conducts literature reviews to understand the most effective evidence-based practices and assesses the existence of mechanisms to support effective program implementation and staff performance management. This information helps improve the magnitude of poverty-fighting outcomes achieved by using evidence-based interventions with the greatest efficacy.
- Identify the models used to deliver programming
- Research the models that evidence supports as the most effective
- Validate the efficacy of the program models employed versus research verified best practices
- Identify the processes used to help ensure program models are implemented as designed
- Assess the consistency in which participants would experience a program model as designed
- Identify the existence of important practices that support staff performance in effectively administering programs
- Validate the efficacy of these practices versus research-verified staff performance management best practices
Overview of Measurement Infrastructure Dimension:
Measurement infrastructure evaluates the effectiveness with which an organization collects, analyzes, and uses data to improve decision making as it relates to fighting poverty.
Slingshot has developed a framework that allows each component of the measurement infrastructure to be assessed: measurement objectives, metrics collected, quality assurance, storage systems, data analysis, and decision making. This information helps enhance the foundation for an evidence-based continuous improvement process that enables greater poverty-fighting impact.
Existence of standard processes to use
data and analyses in decision making
Purposes for collecting, analyzing,
and using data in decision-making
Specific data collected about program
participants, outputs and outcomes
Processes to ensure data accuracy,
completeness, consistency, and currency
Quality of storage systems and their ability
to aggregate data for analyses
Effectiveness of analyses and tools
to measure impact and provide insights
