Evaluating Social Change: What Is Sustain Dane’s Real Impact?
Posted: 10:54AM February 7th, 2014 | Comments
Submitted by Lauren Beriont
We here at Sustain Dane work to promote socio-environmental sustainability. We strive to inspire individuals in the Greater Madison Region to be sustainability champions in their homes, in their neighborhoods, and in their workplaces. We aspire for Madison to be a national - heck international - model for sustainability. When I joined Sustain Dane in October, right away I saw such diligence in the staff and interns working towards this vision. Still, I wondered, “How do we really know that we are making real progress toward this vision?”
This is where my obsession with evaluation kicked in.
In the nonprofit sector, program and organization evaluations are critical measurement tools that allow us to better understand if we are meeting our goals. Are we inspiring individuals to be sustainability champions? Does this lead to widespread and lasting changes in behavior? How do we know this is true? Evaluations provide us the answers to these types of questions with metrics of successes and failures so that we can continue to work more effectively and productively to accomplish our goals.
Traditionally, evaluations have gathered quantitative data: the number of people who attend an event, the number of Twitter followers, the number of dollars fundraised. Recently, however, the nonprofit sector has made an intentional shift towards “impact evaluations.” - a transition from measuring a return on investment to a return on engagement.
At Sustain Dane, we see a return on engagement as a holistic reflection of the broad-based influence and ripple effect of our aggregate programs and interventions. Return on Engagement is a lens that captures the intended and unintended outcomes of our entire organization. Some things we begin to ask and set out to measure:
Are our constituents consistently engaging in the Sustain Dane network?
Are we instilling individuals with sustainability communication skills?
Is our success replicated outside of the Madison region?
This type of evaluation provides us the data and metrics necessary to accurately share our story of empowerment, improve successful programs and tactics, retire ineffective interventions, and justify Sustain Dane’s past and future presence in our the Madison area.
We are beginning to count something that counts to both us and to ourcommunity.
So, how do we measure return on engagement?
At Sustain Dane, we started with a single event - Badger Bioneers 2013 – our annual gathering of local sustainability champions. For this event, we developed an evaluation process that integrates current research on program evaluation and best practices from other nonprofits, and that matches Sustain Dane’s specific needs and goals.
In order to accurately measure many aspects of our impact, our Bioneers evaluation used mixed methods:
• Retroactive Pre-Conference Survey
• Post-Conference Survey
• Facilitated Staff Reflection
• Conference Facilitator Survey
• Follow-up Surveys at Bioneers Film Screenings
• 3 Month Follow-up Focus Group
Our initial data for the 2013 conference are really exciting! Here are some of the key findings:
91.15% of 2013 Badger Bioneers plan to attend 2014 Bioneers
96.5% of 2013 participants agreed that the skills and techniques presented at the conference are relevant and useful for their sustainability work
97.37% of 2013 participants agreed that the skills and techniques presented at the conference are relevant and useful for their sustainability work
100% of Bioneer participants intend to act on their personal commitment to sustainability after Bioneers.
We saw statistically significant increases in all our measured variables; please see schematic below for a visual representation.
We will use this process and findings to learn more about social change evaluation relevant to sustainability in the Madison region. Sustain Dane has committed to making evaluation a formalized part of every program and event we run, and the Badger Bioneers evaluation serves as a pivotal starting point for a broader, longitudinal measurement of sustainability and impact in the Greater Madison Region.
As part of that long-term vision, we asked participants at Badger Bioneers and the Bioneers encore screenings to make a commitment to sustainability by writing an action pledge to be initiated by Earth Day 2014. We collect these pledges on postcards, which we will mail back to them as a reminder, and we will follow up near Earth Day 2014 to measure the number of actions taken.
Can you make a commitment? Pledge to take a sustainability action, join the Sustain Dane community of sustainability champions.