Benchmarking Methods for Social Impact Bonds

As this week covers Capitalization and Impact Assessment, I find that the concept of assessment is the most ambiguous aspect of the topic. While numerous readings explained that Social Impact Bonds are only paid for by the government if the program succeeds, I felt they lacked attention to identifying what defines “success”.

The concept of establishing measures of success was recently discussed in my Creating Results Oriented Programming class in Heinz. Taking the ideas from the 7th edition of “Evaluation: A Systematic Approach” by Peter H. Rossi, Mark W. Lipset, and Howard E. Freeman, their method of developing measures of success involves identifying the most important inputs, outputs, and outcomes of the program, determining a metric upon which to measure them by, and finally identifying the strengths and weaknesses of this measurement. Realizing the strengths and weaknesses plays a key role. For example, some measurements are quantitative and therefore easy to collect but don’t necessarily provide deep insight, while others are qualitative and hard to collect data for but will provide extremely meaningful insight. Overall, this method allows for easy identification of outcome level and outcome change, which is the difference between outcome levels at different points in time. The net program effect can then by determined by finding the portion of an outcome change that can be attributed uniquely to a program as opposed to the influence of some other factor. This method is effective in identifying the success of the most useful aspects of a program, which seems like the most logical way for SIBs to be evaluated.

Diving further into the idea of assigning quantitative measurements to qualitative areas reminded me of episode 24 of the Exponential Wisdom podcast hosted by Dan Sullivan and Peter Diamandis. In this episode, titled “Scorecards (Without A Target, You’ll Miss It Every Time)”, they discuss the concept of using scorecards as a benchmarking tool to objectively evaluate progress. Creating criteria upon which to measure outcomes provides structure for evaluation, because “if you can’t measure something, you can’t experiment and improve it.” They suggest taking areas that are essentially qualitative in that they are experienced emotionally and practically and quantifying them with numbers so that personal and organizational performance and improvement is possible, mentioning specifically that this method is particularly useful for evaluating startups and investments because you can compare and see which are successful. Additionally, if a program has high scores in all but one category, it shows exactly where their focus should be for improvement. I think this would be extremely useful for SIBs. Creating a scorecard for a specific program would identify if the program is reaching its full potential and worthy of government funding. Alternatively, a comprehensive scorecard would allow social programs to be compared to other social programs, thus accomplishing the same goal by identifying which ones are performing most effectively.

Would developing scorecards for new social programs improve the consistency and accuracy of impact assessment, and what kind of qualitative measurements are most important to identify?

Exponential Wisdom Podcast: http://podcast.diamandis.com/page/2/