Outcome Measurement Applied Locally

This year I started a work study position at a local nonprofit organization that wished to remain unnamed. I found this week’s readings particularly applicable as I was hired to assist with measuring the outcomes for its programs. The organization reflects the narrow scale and wide scope of Harlem’s Children Zone, the most cited example of an organization that vertically controls outcomes in the readings. In this post, I will inspect the various aspects of my organization’s operational mission and outputs to address whether it is reasonable to conclude any outcomes from the program.

The local nonprofit defines its organizational mission as serving the community by addressing the whole needs of the person. To serve this mission, the organization widened its scope to include programs that encompass food security, education, workforce preparation, and housing. Its operational mission incorporates these programs in its scope, seeking to provide comprehensive support to create self-sufficiency for all its constituents.

In scale, this organization is highly local. Its reach spans a third of Pittsburgh with no intentions of growing. It partners with a few organizations to provide for potential applicants that they cannot take in, but often receives pushback for their radical, hotel-like approach to providing for constituents. It is clear from my short time there that they actively engage with whatever community partners they can without compromising their commitment to their mission.

To measure the efficacy of their operational mission, the organization uses a widely acknowledged “self-sufficiency matrix”[1]. To focus their measurements, the organization mainly focuses on the following categories: Housing, Employment, Income, Food, Life Skills, Mental Health, and Substance Abuse. For constituents that moved from less secure stages to more secure stages in the ordinal scale, the results are considered positive. For those that moved down the security scale, they had a negative result. Those who made no changes in status have neutral (0) results. The recorded outputs for these measurements are positive for most constituents and neutral for the rest with one exception for substance abuse. These result show that their outputs support their operation mission, which is indicative of the success of its organizational mission.

The question remains: do these outputs lead to outcomes for the constituents? The organization certainly tries to care for as much of the self-sufficiency matrix as possible with its limited resources. The self-sufficiency matrix is intended to correlate with longer-term outcomes for the organization’s clients, with information confirming that the matrix has provided useful predictive modeling for the efficacy of a program[2]. To some degree, the lack of comprehensive adherence to the self-sufficiency matrix does make it hard to confirm enough vertical integration to point to definitive outcomes, but the core categories do get the most bang for the buck. The organization’s collected data has not yet been analyzed to identify the most impactful components of its program, but by the end of the year I hope to do just that.  

---

[1] “Self-Sufficiency Matrix”, Minnesota Housing Finance Agency, Accessed September 25^th, 2017, http://www.mnhousing.gov/get/MHFA_010996

[2] Culhane, Dennis P.; Parker, Wayne D.; Poppe, Barbara; Gross, Kennen S., Sykes, Ezra. “Accountability, Cost-Effectiveness, and Program Performance: Progress Since 1998”, National Symposium on Homelessness Research, 2007, accessed September 25^th, 2017, .https://aspe.hhs.gov/report/toward-understanding-homelessness-2007-national-symposium-homelessness-research-accountability-cost-effectiveness-and-program-performance-progress-1998/case-study-arizona-evaluation-project

Two potential obstacles of SIBs

“A SIB is a new approach for expanding successful social programs. It is a partnership in which private investors —not governments—provide capital for nonprofits to scale up”[1]. Intermediaries make contracts with governments, raise capital from investors, help direct service providers, select independent assessors to evaluate impacts of social programs. If impacts achieved predefined goals, investors will get repaid with capital plus a return, which comes from government; If the programs fail, investors bear the loss. This pattern benefits constituents through preventive solutions, makes it easier for social programs to raise funds, lowers governments’ financial risk, provides investors with a way both to do well and do good. With a lot of advantages, however, there are still two difficulties should be addressed.

Firstly, evaluation of social programs is important because it determines returns of investors, but it can be hard when quantitative measurement is not enough or collecting data itself is costly. For example, there is an NGO in Hongkong focusing on enriching life of immigrant labor by giving art lecture. The number of lecture is far from enough for evaluating the impact because what really matter are whether the lecture fits the interest of immigrant labors, to what degree they feel enriched from the lecture and what if the organization spend the money to held a party for them. To know how the targeted constituents feels, a survey should be conducted which can cost a lot of money and human resources.

So, to make evaluation more doable, at this early stage, it is better to conduct SIBs in areas where outcomes can be clearly defined and historical data are available. In addition, SIBs are more appropriate to be carried out where preventive interventions exist that cost less to administer than remedial services; some interventions with high levels of evidence already exist; political will for traditional direct funding can be difficult to sustain[2].

Secondly, nonprofit direct service providers in less developed countries are not mature and capable enough to implement social programs well, which increases risk for investors. Meanwhile, community of impact investors in these areas is not as large as that in developed countries, so it will be hard for SIB to attract investors. Therefore, SIB in these areas can include some governmental service providers to ensure the quality of service so that investors will come. One example is the poverty relief program in Shandong, China, in which a governmental department takes responsibility of social programs such as constructing power stations to creates jobs and to improve infrastructure. Five commercial banks have invested about a hundred million dollars in total. This program is estimated to directly benefit 22,000 people across 125 poor counties.

Breaking boundaries of nonprofit, public and private sectors, SIBs make good use of resources to provide preventive solution to social problems. However, with more parties included than the original approach (where governments give money directly to nonprofit service providers), how can we ensure the efficiency of work and avoid unnecessary cost?

---

[1] From Potential to Action: Bringing Social Impact Bonds to the US (Callanan, et.

al., May 2012)

[2] Fact Sheet: Social Impact Bonds in the United States (Center for American Progress, Feb 2014); https://www.americanprogress.org/issues/economy/reports/2014/02/12/84003/fact-sheet-social-impact-bonds-in-the-united-states/

Integrating Impact Metrics as a Social Entrepreneur

A common theme of this week’s readings involved the challenges and limitations of impact metrics for social enterprises.  While many of the articles focused on measuring social investment from the perspective of funding organizations, like foundations and governments, I believe it is also necessary to consider impact measurement from the view of the social entrepreneur.  What metrics are most valuable to the social entrepreneur?  What measurements might influence social venture success? How does a social entrepreneur implement metrics?

From the perspective of a social entrepreneur, a strong understanding of metrics could be a significant success factor.  Ebrahim and Rangan made a case for operating organizations to focus on measuring inputs, activates and outputs, while using scope and scale to determine if outcomes are closely aligned.[1]  However, I feel that the What Impact? article did not address some important aspects of metrics for social ventures.

First, social entrepreneurs should be proficient at assessing and exploring opportunities that have potential to produce repeatable and scalable business models.  This means analyzing market viability and consumer needs.  Being able to effectively measure consumer behavior and identify suitable markets and segments could profoundly enhance a social venture’s appeal to fund providers and improve its chances of a successful launch.

Next, while I agree with the idea that social venture management teams should start with measuring inputs, activities and outputs, it is important to ensure that these measurements are meaningful to the mission and key stakeholders.  Collecting and analyzing data without a distinct purpose could be a waste of time, at the least.  Coordination of the venture’s mission with community groups, funders and governments, for example, could help to shape and define the most effective metrics for a specific activity or output, in order to inform an outcome.

Finally, social entrepreneurs should be engaged in projecting outcomes and impacts that might be affected through their activities and outputs.  While detailed evaluations of outcomes and impacts may be too complex for many social enterprises, an informed projection would help to align activities with other stakeholders.  Balancing and leveraging partnerships across non-profit, for-profit and government organizations could enhance a social venture’s ability to support the overall achievement and measurement of outcomes and impacts.

Devin Thorpe’s article at Forbes.com highlights early social venture measurement advice from an array of successful social entrepreneurs. Much of the advice from these experienced social enterprise leaders supports my points above.[2]  To effectively measure impact, social entrepreneurs should: apply metrics from the start, align measurements with the mission and coordinate with key partners.

 

[1] Ebrahim, Alnoor, and V. Kasturi Rangan. "What Impact?." California Management Review 56, no. 3 (2014): 118-141.

[2] Thorpe, Devin. "How Social Entrepreneurs Begin To Measure Impact." Forbes. March 23, 2017. Accessed September 24, 2017. https://www.forbes.com/sites/devinthorpe/2017/02/26/how-social-entrepreneurs-begin-to-measure-impact/3/#6ed0f866410e.

Scaling Growth is a Dynamic Process

This week’s readings covered the efficiency and social impact challenges that social ventures face as they evaluate and implement growth strategies. In addition to considering their short-term and long-term capacity for scaling, a venture’s growth strategy can also be influenced and sometimes strained by the metrics and relationships defined by partners and trends among competitors in the market.

The ubiquitous and traditional idea that rapid business growth equates to success[1] is outdated and not applicable to most social ventures. Oftentimes the success of a social venture must consider the long-term implications of their growth on their mission.  

Depending on the kind of impact that is sought, slow and conscientious development is best. I witnessed these challenges and the like at my most previous position in a human services non-profit that served homeless families in Norfolk, VA. Our goals for growth were often countered by new funding restrictions and guidelines put into place by our donors and even the federal government. For example, the federal government slowly decreased their funding of homeless shelters nationwide and this impacted our growth model.

While we understood how the new funding restrictions were modeled after an approach geared to serve more families, our agency saw how this pressure to serve more families would lead to a decrease in service quality. In the short term, we saw how this approach lead to less stability for the families we served. To counter this pressure to change our program model, we evolved our fundraising team and strategy to fill the gap of less funding. This was a case where our staffing and fundraising partnerships became a growth priority.

In the case of larger benefit corporations, like Patagonia[2], assessment platforms like B Impact Assessment proved to be a useful and effective way for the company to measure their performance and their impact comparatively in the market.

---

[1] https://hbr.org/2013/01/its-not-all-about-growth-for-s

[2] http://bimpactassessment.net/case-studies/elissa-loughman

“For-Profit” Cleanup of India: The Threshold of Social Campaigns and Profit

“For-Profit” Cleanup of India: The Threshold of Social Campaigns and Profit

This week’s theme of venture development and growth provoked me to think about a ‘for-profit’ cleanup of India. This piece is not an attempt to explicitly prototype a project. Instead, I aim to bring insight to the issue of trash disposal in India and make suggestions relevant to the theme of having successful social impacts and formulating growth in social ventures.

I spent my last summer, vacationing in India, unable to refrain from carrying public policy in my mind for the duration of my trip. During the course of my travels, I encountered an artistic project or innovation of some sort, vouching to clean up India. A combination of bright paintings and slogans portraying good moral standards were painted on large walls across the country. This, however, did not disrupt the unsanitary behavior of the country. The status quo of India in plain language: people are living in a garbage dump.

In my journey, I traveled to northern and southern parts of India for over two months and witnessed first-hand, rural parts of India and over-populated cities being central to this issue. As seen in the Washington Post article, India begins ambitious campaign to clean up dirty cities and villages; Prime Minister Modi attempts to tackle precisely these communities. The affluent middle class recognizes this issue and this is seen in the behavior and regional settings of individuals from the upper middle class (and of course the elite). 

Prior to reading this week’s articles, I assumed that the main concern of social organizations is to have an impact, not money. The articles, How Misinformed Ideas about Profit are Holding Back the World’s Poor and Profits at the Bottom of the Pyramid explain otherwise. Important aspects to success are changing consumer behavior and the way products are delivered. The current cleanup campaign ought to consider their product (art) and outcome (the behavior of citizens). The flowery art pieces across the country attempt to promote better sanitation standards yet the population has not budged.

Villagers, residents, and owners of rental properties should be held responsible for the cleanliness of the area in which they reside in or hold ownership to the property of. If this requirement is not met, they should be fined a penalty issued by the government. Having an on-spot fine for littering and overall a no tolerance approach to inappropriate garbage disposal will pertain to a disruption of behavior.

Opponents may argue that the government is profiting at the expense of poor villagers and struggling lower-middle class citizens. The larger impact of cleaning the country, however, spreads results in less pollution, better public health, efficiency, and an economically attractive market in the long term. Should we rely on citizen’s reaction to art in changing behavior against garbage disposal or should the government take a rather profitable measure? 

The Power of The People

My submission speaks to the power that consumers have had throughout American History in altering society and redefining social equilibriums. This relates to this week’s theme because it provides historical evidence of economic actors transforming a system to bring about sustainability.

This week’s readings on venture development and growth touch on effective strategies that social entrepreneurs use to achieve financial sustainability. As outlined in Two Keys to Sustainable Social Enterprise, the key insight that struck a chord with me was the idea that changing the economic actors involved in an existing system can create sustainable financial models that permanently shift the social and economic equilibrium for their targeted beneficiaries. The article also notes that social and economic problems often reflect an imbalance of power among the economic actors involved. Additionally, the article mentions that adding new actors can transform a system. Taken a step further, the removal of actors can have this same effect.

While reading this article, I immediately thought of the critical role that economic boycotts have had throughout America’s history. From the Boston Tea Party, to the Montgomery Bus boycotts during the Civil Rights Movement. The former played a key role in the American Revolution. While the latter contributed to the end of segregated busing in the United States. Each of these events highlight consumers shifting the power balance of the systems in which they exist. Ultimately, they each led to a redistribution of the social equilibrium within their respective systems.

As detailed in It’s Not All About Growth for Social Enterprises, understanding how a venture is growing requires an understanding of how that venture is working. The evidence of how the social movements that I described above worked is now apparent. The United States exists as a sovereign nation and legalized segregation is abolished in our country. Thus, as the article states, social impact is more important than the growth of any particular organization.

Consumers are the key to growth within any social enterprise. However, the success of each of these boycotts only became apparent years after each event occurred. The results of changing actors within an economic system are slow to reveal themselves. In an ever-changing society, where momentum is arguably as important as revenue, can a social venture that wants to maximize its impact afford to wait for this potential payoff?