Do We Need Numbers to Assess Social Impact?

I do not agree with the assessment tool Ted London proposes in his article Making Better Investments at the Base of the Pyramid to measure organizations’ performance and social impact.  I agree with the idea that business, non profits, and other organizations who deliver products to and purchase goods from the base of the pyramid cannot rely on measures such as tasks completed or milestones achieved.  I find Ted London’s framework extremely useful to set the long term objective in an organization, or to identify and enhance the positive effects, or to find and avoid the bottlenecks.  Nonetheless, I think his framework is weak and inconsistent to measure social impact and performance. 

In my opinion, Ted London’s framework has five flaws:

1.       It is not clear in how to determine which effect would be a major one and which a minor one. It seems that you have to make that calculation by intuition or according to your personal interpretation.  Usually, these kinds of calculations lead to personal biases.

2.       Let’s suppose that you have two major effects and one is positive and the other one is negative, how do you determine which one is the most important one?  It can be the case that each stakeholder has a different weight for those effects. Well, which is the good one? How do you know that? In a large organization with many stakeholders this method would be chaotic to make decisions.

3.       The framework has zero objectivity. You have to determine if an effect is positive or negative. The problem is that social issues can have many interpretations.  What is good for me, it is not necessarily good for you and vice-versa. There is always a trade off in resources. For example, in the case of VisionSpring, the new glasses affected positively elderly artisans and negatively young ones and the author thinks that the effect for younger artisans is a minor one. How does he know that?

4.       In trade and economic transactions, there is an infinite chain of effects. How do you enlist the all the positive and negative effects? It is a complex system and you would have many effects interconnected; how do you determine which are important and which are not?

5.       Finally, I agree that the organization has an impact in the community, but what is the scope of the community? How do you define “this is going to be my community”? A community could be only the neighborhood or the village, or a city, or a social community related to a church for example.

I think the three measure dimensions the article proposes (economic, capabilities and relationships) cover a wide spectrum, but they are not properly defined. They try to range over many aspects and at the end they are very ambiguous.  

In my opinion, it would be better for an organization to identify first its objective, if it has a single one or if it has many.  Then, they should select some variables or indicators to measure the accomplishment of those objectives over time. For instance, in the VisionSpring case they should set as objectives to increase women and elderly income, and focus into evaluating only these two through a couple of variables.  It is tough to measure social impact and I cannot figure out another way to do it, it is difficult to define which is the scope we want to measure. There are too many variables to take into account and the criteria to define which are important and not is hard to define. As well, I am reluctant to the idea of not using numbers to measure social impact. Do you think we need numbers and variables to measure social impact? I think yes we do.  In any case, conceptually I like more the approach to social impact developed by Geolff Mulgan, who understands social innovation as an interaction of supply and demand.