"Proofiness" and social impact

A consistent problem in social enterprise is measuring social impact. We're all familiar with regressions, correlations, and reading MSE and R-square values. But do these tell the entire story, and can they ever really make a convincing causal argument? And how often do we simply know in our hearts that what we're doing is making a difference, so we go out and find data that support our intuition?

The New York Times recently interviewed the author of "Proofiness," whose advice I'd recommend we follow when conducting and reading impact analyses. "Proofiness" refers to "using numbers to prove what you know in your heart is true, even when you know it’s not." An excerpt from his interview:

I think the biggest thing to take home is that you have the right to question research, the right to think this number doesn’t make sense. I think the best thing to do is if something doesn’t make sense to you, you’re going to learn something by examining it. Sniff it. Figure out where it’s coming from. A little degree of skepticism is usually warranted, especially when there is a number that doesn’t make sense.

It's all well and good to be able to question others' analyses. But how can we assess our own impact in a convincing and meaningful way?