Blog

Measuring success

Several thoughts in the last few days have come together to prompt this post. I was thrilled to see that Disrupting Philanthropy was recommended as summer reading, along with Monitor's report What's Next for Philanthropy. What a nice shout out from Phil Buchanan and great company to be in - "What's Next" is a wonderfully well-written set of recommendations for foundations.

Phil's blog post was flattering and it made me think about how we decide what matters. It came on the heels of my being asked to tell one of the nation's largest foundations about the impact of my work - some of which was once done under contract to that foundation. In other words, I've been called to measure my impact.

I've been asked many times over the years to advise efforts on measuring social impact and I've served on countless advisory boards to all kinds of efforts from commercial software firms to nonprofit alliances to government selection committees to who knows what all. Most recently I've been brainstorming with an odd bunch of tech entrepreneurs, web developers, and social businesses on precisely these kinds of questions of meaningful influence and impact.

Since I have, on many an occasion, opined about the importance of data I often get (mis)labeled as a "data wonk" or "an outcomes geek," or something along those lines. I always find this a little funny, because in the very same paragraphs in which I've written that "data are the new platform for change," I've also pointed out that philanthropy is inherently irrational and data are only likely to be used within certain limits. I am a big believer in data but I also know its limits. I believe in asking the question about influence and impact more than I believe we can find one easy, always-useful answer.

Finally, my company, Blueprint Research & Design, is up to our eyeballs on projects where "social analytics* meet community engagement." So even as we're being asked to account for our "impact" we're helping our clients help their grantees do this same thing. This keeps us up-to-date on a wide range of social media impact work - ranging from Jessica Clark's work on public media to the new "21st century statecraft" discussions that have been personified in Alec Ross and Jared Cohen.

Here's my prediction for the future of understanding community change - we need to integrate what can be learned from social analytics into our measurement approaches to community engagement.

Finding the ways in which "data trails on the web" - click throughs, "likes," tweets, forwards, favorite lists, recommendations, diggs, etc - relate to action on the ground are key new measures of activity. Cracking the code between actions on the web and actions in the real world - what predicts what, what leads to what, what foreshadows or follows certain kinds of activities - by whom, how and when - is a key set of skills and methods for project designers, managers, funders, evaluators, policy makers, and community activists.

Why? Because more and more of our social change efforts are being designed and delivered on the web. Those that aren't fully digital (web or mobile) almost invariably have some kind of web/mobile component - whether it be for outreach, awareness, monitoring, or partnership building. To paraphrase Shel Israel, change efforts are at a "flex point" when it comes to digital media.

Just as click through rates changed advertising forever - and with it the news business, recording industry, publishing, real estate - the deep connections between generation and use of data on the web and actions taken in community are going to change how we make change happen. Being able to apply data on page views or unique visitors in a meaningful way to how neighbors organize or communities protest or vaccines get delivered or votes get cast will be as important going forward as old standbys of community organizing (serve pizza at the meeting) have been in the past.

There are whole methods and sciences of "social analytics"* being developed that look at how we travel through our web and mobile and social network worlds - but most of what is known is known from a marketing and sales standpoint. This means that major advertising companies (like Google) and corporate marketing departments (like those at Amazon or Starbucks) know how to answer these questions:

• How many times did I abandon my shopping cart on the web site?
  
• How many times was I shown a certain recommendation before I bought the book?
  
• Did I use the coupon that was texted to my mobile phone to get a free cup of coffee?

We need to take this kind of intelligence and apply it to other questions, such as:

• If I join a disease-oriented social network do I manage my medications better and am I healthier because of it?
• If I read and comment on a story on my neighborhood on a local blog am I more or less likely to show up at the supervisors' hearing on a subject than if I read about it in the print paper?
• What about if I submit a story to that same blog?
• If a follow the tweets from a nonprofit am I more or less likely to donate or volunteer to that organization?
  
• If I become a "fan" of an organization on a social network site will I do anything else to raise awareness of the group? Will I take any offline action to support its work?

Social analytics don't hold all the answers. They may not hold many of them - anyone with a blog knows that page views and unique visitors are highly variable and easily manipulated stats. At the same time, everyone with a Facebook account knows how easy it is to count their friends, Twitterers count retweets and all of us sigh at the number of emails in our inbox or texts on our phones. We've all gotten good at dousing ourselves in our own web-wash of data.

But we don't yet know how to connect those numbers (and which numbers to use) to our actual goals for our work or to our other methods of measuring our success. Do I tell the foundation that asked about my success measures about the number of blog readers and twitter followers? Do I do a citation search on the articles and books I've written to see who references my work and how often? Do I count the number of organizations around the globe that have hired my firm to do strategy work? Should I take the results of our client satisfaction surveys, multiply the positive responses by the number of blog readers, Scribd downloads, and Blueprint 2010 purchases and then divide by the number of negative responses? What about interviews with organizations we worked with years ago who've subsequently referred other organizations to us? Measuring success of my firm and my work hasn't necessarily gotten any easier or better since we've gotten all these potential data points - but it sure gives us lots of mathematical possibilities.


How many twitter followers you have may not matter as much as who they are and who follows them. We still need constituency voice surveys, focus groups, interviews, and ethnographic analysis to really understand how and where change happens. The web is full of data - every time you do anything on the web it generates more data. Some of it is going to be useful. What an opportunity to learn how digital data can help us all do our work better.


*Social analytics, defined by Jeremiah Owyang of Altimeter Group as "the practice of being able to understand customers & predict them using data from the social web."

Posted from at 37000 feet above sea level - I blame the typos on turbulence.
…

Policy change over time at home and elsewhere

I think it is pretty interesting that the Chinese have launched the China Foundation Center (CFC). A research center on philanthropy at Beijing Normal University also launched this year.

Think about it - if you were the most populous country in the world, with one of the fastest growing economies (with all the accompanying good and bad that comes with that) and a rapidly shifting philanthropic sector -- what type of supports for this emerging sector would you build out first? Technical Assistance? Advocacy? A data center? Research? Legal and financial agents, shared space providers, trade publications, professional development providers?

Philanthropy in China is, of course, not new. Foundations as an institutional form, including many of the 1800 whose data will be archived at the CFC are relatively new, as the country continues to shift the roles of the government and the private sector the third space also shifts.

U.S. news articles about the centers focus on the need for transparency and trust of these new foundations by the people. Of course, the information, data and research conducted by these two centers will also be available to the government. It gives us outsiders a chance to watch the dynamics between philanthropic, public, and private from a distance and ask the questions about roles, accountability, transparency, data, access, and trust that are sometimes (always?) easier to ask of others than of ourselves.

Closer to home I've been thinking about a different kind of development on the public-private-independent relationship front - that of the proposed United States Council on Nonprofit Organizations and Community Solutions that Representative McCollum of Minnesota introduced in the Nonprofit Sector and Community Solutions Act (H.R. 5533). Most of what I've read about the act - which isn't much yet - has been neutral to favorable. I'm not so sure - I am still thinking about it. If you can point me to other resources or thoughts on the bill I'd appreciate it (please note them in comments so all can read)

More to come.

…

Gone fishing

Well, no, not really. But I am going on vacation and will be offline for a week. Enjoy yourselves.…

What kind of data are you talking about?

I got this email from Dennis Whittle of GlobalGiving in response to this post on "new solutions from data and crowds."

"I had my own aha! moment when reading your post. As you know from some of our discussions, I am resistant to the idea that more data is "the answer." This is because I chased the holy grail of cost-benefit analysis during my World Bank days, and I came to realize that a) there are fatal conceptual flaws to the idea you can rank initiatives by the same metric, and b) such rankings don't motivate behavioral change in practice anyway. From your post below, however, I realize that you mean data in a broader sense - not just numbers but *information*. Now I can AGREE with that! And it is worth more discussion when we get a chance."

I've had a great time talking with data wonks, open government types, NGOs, communities, activists, White House staff, hospital IT directors and all kinds of other folks over the last many many months about data and the role they play as a platform for change.

But how does this work - why do data matter so much? And what kinds of data am I talking about?

Second answer first - any kind of digital data - photos, videos, stories, numbers, financial information - can play the role as platform for change. For example, think about some of the recent photos of oil covered birds from the Gulf of Mexico. They spark giving of time and money to animal and environmental groups (data encourage action). Some photos are the result of volunteer action - such as the pictures taken by GrassrootsMapping kite and camera systems.

As far as philanthropy is concerned, data MIGHT be anything - grants information, evaluation findings, videos of work happening, pictures from partner organizations, citizen provided survey responses about the state of the local community, text messages that map local crime or that tag community resources.

All of these data matter. They might be useful to lots of people for lots of reasons. If you think of data as anything that can be digitized, and realize that this is what we are sharing using communications technologies, you also quickly realize that data are why we use these technologies. I don't have any interest in what kind of email system is better than the other, I care about the news from my friends, family, and colleagues. This is why I use email. It's the data that matters (the news from my friends) not the technology (Eudora v outlook v Mail)

This recognition matters. It explains the big interest in the iPad from grandparents. They don't necessarily care about all the whiz bang features - they like that it is so easy to use that they can read and send emails to their grandkids.

I don't think data hold all the answers - this blog is named 2173 because I think we are on a constant cycle of learning what we didn't know before - which includes learning that what we thought was right is actually wrong. I don't think data are objective - what we collect, how we frame it, how we present it - every one of these is as subjective as the day is long and have, over the years, led to every kind of human suffering from eugenics to racial segregation to genocide.

And now the answer to the first question - how is it that data matter so much?

I think data are inherently subjective. And that is part of why I think sharing data is so important - as coders says "Many eyes make for shallow bugs." In other words, the more people looking at datasets the more apparent the biases of a few become. For centuries, only "experts," the powerful, and the wealthy had access to most data - whether we are talking about government data that has been locked away and hard to get, photos of abuse at prisons, the location and numbers of oil soaked birds, or health information that would be useful to patients and caregivers but was only accessible to researchers.

The whole power dynamic is shifting around data - THIS is why data can be so powerful. Read Joe Flood's incredible book, The Fires, for a recent and local (1960s, New York City) story about what can happen when "experts with data" don't listen to "experts from the streets." In a book reading I attended in Brooklyn about The Fires, Steven Berlin Johnson asked Flood if the story shouldn't be read as a warning about our faith in data. Flood answered (and I paraphrase here),

"No. The problems come when both data and decision making are centralized. I think the lesson of the book and more recent urban data experiments is we should centralize the data - by which I mean clean it up, store it, and make it mixable and readable - and decentralize the decision making."

I was reminded of this in reading about Sergey Brin's data centric approach to finding a cure for Parkinson's. In this month's cover article of WIRED about Brin's quest, Thomas Goetz writes:

“Generally the pace of medical research is glacial compared to what I’m used to in the Internet,” Brin says. “We could be looking lots of places and collecting lots of information. And if we see a pattern, that could lead somewhere.”

In other words, Brin is proposing to bypass centuries of scientific epistemology in favor of a more Googley kind of science. He wants to collect data first, then hypothesize, and then find the patterns that lead to answers."

This kind of thinking simply wasn't possible before the age of massive data. Time was the scientific method relied on a process of hypothesis - stating what you were looking for and then looking for it. Brin's proposed approach is to look first and ask questions later. The possibilities that lots of people might find lots of things - "Looking for a cure for cancer? Don't overlook this finding, which might be a cure for Parkinsons" - is exciting.

And now return to philanthropy and communities. Imagine if the stakeholders in a community - be it the Bronx in the 1960s or those with Parkinson's and medical doctors and researchers - could bring their individual kinds of expertise to bear on a dataset? This is what happens with sites like Crimestopper and PatientsLikeMe. It is also what happens when people can act on their right to know, a shift marked by Freedom of Information and the #opendata movement. Read this story in Tuesday's New York Times for examples of how information access can change the behavior of the poweful vis-a-vis the poor.

The technologies to do this exist - the challenges in making this happen are about power, privacy, and organizational culture.


…

Buzzword 2010.3 Networked

We hear about networks all the time in every aspect of our lives. Network is a long overdue buzzword - but today is precisely the right day to declare it as such because June 21 marks the virtual launch of the must read manual on the topic, The Networked Nonprofit by Beth Kanter and Allison Fine.* I hereby declare Network as Philanthropy Buzzword 2010.3!

Change is all about the network. Your social network - who you know. The organizational network - how you interact with other organizations and how permeable you make the "walls" around you. The technological network - how you connect electronically.

Now, it isn't every buzzword that comes with its own user's manual. In fact, this is the very first one. And The Networked Nonprofit is exactly that - the user's manual for today's activists, professionals, donors, and volunteers. What David Pogue does for tech consumers, Kanter and Fine have done for change agents.

It is the most complete practical guide for making change in our global, digital, always on world. Beth and Allison have worked with, experimented with, and documented just about every iteration and evolution of social media tools. From blogs to wikis to social networks to video to slide decks to online competitions to twitter fundraising to geolocation tools - you name it, they've played with it, used it, tested it, learned from others, and shared their wisdom.

But the book is about more than tools - it is about an operational culture that starts with the values of openness, sharing, and connections and uses those traits to accomplish a purpose. A perfect example - the book's launch party is all about working in a networked way. Starting on June 21 at 4 pm EDT/1 pm PDT the authors are throwing a virtual launch party - on twitter at #netnon and on u-stream - geared toward driving sales of the book and donations to The Sharing Foundation and Hope for Henry.

Kanter and Fine live and act like the very types of organizations they explicate in the book. As leaders and learners they connect, share, give credit, invite, discuss, rehearse, improve and introduce. They try things out in public - the book was written collaboratively across different time zones, drafted and shared in countless speeches, slide decks, workshops and twitter feeds.

And they've done their homework. The Networked Nonprofit has a dozen examples for every idea it offers - from big organizations and small, digital native enterprises and transformed "old line" institutions, freelance activists and professionals of every stripe. You might read the book from cover to cover, as I did. Or skip from chapter to chapter, looking for nuggets as you need them. Either way, bring your marker (actual yellow highlighter or the e-book reader button of your choice) - you'll dog ear, post-it-note, and underline your way through this. And clear space on your bookshelf - real or virtual - this one's a keeper. Technology changes quickly, but the culture shifts and modes of operating that Kanter and Fine describe are here to stay.



*Full disclosure: I know both Beth and Allison and have learned a tremendous amount from both of them.

…

Philanthropy and Public Policy

I usually write about public policy that shapes philanthropy. But this notice in Mike Allen's Playbook newsletter of June 7, 2010 caught my attention on the much broader issue of philanthropy's role regarding public policies writ large.

"TOM DASCHLE AND VICTORIA KENNEDY, the late senator's wife, are expected to be named co-chairs this week of a well-funded campaign White House allies are rolling out to defend health reform against critics and help states implement it. The Health Information Center is being started by Andrew Grossman, a veteran Democratic operative who founded Wal-Mart Watch, a labor-backed group to challenge the world's largest retailer. Grossman told us the lessons of Wal-Mart Watch will be helpful on health reform: 'When you treat people with respect and try to understand how they interact with businesses and politics, you can move them.'

Estimated budget: $25 million a year, for five years!

The group will eventually have a staff of 10 to 15 people and will be funded by unions, foundations and corporations. Anita Dunn, former White House communications director, is a consultant to the center, working daily with Grossman, who's president. The current board members are Grossman; Erik Smith, who heads Blue Engine Message and Media; and Sheila O'Connell, longtime political director for EMILY's List. It's a 501(c)(3), and later a 501(c)(4) will be added. The group plans to recruit other board members to represent a cross-section of American life -- including members of the corporate community -- in an effort to depoliticize the Affordable Care Act. 'The law is in effect, and the best thing we can do now is explain it,' Grossman said. "

Twenty-five million dollars to defend a law from its critics. That's a lot of money. And it touches on an issue that's been bubbling around my attention span for awhile - the interactions between foundations and government. Nothing new here, except some notable aspects of scale and visibility. Note the above story is about an independently funded (foundations, unions, and corporations) effort to build support for/defend enacted legislation.

I wanted to understand just how big the public commitment to working with philanthropy has become. We've all heard about the Social Innovation Fund ($50 million gov't funds; $50 million in foundation match); the I3 initiative of the Education Department (which led to the I3 repository and announcements of $506 million in foundation money to match $650 million from the government). But I was sure this was the tip of the iceberg.

I had two interns work on the attached scan of White House, the Cabinet departments and the 50 Governor's offices to find philanthropy-related initiatives, liaisons, offices, and programs. It's not easy to add up these numbers - and we're quite sure this list is incomplete - but it is a start.

Engaging the Social Sector - Fed Government FINAL

Check out both documents - Federal and State - and let me know what we got right, wrong, and missed. Big thanks to Coro Fellow David Koken and Stanford junior (And PACS Center intern) Rebekah Morreale.

At this point, I'm just gathering information on philanthropy's current roles vis-a-vis public policies and public programs. All I know for certain - there is a lot of money involved. It's not always direct funding - this article in the Columbia Journalism Review raises some important questions about foundation funding of journalism and what that might mean for public information and debate about public policies. This article from Mark Schmitt at The American Prospect looks at the influence of really large foundations in some public arenas.

In other words, this is classic "raw material, raw thought" blog post. Got ideas, leads, questions, suggestions? Please let me know.


…

FutureLab

NetChange Week (#ncwk) is underway in Toronto, at the MaRS Discovery District. The MaRS Center is a location-based incubator, accelerator, learning community and set of tools that mix technology, innovation and social enterprise. NetChange Week is a conference + skills exchange + problem solving/brainstorm space. Beth Kanter gave the opening keynote, drawing from her new book, The Networked Nonprofit (Co-authored by Allison Fine, available June 21)

I'm headed to Toronto for this session on the Future of the Web and the World. I'm looking forward to the conversation with John Thackera and Gerri Sinclair and learning with all the folks in the room. But what I'm really looking forward to at NetChange is FutureLab.

FutureLab is a two day effort matching social media innovators with community organizations to brainstorm and present possible solutions to real problems. The teams and problems were selected a few weeks ago. The teams will start work with a group of advisers beginning on Thursday morning and they have until 2:30 local time on Friday to come up with solutions to present to the conference. I like this approach because it puts the problem and relevant knowledge first, and technology/social media in the support position. I like it because it has a little more time to it than typical app contests. I like it because the larger community is (at least a little) involved. And I like it because it takes advantage of all the wisdom that gathers at a conference and gives that wisdom something to do.

Here's the list of problems they'll be working on:

1. Designing a National Digital Address System:
How can the Internet be used to better connect citizens with their government and the government with citizens? The SI GovConnext group will develop a series of protocols for opening up a dedicated digital channel between government and the public.

2. Recruitment for rare medical disease research
Clinical trials for medicines require an adequate number of appropriate people participating. This is really hard for rare diseases. The Lab will be used to develop an outreach strategy to people most likely to develop a rare disease like ALS, those genetically predisposed to it. This project is a collaboration between Emory University and Sunnybrook Health Sciences Centre.


3. Independent media/information dissemination in areas of conflict
In areas of extreme violence reporting the news is a life-threatening endeavor. In some areas of Mexico entire neighborhoods stay inside in order to stay out of harm's way. Reporters have to focus on mundane stories or risk their lives - because if you share what people need to know you put your life in jeopardy. Can social media help get the news out and rebuild a sense of humanity in broken societies?

4. Sustainable Behaviour Change
Helping people "see" the energy their homes waste and "see" savings has been shown to increase use of energy conservation strategies. "There is a $28.8 billion opportunity for cash positive energy retrofits to Canadian Homes. Based on environmental education research, it is proposed that if Canadians could see the energy savings potential of their homes, compare themselves to their peer group and get rapid feedback of their relative energy performance, we would see transformative instead of incremental change. Can innovative social media be the key to unlocking the potential?"

Can't wait to see how this works and what the groups come up with. Follow the tweets at #ncwk and tune into the Vimeo channel.

…

New solutions from data and crowds

I'm two-thirds of the way through Clay Shirky's new book - Cognitive Surplus: Creativity and Generosity in the Connected Age - and I wasn't planning on writing about it until I was done. But in a perfect meta-example of both my point and the behaviors of distraction described in excruciating detail in today's New York Times, I got an email, linked to a blog post, commented on the post and now here I am, madly blogging before I get back to work.

I'm reading Shirky's book with pencil and notepad (!!) in hand - taking copious notes and loving it. Shirky does an incredible job of illustrating a point that I care about deeply - it's not the technologies that matter, it's the behaviors and expectations that they allow. (See Disrupting Philanthropy on this point).

What Shirky adds are clear research and compelling stories that make us understand that our human motivations are age old, technologies provide new opportunities, and that this combination (motivation + opportunity) can change our behavior. Not only that, the nature of our current technologies changes the scale of our individual behaviors.

Reading this was an Aha! for me. I've been trying to make sense of the relationship between "data" and "crowds" as a I work on my own forthcoming book. The DataJam crystallized something for me - data and crowds are two sides of a coin. Crowds can organize around data - think of people coming together around photos on Flickr. Then they use the data - commenting on photos, linking to other photos, tagging and grouping photos, adding photos - and they become the source of more data. Another example - Ushahidi unlocks crowdbased data in terms of geotagged tweets or SMS messages. These data are then integrated with, compared to, considered by holders of "central data" - such as the State Department or Red Cross - and suddenly we have a different type of coordination solution for getting aid to earthquake victims. Crowds use data and crowds generate data. There is a feedback loop that can grow and accelerate.

But Shirky takes this further and argues that the feedback loops and the data and the crowds add up to more than the sum of the parts. They provide a new "soup" (my word, not his) in which we behave, old behaviors come undone and new relationships and linked behaviors get created. Our individual behaviors create a collective set of behaviors, and, in turn, the collective changes our individual behaviors. Shirky's focus is on how individuals and groups shifts along these lines, and he makes a clear and accessible connection to the re-emergent interest in the commons as a form of production and governance. (I'll be back with more on Shirky's book - but for now, let me tell you, it is a must read. Available in bookstores on June 10 - Thursday!)

So I read until late last night, got up this morning, read the NY Times about how distracted technologies make us, checked my email and found an email and a link to this post at Change Charity - Collective Venture Philanthropy. I get sent a lot of things like this - but I read the email and clicked through on the link because I had seen a tweet about the post last night, emailed it to myself as a reminder to read it and the two emails (mine to myself and the one from the post's author, Jeff Raderstrong) came up in my inbox back to back.

Jeff's post is well worth a read. He posits the possibility that social media will allow small donors a direct role in launching, creating and funding riskier new startups - moving them to a new place in the philanthropic capital stream then ever before. He uses Kickstarter as a model and reflects on the success of Kiva and DonorsChoose in engaging individual donors in new ways.

I think Jeff is raising the right questions. When we wrote Disrupting Philanthropy, my colleagues Ed and Barry and I had endless debates on the meaning of two of the graphs in the book - The Long Tail of Giving (p 11) and The Long Tail of Receiving (p 12).






Each of the graphs is accurate in and of itself, and they look a lot alike. But we disagreed on what they meant when considered together. We couldn't find any conclusive data or analysis of data that satisfied all three of us on how these two pictures relate to each other. Who funds the long tail of nonprofits? Big donors, small donors, and in what mix? For that matter, who funds the head of the tail of nonprofits - big donors, small donors, or both and in what mix?

We don't have great data on exactly how this relationship has worked - but the power of technology and social media is likely to change whatever relationship has long existed. People are using online giving platforms to find information on giving options, to share their own ideas, to propose their own solutions, and to find like-minded donors. They are cross referencing Nick Kristof columns with Charity Navigator data, searching for organizations on Guidestar and then making a gift on GlobalGiving. They are encouraging their friends to join communities on Facebook and proposing their own community garden projects to charity contests on JustMeans.* We are using and creating a new ecosystem of information (What I call an Infostructure) that informs us, raises awareness of new types of experts and expertise, and generates new ideas for new sources of information. They are using data and adding data. They are behaving as individuals but the collective impact is changing the landscape of giving. And the collective landscape is changing how each of us gives the next time around.

I commented on Jeff's post here - and won't repeat the whole thing on this site. Go check out Jeff's query and my comment and join us in the discussion either here or there. This is a real time, living example of the very forces that Shirky describes, that today's NYT bemoans, and that I believe are here to stay and are fundamentally shifting how we make change happen.

*Full disclosure: The Little Haiti Community Garden Project linked to on JustMeans was brought to my attention in an email from a friend who volunteers there.


…