K Street Cafe

Cross-posted on e.politics

A tough question came up in a conversation with a visiting group of Danish communications professionals last week — how do you actually measure the effectiveness of social media outreach? At that moment, the questioner seemed to be looking for some grand sweeping mechanism, but I think the reality is much more complicated: how you measure social media depends on what you’re trying to make it do.

Trying to Grab Hold of a Cloud

Here’s the problem: as with so much communications work, the effects of social media outreach can be quite diffuse. Say your advocacy campaign has a video on your issue out on YouTube — how do you measure the influence it has on the public mind? Some thing with that network of activists you’ve laboriously built up through Facebook — how do you find out how much good they’re actually doing you?

In traditional media outreach, we usually rely on proxy measurements to get hints of the answers to questions like that. How many press hits did we get, and in what publications? How many reporters participated in our media briefing calls? On the Hill, the answers are sometimes more concrete — your bill either passed or it didn’t — but most legislative change is (excruciatingly) incremental. Goals are often more long-term than getting a measure passed (or defeated) in a single session, and so we’re back to proxies.

Let’s apply this idea to social media and see what we get. A couple of quick considerations — first, don’t fall into the habit of focusing on what you can measure rather than on what matters. Proxies should be proxies, but they can become ends in themselves when we forget that they’re really just indicators of something more significant. Obsessing about process rather than results is a frequent trap in the nonprofit/advocacy space.

Second, online technologies often suffer BECAUSE they’re measurable — for instance, when a 1% click-through rate for online ads is considered to be good, people can start to question why they’re bothering to advertise on the ‘net at all. What’s not so obvious is that we usually don’t have equivalent numbers for other media (in most cases, you don’t have a mechanism in place to measure the conversion rate for a single print or TV ad), and the effective “click-through rate” for other forms of outreach may be WORSE!

Goals Provide Yardsticks

In the advocacy world, most outreach efforts involve one or more of three basic goals: trying to affect government legislation and/or regulation, trying to build a base of supporters for long-term fundraising and/or grassroots action, and trying to influence the broader public policy discussion. Right away, it’s obvious that the first two are going to have the most concrete measurables: for instance, you get what you want on the Hill (more likely, part of what you want) or you don’t. Likewise, support-building usually equals list-building, and you either build a list or you don’t.

For Hill/regulatory work, the measurables are likely to include counts of grassroots advocacy emails sent, constituent visits to district or DC offices, constituent calls to legislative offices, supporter quotes in mainstream media news stories, and letters to the editor in local newspapers. They may also include analogs to traditional media work, with blog hits and mentions in influential online discussion groups and listserves substituting for mainstream media hits. The most useful hints will probably come from your lobby team, since they can measure staff and member opinions directly — they’re likely to hear about it if you’re flooding congressional offices with calls. The ultimate measure of effectiveness is whether or not a congressmember comes to support your side, though it may be hard to isolate a single cause for the conversion.

Grassroots support-building is easier to gauge, since it almost always comes down to list-building. Regardless of whether you’re building your database through social media outreach, at in-person events, through phone banks, etc., you’re still building a list! Besides the raw numbers of supporters that come in through different channels, you can also look at the proclivity of members to take action — do the people you bring in through Facebook or through that clever online game contribute as much as people recruited through other means, for instance. A list is almost always a means to an end rather than an end in itself, but modern CRM (customer/constituent relations management) tools will let you slice and dice your database to help you get the most out of your outreach regardless of why (or how) you’re doing it.

The most difficult social media goal to measure is almost always going to be your influence on the public discussion. Here we’re close to traditional media work again, and the yardsticks shouldn’t sound too alien to anyone who’s spent time in a press shop. How many times are you mentioned in the top-level blogs in your topic area? How many bloggers were on your conference call, and how many posts resulted? How many times has your online video been viewed, what websites have embedded it on their pages, and was it picked up on Digg and other social news sites? How many people have put your promo graphic or widget on their MySpace profiles? You might also measure your effectiveness by your enemies — how many comments and video responses did your YouTube masterpiece spark, and how many hostile bloggers reacted violently to what you said?

If you’re really making a splash, you may see the results of your work in public opinion polling, but again it’ll be hard to separate out the influence of any one communications channel. A more obvious indicator is the spread of your message beyond the channels you’re using directly, for instance when your online video gets picked up on cable news and is seen by millions.

Let’s Get to Work

Going back to the original question, social media outreach in the political space is more measurable than it might seem at first glance, since you should have concrete goals to hold it up to (and if you don’t have concrete goals, you probably ought to start asking why you’re in this business at all!). It may not always be possible to pin down the exact results of a given outreach effort, but proxy measurements should let you compare the relative usefulness of different strategies, at the very least. Just remember, don’t focus on the measurable and forget about what you’re actually trying to get done.

– cpd