Why Social Media Measurement is like Gourmet Cooking

It is easy to predict that we will be talking a lot more about social media measurement and ROI in 2009. The question of ROI becomes more important when there is a market transition from an established form or investment (i.e. traditional media) to an emerging form like social media. Even though the ROI of current or traditional investments may be no less certain, the test is always a higher one to justify a change.

This has many people asking, “How do you measure social media?” There has been a lot of valuable conversation about this topic already. Kate Niederhoffer has assembled a really interesting set of predictions for measurement in 2009 in her post: Go Back: The Future of Measurement. I’m glad Kate invited me to join the group, but unfortunately I didn’t have enough bandwidth to participate in the conversation prior to her publish date, but of course the conversation will continue. Here are a few of my thoughts on social media measurement in 2009.

Traditional Metrics

First let’s look at traditional media and its measurement. For a brand or corporation, traditional media has primarily held a single role or purpose, to disseminate one-way information to potential customers either through advertising (paid media) or through news coverage (earned media). For print media, for example, the industry has largely converged on two intermediary metrics: reach & frequency. You basically know how many people receive the publication, how often, and perhaps some information about the reader demographics. Everything else has largely been a derivative of these two metrics. The same comparison can be made for online advertising and the metrics of page views and unique visitors.

I labeled the traditional metrics as intermediary because they do not directly measure or represent results associated with the end goal (ex: what awareness did it create, what sales did it drive, what new relationships did it establish, what actions did it incite, etc.). Did the industry really land on these metrics as the chosen standards because they were the best ones for determining ROI, or were these the only things we could measure so we had to work with them? Because this traditional medium is not directly measurable, we don’t know what “actually happened” as a result of our paid or earned message. Did the message get noticed, or generate conversation, or cause word-of-mouth propagation, or increase brand advocacy, or was it largely ignored?

Beyond Bread and Water

My first law of social media measurement: If all we have is bread and water, then we will learn to make do with bread and water.

It is easy to agree on what we need to prepare for dinner when there isn’t much to choose from. There is not much room for innovation or creativity in preparing a meal with only bread & water. So, bread and water, it has been. Social media measurement today is moving us beyond bread & water.

Social media has several key/unique characteristics that are different from traditional media. First, it is a multi-purpose communications medium. In many ways, social media is less like traditional media and more like email or the telephone/communications network in terms of its multiplicity of purpose. It isn’t restricted to sending one-way messages nor is it limited to a single purpose. Furthermore, the social medium creates many artifacts, or digital breadcrumbs, that are directly measurable as people participate. It isn’t just a medium with a message, but it is also a medium which contains and records actions. The same questions that I posed above are now much easier to answer: did my message get noticed, or generate conversation, or cause word-of-mouth propagation, or increase brand advocacy?

Gourmet Cooking

Social media measurement is like gourmet cooking. We now have plenty of high quality ingredients to work with. There would not be a gourmet chef profession if we only had bread and water to work with. Is there a “standard measure” in the gourmet food industry for how to make a good Fois Gras or Duck D’Orange? No. Of course, it isn’t hard to recognize gourmet cooking, nor is it hard to detect bad cooking. However, the experts will continue to innovate, experiment and advance the art… and, they will argue vehemently about the precise measure of good Fois Gras.

Social Media measurement is like gourmet cooking because the social web produces a vast and growing array of metrics that can be gathered and combined in various ways to extract meaning, insight, and measure the effectiveness of one’s investments & efforts. The list of ingredients is growing all the time: comments, inbound links, votes, views, likes, bookmarks, favorites, tweets, re-tweets, social graph connections, etc… countless social breadcrumbs that are directly measurable both on page (directly connected to your content) and off-page (located in other places, but related to your content) plus they can all be measured temporally adding that important perspective of time: velocity, transience, sustainability/stickiness, etc. Additionally, the types of business functions & endeavors that brands can pursue using the social web are as numerous and diverse as the metrics available – it isn’t just an advertising medium. Measuring the ROI of your investment in providing customer support using the social web is very different from measuring the ROI of your efforts in influencer outreach.

As any industry matures there will certainly be a growing alignment on some best practices & common measurement approaches, but the art (and the conversation about it) will continue to improve, innovate, and change.

Let’s take the measurement of influence as an example. Perhaps Technorati started things off for blogs with a measure they labeled “Authority” which is based on inbound links to blogs (directly measurable). Technorati counted them up over a specific time span and called this Authority. As the social web matured and the discussion continued, many experts started to agree that this simple measure, albeit useful, was too narrow. Steve Rubel called it dead in a July 2007 post discussing a new system for measuring online influence, “The practice of measuring online influence by links is truly dead”. The metric didn’t include a total view of an influencer’s activities on multiple social networks, for instance, and blogs are now just one part of the whole of social media. It also did not include the concept of topical influence and so it worked more like a popularity metric and could not be used as a true measure of influence for a given topic. It did not include the many other metrics that have emerged and become effective markers for inclusion in a measure of true influence. Does a blogger who has more unique commenters, engagement, and two-way on-topic conversations in the comments of his posts have more influence? This was not part of the original equation. Innovators and experts will continue to propose new formulas for measuring influence and will continue to add new social metrics as they emerge on the web. Influence, of course, is just one of many types of metrics as well.

The Standard Measure for Social Media

Will we ever land on a “standard measure” for influence or any other standard measurement in social media that some are calling for? I don’t think so. Call this a 2009 prediction if you like: there will never be a single standardized metric for measuring social media. We have forever moved beyond bread and water. And this is good news!

The complexity and diversity of the raw ingredients available means that amazing things can and will be created by skilled individuals. The industry will generally swarm around some best practices, but the art and applied science of measurement will continue to increase in complexity and effectiveness. Now the experts can use this ever growing set of metrics, and apply them skilfully to the specific business objective for which the investment is being made in the first place. As my friend Katie Paine, a five star Master Chef of social media measurement, always reminds everyone: you have to start with your specific business goals. She says in this post on social media measurement, “There can be no standard because there is no standard goal for communications. (…) So establishing a standard metric for such diverse goals is clearly a waste of time.” Katie, will you be posting your 2009 predictions this year?

Good Enough For Practical Purposes

Early in my studies I was drawn to engineering because I liked the idea that engineers were focused on the practical application of science: making things work. While a science class might measure, for example, the temperature of water and argue over its precision (i.e. it is -10.21543 degrees Celsius) an engineer will first asks why you are measuring it? If I want to make ice, I don’t need to know if the water/ice temperature is -10.215 or -10.225 degrees, I just need to know that it is below freezing and -10 is good enough. The same goes for social media metrics. If my business goal is to determine who I should be building relationships with online regarding a specific topic or which communities are most active about the topic, then it is less important to fuss over the precise influence formula, scores & rank for influencers #4 and #5, since I will reach out to both of them anyway. It is more important that my measurement includes the broadest set of social media properties, media types, content coverage, and the most comprehensive set of metrics such that my resulting scores and lists of influential people and communities are not too narrow, potentially leaving off important communities of influence. In fact, I may want to tweak the influence formula, giving certain metrics higher or lower weightings in order to look at things from a few different angles if it fits my purpose.

If I could be allowed to propose a first principle of social media measurement for 2009, it would be to apply this motto: “good enough for practical purposes”. It does two things: a) forces the practitioner to start by articulating their business goal & purpose and b) drives the science of measurement toward a practical versus theoretical end. It puts measurement in its proper place as a means to an end, albeit a very important one.

More Predictions

If you are interested in 2009 predictions, Peter Kim has aggregated some excellent predictions from several experts I know & respect like Jason Falls, Todd Defren, Rohit Bhargava, Chris Brogan, and Charlene Li. You can download this PDF version of the predictions from his blog.

December 29th, 2008 - Posted in Social Media | | 22 Comments

Too Transparent?

I thought I would share this, mainly because I found it funny, but I also because I think is makes a really interesting statement & observation on a social phenomenon happening online with respect to transparency. I am a big fan of transparency, by the way, but I am also a fan of Microsoft’s umbrella social media policy - “Don’t be stupid”. Is there such a thing as too transparent?

Also, be sure to click on the image and check out bitstrips.com for more great cartoons.

December 12th, 2008 - Posted in Social Media | | 0 Comments

The Top 10 Conversations To Listen For In Social Media

The kind folks at BrightTalk invited me to try out their platform and conduct a live webcast for their Conversational Marketing Summit a few days ago. BrightTalk is a self-serve platform to deliver live and recorded webcasts. I found it quite easy to use and the people at BrightTalk were very helpful too.

I often get questions about the “how to” of listening or monitoring your brand in social media. With this presentation I wanted to focus more on the questions, “what should I listen for?” and “what can I do with what I find?”. So I would like to provide you with a few ideas for the types of opportunities that exist beyond just listening for mentions of your brand.

I hope you find it helpful.

December 2nd, 2008 - Posted in Social Media | | 2 Comments