Not all Twitter Analytics studies are created equal. As is the case here where social media measuring company PeopleBrowsr and O’Reilly Media have provided an entire differently look for BrandBowl 2010. What is the 2010 BrandBowl…?

The 2010 Brandbowl is an online Twitter analysis of the most effective Super Bowl advertisements. Created by Ad agency Mullen and Radian6, the study featured overall effectiveness by volume of Tweets and basic sentiment analysis to determine which brands had the best reaction on Twitter.

The winner, according to Radian6 Mullen was Doritos, with Google, and Focus on the Family coming as second and third place of the 2010 BrandBowl.

Introducing PeopleBrowsr/O’Reilly Media Perspective

Debuting at the Gravity Summit in Beverly Hills CA, PeopleBrowsr and O’Reilly Media produced an in-depth look at the effectiveness of the 2010 Brand Bowl (see entire slideshare presentation here). PeopleBrowsr extracted 103,000 tweets about the ten top brands during the Superbowl and applied an in-depth sentiment analysis with the help of O’Reilly Media.

In comparison to the Radian6 Mullen study, the PeopleBrowsr O’Reilly Media results are polarized. Here is how they differentiate:

Tweets By Volume

Radian6 Mullen ranked brands by totals mentions. PeopleBrowsr O’Reilly took a different approach and ranked brands by positive mentions. As you can see the results are contrasting with Focus on the Family dropping seven postions.

Radian6 Mullen Sentiment

Radian6 Mullen provided a basic sentiment analysis, described as identifying ”positive” or “negative” Tweets to determine who was the “most popular.” They identified the following three brands for having the best sentiment (none were in the top 10):

1. McDonalds – 9.73 index**
2. Dr Pepper – 9.46 index**
3. Universal – 5.26 index**

**Index methodology can be found here. Actual data points could not be found anywhere in the study.

PeopleBrowsr O’Reilly Sentiment

PeopleBrowsr O’Reilly ranked brands on sentiment by using a turk method. Turking is when humans actually review posts – far more accurate than machine review as human sarcasm and double negatives make it impossible for machine sentiment to be reliable. Here is the PeopleBrowsr O’Reilly top three (all were in the top 10 of the BrandBowl):

1. Snickers – 98% positive
2. Google – 91% positive
3. Doritos – 79% positive

Additional Findings from PeopleBrowsr O’Reilly

POSITIVE SENTIMENT BRANDS

LOW SENTIMENT BRANDS

Apples to Apples

The two studies used the same data, same community (Twitter), same time frame (Superbowl Sunday), yet the results couldn’t be more polarized. Why? Sentiment.

The clear purpose of 2010 Brand Bowl was to see what the public “thinks.” The scoreboard was supposed to tell us “which brands are winning in the hearts and minds” of the viewers.

Unfortunately the volume method does not do anyone, or any brand, justice. If you used this method right now (Mar 2010) to determine the best auto manufacturer based on what people are saying in Tweets, the hands down winner would be Toyota. Why? Because we’re in the height of the “Toyota Recall” and the volume method used by Radian6 Mullen simply tallies the number of Tweets and doesn’t account for what is being said (aka, sentiment, aka thoughts, hearts & minds of what people are saying, aka the purpose of the Brand Bowl study).

This is why Focus on Family was #3 on Radian6 Mullen and was #10 (last place) on PeopleBrowsr O’Reilly Media. People HATED that ad!

Kudos to PeopleBrowsr O’Reilly for representing the people’s thoughts in ranking positive mentions:

1. Doritos
2. Bud Light
3. Google
4. Budweiser
5. Snickers
6. GoDaddy
7. Coca-Cola
8. Kia
9. Hyundia
10. Focus on the Family

The Actual Winner is…

Google. Why?

Effectiveness.

Google had 1 ad that created 12,100 positive mentions.
Doritos had 4 ads that created 41,748 positive mentions.

Pound for pound, dollar for dollar, ad for ad, Google pumped out 12,100 positive mentions per placement, whereas Doritos only produced 10,436.

In Closing

Mullen, the Ad agency responsible for the volume ranking is offering the last place finisher of their study “free creative” for next year’s commercial.

Clearly no one wants to come in last place,” said Edward Boches, chief creative officer at Mullen. “So to ease the pain, we’ve made a commitment to recommend, for free, to the losing brand, creative ideas for next year. Perhaps we’ll even crowdsource those ideas, asking the entire social media community to join us in offering up free recommendations.

I have a recommendation… Use sentiment.