Not all Twitter Analytics studies are created equal. As is the case here where social media measuring company PeopleBrowsr and O’Reilly Media have provided an entire differently look for BrandBowl 2010. What is the 2010 BrandBowl…?
The 2010 Brandbowl is an online Twitter analysis of the most effective Super Bowl advertisements. Created by Ad agency Mullen and Radian6, the study featured overall effectiveness by volume of Tweets and basic sentiment analysis to determine which brands had the best reaction on Twitter.
The winner, according to Radian6 Mullen was Doritos, with Google, and Focus on the Family coming as second and third place of the 2010 BrandBowl.
Introducing PeopleBrowsr/O’Reilly Media Perspective
Debuting at the Gravity Summit in Beverly Hills CA, PeopleBrowsr and O’Reilly Media produced an in-depth look at the effectiveness of the 2010 Brand Bowl (see entire slideshare presentation here). PeopleBrowsr extracted 103,000 tweets about the ten top brands during the Superbowl and applied an in-depth sentiment analysis with the help of O’Reilly Media.
In comparison to the Radian6 Mullen study, the PeopleBrowsr O’Reilly Media results are polarized. Here is how they differentiate:
Tweets By Volume
Radian6 Mullen ranked brands by totals mentions. PeopleBrowsr O’Reilly took a different approach and ranked brands by positive mentions. As you can see the results are contrasting with Focus on the Family dropping seven postions.
Radian6 Mullen Sentiment
Radian6 Mullen provided a basic sentiment analysis, described as identifying ”positive” or “negative” Tweets to determine who was the “most popular.” They identified the following three brands for having the best sentiment (none were in the top 10):
- McDonalds – 9.73 index**
- Dr Pepper – 9.46 index**
- Universal – 5.26 index**
**Index methodology can be found here. Actual data points could not be found anywhere in the study.
PeopleBrowsr O’Reilly Sentiment
PeopleBrowsr O’Reilly ranked brands on sentiment by using a turk method. Turking is when humans actually review posts – far more accurate than machine review as human sarcasm and double negatives make it impossible for machine sentiment to be reliable. Here is the PeopleBrowsr O’Reilly top three (all were in the top 10 of the BrandBowl):
- Snickers – 98% positive
- Google – 91% positive
- Doritos – 79% positive
Additional Findings from PeopleBrowsr O’Reilly
POSITIVE SENTIMENT BRANDS
LOW SENTIMENT BRANDS
Apples to Apples
The two studies used the same data, same community (Twitter), same time frame (Superbowl Sunday), yet the results couldn’t be more polarized. Why? Sentiment.
The clear purpose of 2010 Brand Bowl was to see what the public “thinks.” The scoreboard was supposed to tell us “which brands are winning in the hearts and minds” of the viewers.
Unfortunately the volume method does not do anyone, or any brand, justice. If you used this method right now (Mar 2010) to determine the best auto manufacturer based on what people are saying in Tweets, the hands down winner would be Toyota. Why? Because we’re in the height of the “Toyota Recall” and the volume method used by Radian6 Mullen simply tallies the number of Tweets and doesn’t account for what is being said (aka, sentiment, aka thoughts, hearts & minds of what people are saying, aka the purpose of the Brand Bowl study).
This is why Focus on Family was #3 on Radian6 Mullen and was #10 (last place) on PeopleBrowsr O’Reilly Media. People HATED that ad!
Kudos to PeopleBrowsr O’Reilly for representing the people’s thoughts in ranking positive mentions:
- Doritos
- Bud Light
- Budweiser
- Snickers
- GoDaddy
- Coca-Cola
- Kia
- Hyundia
- Focus on the Family
The Actual Winner is…
Google. Why?
Effectiveness.
Google had 1 ad that created 12,100 positive mentions.
Doritos had 4 ads that created 41,748 positive mentions.
Pound for pound, dollar for dollar, ad for ad, Google pumped out 12,100 positive mentions per placement, whereas Doritos only produced 10,436.
In Closing
Mullen, the Ad agency responsible for the volume ranking is offering the last place finisher of their study “free creative” for next year’s commercial.
Clearly no one wants to come in last place,” said Edward Boches, chief creative officer at Mullen. “So to ease the pain, we’ve made a commitment to recommend, for free, to the losing brand, creative ideas for next year. Perhaps we’ll even crowdsource those ideas, asking the entire social media community to join us in offering up free recommendations.
I have a recommendation… Use sentiment.
Love that you guys are trying to get on board here. Dialogue is a good thing. Don’t know much about your capabilities other than you sell paid tweets and some of your conclusions are correct. But, you neglect to point out a few things. There were multiple points to BrandBowl. 1. Buzz, we were interested in how much total conversation can be generate by an event, so volume mattered a lot. Take a look at any of these brands, pro or con and they got chatter specific to this event. Lessons from that (i.e. how much chatter can be induced via social) offer value in understanding how to mobilize an analog event. 2. Sentiment was a second thing. Our winners were based on buzz, true. But we also had sentiment winners and published them just as you did; so yes, we had the same data, just offered more ways to look at it. 3. It was participatory experience, with value for users. That’s why it was featured on TV, radio, national press and accounted for .07 percent of all Twitter content during portions of the game. Your data is awesome and well presented. But you compared it to only one third of the data that we made available to participants, press, advertisers themselves and others. Love the fact that you guys are talking about measurement of social. As you know, both Mullen (through our own analytics group) and Radian 6 are big believers that no program is complete without them. Thanks so much for this post. Hope this clarifies some of your assumptions.
I really appreciate your input Edward.
Big fan of Mullen. No cheap shots here. Love dialogue.
I think you’ll agree that the things I pointed out were simply the main points that you presented to the people. For example, your core analysis was featured in a creative piece that ranked brands by volume of Tweets (as you say “buzz”) depicted in a massive scoreboard, roughly 340 pixels x 600 pixels.
Your approach to presentation was my approach to analysis… Fair? Right?
In regards to the “few things” that I “neglected to point out,” I’ll assume you’re referring to the size 10 font below the fold, located down and to the right (http://brandbowl2010.com/). I actually spoke to all of them expect one.
I didn’t speak to “Brand Bowl Top Three” because you presented the results with three digit numbers next to the brand name: 156, 103, and 81. I have no idea what those numbers are suppose to mean and there wasn’t a link to explain the methodology. So I didn’t write about it… Fair?
In regards to the other data points, I did in fact write about both… Volume. Sentiment.
In conclusion, I love the execution! A+ for creative style, A+ for the idea but the study itself failed to achieve the premise of the entire execution which was suposed to be a “scoreboard that will tell us which brands are winning their hearts and minds.”
Key supportive data point: Focus on the Family #3 (this was hated by the people)
Have I not been fair?
Best regard,
Derek
Go Daddy coupon code news. Several of the frequent Go Daddy promo codes are inactive. Here are latest promo codes that are valid. These promo codes will work thru 2011. $7.49 Domain names and renewals – Use Godaddy.com coupon codes GOO3, OK9, or ZINE10. 25% discount on orders of $100+ – Use Go Daddy promo code OK25. 30% Discount when you buy any com domain – Use Go Daddy promo code OK30. $12.99 SSL Certificates – Use Godaddy promo codes GOOSSL, OKSSL, or ZINESSL. Host Plan Promo Code – 20% Off Hosting – Use Go Daddy promo codes OK20H, ZINE20H1 or GOO20H. 10% off any size order – Use Godaddy coupons OK7, GOO1 or ZINE8. $5 Off $30 or More – Use Godaddy promo codes GOO2 or ZINE9. 20% Off Any order of $50 or more – Use Go Daddy promo code OK8.