Bad Data Can Ruin Your Content (As Told By Four Brands)

I’m a sucker for a good data point… and deep down I think we all are. Nothing sends your argument home like a well-placed statistic. After all, you can’t argue with the numbers.

Or can you?

Let’s look at these five brands that look ridiculous because of the data they use.

1. Match.com

The egregious stat: “Did you know that people who join Match are three times more likely to find a relationship than people who don’t?

The sin: unclear data.

Match.com is trying to elicit a feeling of hope in viewers who are trying to find a significant other and most people will walk away from this ad thinking that Match is a top online dating site. The problem rises when you start picking apart the statistics.

  • Does this mean that people are more likely to find a relationship if they join Match.com as opposed to OkCupid or Plenty of Fish?
  • Does this mean that people are more likely to find a relationship if they join Match.com as opposed to not signing up online dating at all?
  • What time frame does this imply? Are they more likely to find a relationship in two weeks? A month? Three months?

Anyone who thinks about the statistic for more than a few seconds will see Match’s credibility crumble away.

Match.com released a survey proving the effectiveness of their product. Out of 3,000 participants, users had 27% more dates through Match.com than their leading competitor. That statistic is true, and it stands up in court, but it’s not interesting. More often than not, marketers go for the interesting numbers over the true one.

Do some testing on your data points and ask people to pull them apart. Your audience might not need to know the sample size and methodology, but leaving huge gaps in information will make them trust you and your statistics less. Adding a few words can go a long way to boosting credibility.

2. Skechers Shape-Ups

The egregious stat: Wearing Skechers Shape-Ups will increase “muscle activation” by up to 85 percent for posture-related muscles, 71 percent for one of the muscles in the buttocks, and 68 percent for calf muscles, compared to wearing regular running shoes.

The sin: Using poor research methodology (and lying).

No one actually believed that Kim Kardashian never sets foot in the gym and has a killer body just because she wears Skechers Shape-Ups, but millions of people did believe the shoes had the ability to burn fat and build muscle tone.

According to the FTC, the data proving that Skechers Shape-Ups had the ability to tone muscles was derived from a one-day study on one test subject. That’s an abysmal and unacceptable sample size to promote scientific claims – especially those related to exercise. Also, the chiropractor who conducted the study happened to be married to a Skechers marketing executive and was paid to find certain results. Not surpisingly, Skechers failed to mention that in their product copy.

The obvious lesson is to never make up statistics, but the other lesson lies in methodology. A one-day study on one patient? That’s inexcusable. Make sure you’re transparent and thorough with your data methodology or you will quickly be disproved and your credibility will be shot.

Side note: “full disclosure” is also an important phrase in this case. Had they been honest about the chiropractor’s relationship it wouldn’t have seemed so scandalous in the long run.

Are you ready for a true data point? Skechers had to pay $40 million in the form of 509,175 checks to refund consumers for their false advertising.

3. Forrester

 The egregious stat: Forrester surveyed 395 marketers [who said] Facebook creates less business value than any other digital marketing opportunity.

The sin: Drawing extreme conclusions.

Earlier this month, Forrester posted an article on their blog dissecting the value of various marketing tactics.  In a desperate attempt to get headlines it attacked Facebook and demanded answers to the data. Facebook called the data ridiculous and major media outlets started calling out the research group on their over-dramatic headlines. Here is the chart:

104441_2

And here are the problems with it:

  • The answers weren’t mutually exclusive, so giving email marketing a high rating didn’t mean social media would receive a low rating.
  • The headline accused Facebook of holding back marketers, when Twitter, Google+, YouTube, and LinkedIn all rounded out the bottom. Facebook alone  isn’t the lowest, social media as a whole is the lowest.
  • The difference in satisfaction between On-site reviews (3.84) and Facebook (3.54) is 0.30. That’s still between a rating of three and four. It’s not as if Facebook was at a one and reviews were at a five.

While the data was bland, Forrester went with shock-factor headlines to get attention. The result backfired and their reputation was damaged. If you offer a quality product and want to seem credible then you need to present your data objectively.

4. USA Today Snapshots

The Egregious Stat: Italy is the largest consumer of pasta in the world. Go figure.

The sin: Presenting common knowledge as if it’s breaking news.

usa-today-infographicVia

What could make this chart look cool? If it just covered which countries eat the most pasta outside of Italy. The idea that Venezuela, Tunisia, and Sweden round out the top seven countries that eat the most pasta is actually pretty interesting.  Instead the reader digests (no pun intended) the common knowledge that Italy eats a lot of pasta.

Fortunately, USA Today isn’t trying to rock the boat with their Snapshots, but they’re quick examples of data visualization for the sake of data visualization. Just because something can be put into a chart or infographic doesn’t mean it should.

Here’s a second example:

usatoday1Via

If Twix or Snickers ran away with a victory regarding favorite Halloween candy, I would be interested. But chocolate as a more popular treat than chewy candy, gummy candy, and candy corn? That’s hardly news.

As we’ve seen from the examples above, bad data can ruin your credibility, isolate your target audience, and make you the laughingstock of your industry. Before you set out to promote a study or base a campaign on a particular data point, triple check the source and context to make sure it’s up to your standards.