I stumbled across this interview with Flickr founder, Eric Costello. My favorite bit is this:
JJG: How did you know what features were connecting with your audience?
EC: We have not historically been a very metric-driven company. We do look at numbers, but really we just keep our ears open. We listen to what people say to us on our forums. The bottom line is, when users sign up and actively start using the site.
We had a lot of signups when we had Flickr Live, but relatively few of those signups turned into active users. Whereas when we started introducing traditional Web-style features, we got a lot more signups, and more of those people stuck around and continued to use the site.
JJG: Could you see how much people were using the new features you were rolling out, or was it just that you were hearing people talking about the features?
EC: It was mainly conversations that we listened to. People talking to each other about the site, people talking directly to us about the site. We didn’t really track feature usage.
Bingo! This is exactly why I don’t like numbers. We tend to spend so much time researching what the best way of measuring success is that we completely overlook the problems AND the successes. Flickr has been an incredible success by any metric, but I’d say (with a great deal of certainty) that the reason they are doing so well is because they actually listened. They didn’t set up focus groups to quasi-listen. They didn’t spend two months developing a series of success criteria/metrics that were outdated or irrelevant the moment they completed them. They didn’t bury their heads in their numbers and spreadsheets.
Instead, they spent that time forming real relationships with their users. They keep their heads up, listening, instead of down, crunching. If you’re unfamilar with the typical marketing process, let me assure you – the amount of time spent "keeping track of success status" is overwhelming. I once had a marketing colleague tell me that they spent 50% of their week trying to stay up to date with consumer reaction so they could update the management with the answer to the question: "How are we being accepted by consumers?" When I asked this person how they were tracking this, they told me that they were conducting online and offline polling. When I asked if they themsevles had actually spoen with consumers, they told me that ancedotal information wasn’t accepted by management because it wasn’t "valid".
Yeah, I think Flickr is proving that theory bogus.
