I suspect Windows Phone team are chasing the wrong metrics.

I’ve had an interesting and insightful conversation with Steve Sinofsky the past 24hrs that has got me thinking about the concept of how we measure success. Firstly, I know its freaking me out how much I am learning from the one person that I honestly thought was the single point of failure at Microsoft – but – I was wrong! as I suspect he had more to offer than Microsoft was willing to absorb that or he was too busy trying to steer whatever ship(s) he was steering to discuss.

To the subject at hand.

When I think about Mobility and how we measure the success and failures, I automatically break open my OneNote file on where the numbers are currently at when it comes to Windows Phone 8, iPhone, Android etc. The first thing I often do is review what the market share looks like and then probably grow agitated at how slowly Windows Phone is moving (yay 50% growth, but they need 300% to break into the 3rd place category).

The problem with just tracking market share is the data has no soul, which leads me into the points I’ve been having with Sinofsky around how data is just a signal of behavior but it doesn’t tell you the entire picture accurately.

How do I mean accurately?

If you look at iPhone today you will no doubt see, it has a huge amount of the market share pie, but in all honesty, that doesn’t tell you much in the way of actual usage or replenishment rates. All it tells us is right now on planet earth there is just a very large amount of iPhones floating about the place and typically, many new customers are switching on iPhones for the first time each day we breathe oxygen.

If you however look to your left or right and ask your peers how often do they buy iPhones they may typically come back with an answer that resembles “one to two years” In that they aren’t the ones likely to retain parity with Apple’s release cycles anymore?

What happens to their existing phones when they fade it out? Is it relegated to the grandparents? Given to the kids as a gaming device? Sold on ebay?

That for me is the metric I want to know the most about, how many new phones are people adopting and lastly how long does it take between phone adoptions. Then if you can layer in operating system, form factor (sizes) and latency between adoptions that would probably give early signs of where movement between adoptions is happening.

Having that data set will also tell us all an open transparent story around how each mobile phone corporation in the race for success is able to sustain their adoption & life cycle. It also would give teams like the Windows Phone team a smarter metric to go after as if a typical iPhone customer today is taking 1-2 years before they migrate to a new phone or upgrade that in itself is the period in which you would need to strike aggressively.

How to navigate these waters with the right data is the key and focusing in on who has the biggest slice of the pie tells us an end total of who was smart enough to figure out the overall collective metrics. It doesn’t tell us a story around who’s strategy was successful and where?

For instance, which phone size right now is the bliss point in size? Which phone color is the best? Which phone feature seems to excite the most? And so on.

These are data points and many more like them that are quite hard to mine for given most companies will hide that as much as possible to save embarrassment or alert competitors of success. Which is fine it just sadly feeds the beast around “% marketshare = success” rhetoric.

You can boost your percentage if you just give phones away for free in China/India as sure, it will hurt your revenue model(s) but it would boost your confidence in the market level(s) and probably lighten the burden on your marketing budget as well. Clearly though that is a terrible strategy given it is going after the % and can’t sustain itself long term.

I guess my end point and the lesson of the week is basically, what the definition of success here is in our industry? Is it to have 40%+ market share for a particular brand which in turn influences our decision(s) to buy or are we being shaped / groomed into buying these devices because suppliers are assuming that having market share means an easier sale? Downside is we are probably buying a form factor or device because of noisy influence vs. the right fit.

Point and case – I bought the Nokia 920 because everyone I knew said it was the best of breed at the time for Windows Phone 8. I automatically forgave its size because I wanted to hear that story around it being the best. It took me a day before I developed buyer’s remorse solely on the size of the device as sure it had qualities that I liked about the phone but I really didn’t need to go above my iphone size?

I went into that purchase with two sets of bias and I allowed the bogus data to shape an outcome that I ultimately did not want. Stupid but interesting how I was influenced.