It’s been a while since I last blogged, and I can only say that I’ve been managing my time poorly! I really need to get better at this. In an ideal world, I’d like to come out with one blog per day, but I feel like I would be setting myself up to fail, which is why I’m not going to commit to that just yet.
In any case, I’ve learnt a tremendous lot since I last posted. The stuff I’ve learnt can easily be the fodder of several blog posts, and this is going to be the first of many. My first lesson is that product management is nothing without data. To run a successful product company means to build a product and a system to collect data from the product.
Collecting data can’t be an afterthought.
You can choose whether or not to analyze the data you collect, but you need have systems built in from Day 1 that have the ability to collect and store this data.
Fortunately, there are now multiple tools that can help you do this which is why there is no excuse for neglecting this step. I’m working on a project that at first glance is intensely creative, but the thing is each decision about the product is driven by data and not gut instinct. I’m prone to saying stuff that is just based off my intuition, things like “this menu is too complicated, users won’t understand it”. I know better now.
This isn’t a conclusion, it’s an opinion—a hypothesis. Every hypothesis needs to be tested and the only way to do that is with relevant data. So in this particular example, I would look at the following aspects of the data to see if my hypothesis is correct.
- How many people even use the menu?
- What buttons in the menu are they clicking?
- Are people getting to where they want to go through the menu?
If a significant percentage of users are able to use the menu effectively, that means my original hypothesis was incorrect. It’s taken a while, but I now know not to make unsubstantiated observations based on my gut (unless something is obviously terrible or amazing). When there’s something new you want to introduce to the product, then A/B testing is the best way to decide which version is most successful.
In a nutshell, an opinion is a hypothesis. Test it until it’s bulletproof. Only then does it become a conclusion. The only way to test anything is data. You need qualitative analysis to understand what the data is saying, but the first step is to collect and store data (which most companies miss from my personal experience). Don’t be that guy.