This is super awesome read. Thank you for putting this out in such structured manner Pranav. Have already subscribed to the newsletter, any way i can get access to the spreadsheet you attached at the end? Would want to absorb the process and possibly use it for my own work.
Thanks Pranav for sharing this lucid guide. The readability was remarkably easy and the reader could follow easily the flow of information. While reading I brought it into action, trying to bring in more formality in my process of building metrics to identify and understand user behaviour for a product I'm consulting.
Hey Georgios! For the secondary (supporting) metrics: we use them to get an understanding of why the north star moved up or down. Like in the ecommerce example, if the number of transactions go up, that likely happens because of a product intervention, let's say, making the payment process easier. So when you're launching this new feature, you should see two metrics move - the primary (increased number of transactions) and the secondary (support) for dropoffs at payments. If both move, then you can claim that because you made payments faster, you now see an increase in transactions. If you don't see dropoffs at payments reduce, then the increase in transactions is likely not caused by you, but rather by something else. Supporting metrics also allow you to see what that something else was.
Now imagine a scenario where you decide to make payments faster, but neither your transactions increase, nor your dropoffs at payments increase. You now want to figure out why your change to make payments faster didn't actually increase the speed of payments. For this, you will need to dive deep into a small part of your funnel - say from the point that the user clicks the 'buy' button, to making payment. You would list these steps down in detail, and track each of these steps, to see what step causes friction. This allows you to see if a button or a piece of text you changed on the product caused users to slow down or get confused.
That's the difference. Your secondary or support metric confirms or denies that you caused a to your north star. Your funnel metrics help you understand what went right or wrong with the change you made. Is that clearer?
To your software/tools question - most large organizations use their own data stores and pipelines. The right tool really depends on the size of the data you want to ingest. Here are some off the shelf tools you can take a look at: https://www.datapine.com/articles/data-analyst-tools-software
This is super awesome read. Thank you for putting this out in such structured manner Pranav. Have already subscribed to the newsletter, any way i can get access to the spreadsheet you attached at the end? Would want to absorb the process and possibly use it for my own work.
Thanks again Pravanv
Thanks Pranav for sharing this lucid guide. The readability was remarkably easy and the reader could follow easily the flow of information. While reading I brought it into action, trying to bring in more formality in my process of building metrics to identify and understand user behaviour for a product I'm consulting.
This is wonderful, thanks for sharing!
Hello Pranav. One quick question, what do you mean exactly by "supporting" & "funnel" metrics?
For funnel I kind understand, are the different metrics at every stage of the user journey, but supporting? Are they the submetrics?
And another one. Which is the best tool in your opinion to measure performance and which is the best software to store those results?
Hey Georgios! For the secondary (supporting) metrics: we use them to get an understanding of why the north star moved up or down. Like in the ecommerce example, if the number of transactions go up, that likely happens because of a product intervention, let's say, making the payment process easier. So when you're launching this new feature, you should see two metrics move - the primary (increased number of transactions) and the secondary (support) for dropoffs at payments. If both move, then you can claim that because you made payments faster, you now see an increase in transactions. If you don't see dropoffs at payments reduce, then the increase in transactions is likely not caused by you, but rather by something else. Supporting metrics also allow you to see what that something else was.
Now imagine a scenario where you decide to make payments faster, but neither your transactions increase, nor your dropoffs at payments increase. You now want to figure out why your change to make payments faster didn't actually increase the speed of payments. For this, you will need to dive deep into a small part of your funnel - say from the point that the user clicks the 'buy' button, to making payment. You would list these steps down in detail, and track each of these steps, to see what step causes friction. This allows you to see if a button or a piece of text you changed on the product caused users to slow down or get confused.
That's the difference. Your secondary or support metric confirms or denies that you caused a to your north star. Your funnel metrics help you understand what went right or wrong with the change you made. Is that clearer?
To your software/tools question - most large organizations use their own data stores and pipelines. The right tool really depends on the size of the data you want to ingest. Here are some off the shelf tools you can take a look at: https://www.datapine.com/articles/data-analyst-tools-software
On second thought, maybe tooling recommendations is something I need to write next. I'll do that, meanwhile. I hope the link I shared helps!
Thanks man, I appreciate!