Four Criteria for Choosing The One Metric That Matters

— Discuss on Hacker News —

Engagement metrics are tricky. There are complicated algorithms which aggregate metrics into “Engagement Scores”. There are funnel analysis techniques which show where people are dropping off. Then there are tools that let you add engagement metrics to each and every button on your website. It’s a mess.

Start Simple

At Kera we’re starting simple with our engagement metrics. We know that measuring too much early on can make it difficult to focus on what’s important. We’re not interested in running around like chickens with our heads cut off.

The One

Ben & Alistair are doing a great job helping product teams discover how to use measurement quickly and effectively. They wrote a post about the one metric that matters. Is it possible to isolate a single metric which radiates enough information? We sat down to establish criteria on what kind of measurement that would look like.

Value

One of the more challenging parts of Kera putting our product out there early is showing the value quickly to prospects. We learned that once users see value they are much more likely to pay. Measure the delivery of value to your customers.

Knowledge

The metric needs to tell us something about how much a user knows about our application. For example: We should know that a user knows how to accomplish task A, B, and C in our product if they have generated that metric.

Revenue

If this metric grows, you make more money. This is related to the ‘Value’ criteria above.

Frequency

Being a Lean Startup we know we need to make decisions as fast as possible. We can’t wait until the end of the week or month before we see the metric moving. The ideal metric happens every day.

Our Metric

So what did we come up with? The metric for us to focus on is ‘Number of customers with one walkthrough play’. This meets our 4 criteria. 

Value: If a learner is playing one of our customer’s walkthroughs they are learning about their product or accomplishing a tough workflow. Our customers obviously see value in that.

Knowledge: We know that if learners are playing a walkthrough our customer was able to author and publish it successfully.

Revenue: We are a SaaS business, the more customers that play a walkthrough daily, the more money we make.

Frequency: Kera is designed to help every user. An ideal customer is one that installs Kera into their application so all their users can benefit. We can instantly see if a customer is in trouble by noticing that they haven’t had at least one walkthrough played in their application in a day. We can also measure this monthly and create a DAU/MAU ratio to see how our engagement is growing over time.

Feel free to reach out if you need some help figuring out which metric you should be focus on. We’d love to help.

-Cameron (@camwest)