Painting a User Story

- Discuss on Hacker News -

When your software plays host to millions of users, big data aggregates and insights are the only way to manage your understanding of users’ usage patterns.  But a growing business with hundreds or thousands of users still needs to pay attention to the individual.  You need to understand where a user struggles and succeeds.

At Kera, we still rely on individual user feedback to help us make decisions and, of course, help individual users succeed.  It’s important to us that every customer with the will to succeed makes it happen. But try asking a mathematician to define ‘will to succeed’ and the only correct answer you’ll get is: 'that depends’.

What we do know is that trying to contact a customer weeks after they’ve given up or forgotten is weeks too late.  The sooner we know the better.

We have the usual set of metrics set up: ‘last usage’, ‘error count in the last x days’, ‘logins in the last x days’.  But these only provide a glimpse into a larger data set that makes up the user experience.

To give us a the bigger picture, we crafted the User Story graph.


This particular user story graph tells us how the user initially created a tutorial and updated it a few times.  Then a few days later worked extremely hard on making it work. A lot of updates and a lot of errors.  Luckily this user would be caught by our normal metrics that track errors, but it also reveals something we wouldn’t have seen otherwise:  they didn’t open the documentation pages even once.

Is the user just not aware of the documentation?  Maybe they could benefit from our concierge service to make it for them?

We know this is a customer with fierce determination to make it work so we definitely want their business. But they probably need a helping hand.

Now when we reach out, rather than blindly inquiring about the errors, we can also suggest the documentation page or offer our concierge service.  A customer reach-out that’s tailored to the individual.

New knowledge, new direction

Then later, maybe we notice the same pattern on more users. And then we have ourselves a new user profile and metric: (# errors received) / (# times documentation opened) ratio.  

If we see a great deal of users that fit this profile, it brings up new questions. Is the documentation not apparent or obvious? Should there be more inline hints or suggestions?  Should the workflow force a user to open docs?

All of this stemmed from a metric we wouldn’t have thought to put in the forefront with the rest of our usage metrics: the number of times the documentation was opened.

The standard set of metrics give you a glimpse into what you already know to look for, but do yourself a favour and allow yourself to see the entire picture.  You might see something you weren’t expecting to find.

- Dave Wright (@datwright)