If you don’t do A/B testing because you think it’s hard, you should try Optimizely. It’s amazingly easy to set up and you don’t need to bug your developers to deploy any changes. It can do wonders for your conversions!
Take our case. We need users to play our tutorial to understand how cool and powerful our interactive live tutorials are. Once a user has played our tutorial there’s a 45% chance they will sign up. So step 1 - we need to get them to play our tutorial.
How do we do that?
1. Original. Sept 28th - Oct 5th. Results.
We had about 2.6% of people who came to our site sign up. That’s not great. Let’s increase this by getting more people to play our tutorial.
2. Testing. Oct 6-16th. Results while running.
We started doing A/B tests in this period by running some variants. The original button said “See An Example”. We tried “Play Tutorial” and “Watch a Tutorial.” We tried a bigger button and a different heading “Live Tutorial” also.
The results show that “Play Tutorial” button gets 24.8% of visitors to click and the original button only 15.9%. That’s a 56% improvement!
*The long straight lines in the middle appear because experiments were completely turned off for those days.
3. Solidify learnings. Oct 17th - Oct 22th. Results.
On Oct 17th, we rolled out the variant “Play Tutorial” button to all visitors. This is our new home page.
You can see the % sign up increased from 2.6% to 3.5% which is almost a nice full percent.
We will continue testing to increase our conversions. This is just an example of how A/B testing can improve your conversions.
By the way, you may have noticed our 3rd metric, Tutorials Created, jumped up during the course of the experiments. That my friend is the power of a Kera tutorial. We rolled out a Kera tutorial to welcome all new users right after sign up and walk them through building a tutorial. Yep, that’s almost a 3X increase in a desired user behaviour.
What interesting results have you found with your A/B tests? What have you found since you implemented a Kera tutorial? Email me at email@example.com or comment.
— Taige Zhang