Jul 22, 2014
1. Is your conversion rate good?
2. Are your engagement and retention rates good?
3. Are the new features and design changes you make actually improving your top goals?
4. How can you grow your user base?
5. What actually makes people love your app more?
Regardless of how great your analytics platform is, there is no way that it can give you answers to questions regarding quality. Analytics provide you with static data. Your conversion rate is x. Your 30-day retention is y. You have z number of MAUs. But what do these number mean? How can you leverage this information to improve your app and drive more revenue, users, or engagement?
The answer is A/B testing.
Industry benchmarks can help you get an understanding of what average conversion rates are, but even with something so concrete, I’ve had trouble finding good figures for native mobile apps. The problem is that the numbers vary a lot based on what your app’s goals are, what industry you’re in, and how your specific users behave. We’ve seen retail apps that have optimized up to a 1% conversion at which point the app is entirely profitable and some retail apps that start optimizing with a 1% conversion rate because a 1% conversion rate is not enough to cover their costs.
The only way to really understand what’s working and what’s not for your specific app is by iteratively A/B testing.
How to find the answers you’re looking for
First, start by defining your goals. Not all apps want to make money. Some apps care about engagement as a branding tool. Some apps prioritize domination of market share with their eye on later monetization. The most standard goals break down into transactions, engagement, retention, and customer satisfaction. Within each set, define the exact metrics that you track that match up to those goals. For transactions, the metrics can be items viewed, items added to cart, items bought, value of item bought, and all the conversions rates through the funnel.
Once you’ve defined your goals and metrics, form a hypothesis for improving those metrics. Have you heard users talk about a confusing check out process? Are you showing the right customers the right items? Is it easy for the customer to find the item they’re looking for? Will a slightly larger buy button attract more purchases?
Now create A/B tests to test that hypothesis. Change the copy on the checkout screen to see if users are less confused and buy more.
If you test a lot, you’ll start to see trends in what your users actually respond to. You might think that your current checkout conversion rate of 1.10% is good by industry standards, but one A/B test might reveal that making the buy button just a little bit bigger actually increased that conversion rate significantly like it did for KAYAK. Maybe 1.10% is ok for the industry, but your app can do much better.
On the flip side, maybe you thought your retention rates were on the low side and you saw that your competitor did something differently like Paktor did. A/B testing can show you that some changes that you were so sure of can actually hurt your app.
As the pros know, continuous A/B testing and a culture of experimentation (like that of Etsy) is the best way to constantly improve your app to optimize goals and metrics.
Executives face an ongoing challenge of creating amazing customer experiences while also keeping costs at bay. This tug of war between satisfying customers and reducing costs takes center stage in stores, branches, and call centers. In particular, organizations have typically...Read More
We hear from customers that planning out your second, third, and fourth A/B tests is one of the hardest things to. Many app managers have a first test in mind when they start experimenting and planning out a series of...Read More
There is a lot of literature about how to build great apps–what works, what doesn’t. But something I found while researching email marketing subject lines is that a lot of research is contradictory. Mailchimp’s data told me that subject lines...Read More