Early morning, dew still fresh on the grass, sunrise beautifully lighting a home office, aromatic steam rising from hot coffee, fog melting away from the world outside. And — unfortunately — the first flashes of carpal tunnel pain go pulsing through your wrist. Indeed, the consequences of a repeated routine can manifest themselves in painful ways. Furthermore, doing something in only one way limits personal growth. My suggestion? Maybe it’s time to stop exclusively typing all your content and start dictating some of it. Dictation is a vastly underused feature across all PC and smartphone platforms. Lawyers have been known…
We tend to make a big deal about leads in the marketing space, and not without good reason. Everything starts with leads. However, for software companies, the real goal is product adoption. We need people actively and consistently using our product. Regardless of our business model, success occurs when users experience that “aha” moment that takes our product from an experiment to a core part of their day-to-day work. So how do we move people from lead to product adopter? How to we give them that “aha” moment? Two words: Strategic Repetition Repetition is a POWERFUL psychological force. Studies have…
Recently, we got our entire team to actively research and contribute ideas for optimization on our website and ran multiple tests. This post is a narrative of what we did after.
Who Is This Post for?
This post will help SaaS growth-hackers, marketers, and optimization experts to predict the business value from a test.
The aim of this post is to not only share the tests we ran on our website, but also introduce a revenue-based framework that predicts the business impact of an A/B test and prioritizing on the basis of it.
Need for a Model
After we propelled our team to suggest ideas for testing, we had more than 30 hypotheses looking at us, but no distinct way of knowing which of these to take up first. Of course, there is a range of prioritizing frameworks available, but we particularly wanted to look at the ones that would directly impact our revenue.
This framework helped us project the potential impact on the revenue from each test. Here’s what we did: Step 1
We decided to identify high-impact pages and winnow the pages that were not as important for our business, that is, pages where no goal conversions take place. We looked at Google Analytics for pages with the:
Highest Amount of Traffic (We used “New Users” to nullify visits by existing customers.)
Highest Number of Goal Conversions (Goal conversion, which contributes to your overall business goal, is the main goal for your website. In our case, this meant all qualified lead-generating forms. A free trial or request a demo qualifies a visitor as a lead with a genuine interest in our product; or, as the industry popularly refers to it, a Marketing Qualified Lead.)
This gave us a list of pages which were high-value in terms of, either traffic generation or last touch before conversions.
We identified the following key pages:
Blog pages (All)
Our main objective was to project an estimated increase in the revenue due to a particular test. If your test increases the conversion rate by say 20%, what would this mean for your business and, in turn, the revenue?
This is how our marketing funnel looked like:
Note: You should use data from the recent 3–6 months, and the average (mean) of each step. This is to accurately reflect what to expect from your testing and be relevant to your business.
For each of the “Key Pages” we identified in the first step, we also dug out the corresponding numbers at each funnel stage. We’ve explained each stage of the funnel and how it is calculated:
a) Key Page Traffic: The total number of pageviews per Key Page (new users in our case). You can find the data in Google Analytics.
b) Total Conversions: The total number of leads generated from each particular page. If there is an additional qualification your company follows, source this data from your preferred CRM or Marketing Automation software. For example, at VWO, we use Clearbit to qualify our leads in Salesforce.
c) Opportunities:The total number of opportunities generated for your sales team. This data will be available in your CRM; make sure to count qualified opportunities only.
d) Customers: The total number of customers created in a month.
e) MRR (New):Or monthly recurring revenue, means revenue booked on a monthly basis; you can use this to estimate annual recurring revenue, or ARR, as well.
Now that we had all the numbers needed in our arsenal, I decided to calculate some more internal benchmarks. This gave us the performance of our marketing and/or sales funnel.
We computed the conversion rate of a particular page, using the following formula: Existing conversion rate = (Total Conversions Key Page Traffic); this is represented as %
The conversion of your leads into opportunities: (Opportunities ÷ Total conversions) × 100, represented as %
The conversion rate of opportunities into customers: (Customers ÷ Opportunities) × 100, represented as %
The average revenue per user or ARPU: Total MRR ÷ Total number of paying customers
Now all you have to do is to impute these numbers in this template. The model uses all of that data and projects how much revenue increase or decrease you can estimate based on your test results. This estimate can give you a good idea of where to begin or prioritize your testing.
Step 4 (Optional)
This is where it may get tricky. At VWO, we sell both Enterprise plans and Standard plans. So to be fair, we must estimate each cohort with separate data and individual conversion rates. For example, Opportunity creation % for an Enterprise plan may be lower, but a Standard plan is easier to convert. You may want to decide what type of plan do you want to focus on. We, for instance, used website traffic and Alexa rank as the benchmark for lead qualification. We attributed more value to the leads that came in through key pages and prioritized them. This led us to the next step, which is the qualification rate of the said lead of high value. This rate may be in the range 30–50%, depending on your definition. It was interesting to note that each page had a different qualification rate. For example, we get better quality leads from our Request a demo page than we do from our free trial or blog post page.
After we had the model in place, we played around with the increase or decrease in our conversion rates. This was to identify what would be our best optimization opportunities?
The free trial pages and the home page were among the high-priority pages, in terms of the impact of revenue. (Unfortunately, I can’t share the exact numbers with you.) We first looked at the hypotheses on the free trial page:
Test 1 – Free Trial Page
Our hypothesis was “Illustrating VWO features and social proof on the free trial page will compel users to sign up for the free trial.” Here is a screenshot of what it looks like in VWO.
Bonus tip: VWO has recently launched a new capability called PLAN that lets you manage and prioritize your testing hypotheses. To learn more about this capability, visit the VWO evolution page.
This is what the control looked like:
Our heatmap data also showed a lot of users clicking the features page after accessing the free trial page.
Screenshot of heatmap data:
We created a variation which included the features we offer to solve this issue. Here’s a screenshot of the same.
We came up with a more targeted copy and changed the existing CTA to Request A Demo. Here is what the variation looked like:
We also wanted to change our positioning due to our foray into Conversion Optimization. The results from this test were that our variation beat the control and had more than 31% improvement in the conversion rate. Based on the first example, we have already implemented the new free-trial page as our main free-trial page now.Based on the second test, we updated our current home page. All in all, this model helped us correctly predict the best optimization opportunities, make our testing better, and more strategically aligned to business goals. Let me know your experience with this model and how you go about testing. Would love to hear your feedback on this!
What separates a Hermès Birkin bag from a high-quality leather handbag you could buy anywhere? A label, a fancy charm, and about $22,000. But unlike the tangible qualities of a purchase, like the grade of leather used or the fact that the utterly useless bag charm is 14 karat gold, the perception of value is what really separates one bag from the other. One bag contains social cachet and the ability to draw envy from other women – intangible benefits so valuable; it justifies the raised price. The nameless bag, however, has its own set of benefits for a different…
Have you ever doubted Google? When it comes to the keyword ranking accuracy, we can be skeptical about rank tracker tools we use or SEOs we hired. But when we check rankings manually, we trust our eyes and Google. But you shouldn’t be so careless. Google is clever and agile. They have a massive list of factors that affect the search results they display for you. Even if you see your website in the Number 1 position, it doesn’t mean you really are on top of the world. Your customers may see a very different Top 10. Fortunately, you can…
Modern customers scour websites and research products they’re thinking of buying before making their actual purchase. When customers are 60% to 80% of the way down the funnel before they talk to anyone at your business, you can’t rely on traditional methods to generate loyalty. At the same time, fewer and fewer clients remain loyal to one specific brand. Loyal customers are profitable customers: repeat customers are cheaper to market to, spend more, and make more frequent purchases. Yet, only 27% of initial sales go on to become repeat customers. Companies need to invest in building loyalty among their customers….
Seriously. A quick google will show you that Unbounce, QuickSprout, Moz, Qualaroo, Hubspot, Wordstream, Optimizely, CrazyEgg, VWO (and countless others), have been writing tips and guides on how to optimize your landing pages for years.
Not to mention the several posts we have already published on the WiderFunnel blog since 2008.
And yet. This conversation is so not over.
Warning: If your landing page optimization goals are short-term, or completely focused on conversion rate lift, this post may be a waste of your time. If your goal is to continuously have the best-performing landing pages on the internet, keep reading.
Marketers are funnelling more and more money into paid advertising, especially as Google allocates more and more SERP space to ads.
In fact, as an industry, we are spending upwards of $92 billion annually on paid search advertising alone.
And it’s not just search advertising that is seeing an uptick in spend, but social media advertising too.
It makes sense that marketers are still obsessing over their landing page conversion rates: this traffic is costly and curated. These are visitors that you have sought out, that share characteristics with your target market. It is extremely important that these visitors convert!
But, there comes a time in every optimizer’s life, when they face the cruel reality of diminishing returns. You’ve tested your landing page hero image. You’ve tested your value proposition. You’ve tested your form placement. And now, you’ve hit a plateau.
So, what next? What’s beyond the tips and guides? What is beyond the optimization basics?
1) Put on your customer’s shoes.
First things first: Let’s do a quick sanity check.
When you test your hero image, or your form placement, are you testing based on tips and recommended best practices? Or, are you testing based on a specific theory you have about your page visitors?
Tips and best practices are a fine place to start, but the insight behind why those tactics work (or don’t work) for your visitors is where you find longevity.
The best way to improve experiences for your visitors is to think from their perspective. And the best way to do that is to use frameworks, and framework thinking, to get robust insights about your customers.
– Chris Goward, Founder & CEO, WiderFunnel
Laying the foundation
It’s very difficult to think from a different perspective. This is true in marketing as much as it is in life. And it’s why conversion optimization and A/B testing have become so vital: We no longer have to guess at what our visitors want, but can test instead!
That said, a test requires a hypothesis. And a legitimate hypothesis requires a legitimate attempt to understand your visitor’s unique perspective.
To respond to this need for understanding, WiderFunnel developed the LIFT Model® in 2008: our foundational framework for identifying potential barriers to conversion on a page from the perspective of the page visitor.
Get optimization ideas with the LIFT poster!
Get the LIFT Model poster, and challenge yourself to keep your visitor’s perspective in mind at all times. Use the six conversion factors to analyze your pages, and get optimization ideas!
By entering your email, you’ll receive bi-weekly WiderFunnel Blog updates and other resources to help you become an optimization champion.
The LIFT Model attempts to capture the idea of competing forces in communication, narrowing them down to the most salient aspects of communication that marketers should consider.
I wanted to apply the principles of Relevance, Clarity, Distraction, Urgency and Anxiety to what we were delivering to the industry and not just to our clients. And the LIFT Model is a part of that: making something as simple as possible but no simpler.
– Chris Goward
When you look at your page through a lens like the LIFT Model, you are forced to question your assumptions about what your visitors want when they land on your page.
You may love an interactive element, but is it distracting your visitors? You may think that your copy creates urgency, but is it really creating anxiety?
If you are an experienced optimizer, you may have already incorporated a framework like the LIFT Model into your optimization program. But, after you have analyzed the same page multiple times, how do you continue to come up with new ideas?
Here are a few tips from the WiderFunnel Strategy team:
Bring in fresh eyes from another team to look at and use your page
User test, to watch and record how actual users are using your page
Sneak a peek at your competitors’ landing pages: Is there something they’re doing that might be worth testing on your site?
Do your page analyses as a team: many heads are better than one
You should always err on the side of “This customer experience could be better.” After all, it’s a customer-centric world, and we’re just marketing in it.
2) Look past the conversion rate.
“Landing page optimization”, like “conversion rate optimization”, is a limiting term. Yes, on-page optimization is key, but mature organizations view “landing page optimization” as the optimization of the entire experience, from first to last customer touchpoint.
Landing pages are only one element of a stellar, high-converting marketing campaign. And focusing all of your attention on optimizing only one element is just foolish.
From testing your featured ads, to tracking click-through rates of Thank You emails, to tracking returns and refunds, to tracking leads through the rest of the funnel, a better-performing landing page is about much more than on-page conversion rate lift.
An example is worth 1,000 words
One of our clients is a company that provides an online consumer information service—visitors type in a question and get an Expert answer. One of the first zones (areas on their website) that we focused on was a particular landing page funnel.
Visitors come from an ad, and land on page where they can ask their question. They then enter a 4-step funnel: Step 1: Ask the question > Step 2: Add more information > Step 3: Pick an Expert > Step 4: Get an answer (aka the checkout page)
Our primary goal was to increase transactions, meaning we had to move visitors all the way through the funnel. But we were also tracking refunds and chargebacks, as well as revenue per visitor.
In this experiment, we focused on the value proposition statements. The control landing page exclaimed, “A new question is answered every 9 seconds!“. Our Strategy team had determined (through user testing) that “speed of answers” was the 8th most valuable element of the service for customers, and that “peace of mind / reassurance” was the most important.
So, they tested two variations, featuring two different value proposition statements meant to create more peace of mind for visitors:
“Join 6,152,585 satisfied customers who got professional answers…”
“Connect One on One with an Expert who will answer your question”
Both of these variations ultimately increased transactions, by 6% and 9.4% respectively. But! We also saw large decreases in refunds and chargebacks with both variations, and large increases in net revenue per visitor for both variations.
By following visitors past the actual conversion, we were able to confirm that these initial statements set an impactful tone: visitors were more satisfied with their purchases, and comfortable investing more in their expert responses.
3) Consider the big picture.
As you think of landing page optimization as the optimization of a complete digital experience, you should also think of landing page optimization as part of your overall digital optimization strategy.
When you discover an insight about visitors to your product page, feed it into a test on your landing page. When you discover an insight about visitor behavior on your landing page, feed it into a test on your website.
It’s true that your landing pages most likely cater to specific visitor segments, who may behave totally differently than your organic visitors. But, it is also true that landing page wins may be overall wins.
Plus, landing page insights can be very valuable, because they are often new visitor insights. And now, a little more advice from Chris Goward, optimization guru:
“Your best opportunities for testing your value proposition are with first impression visitors. These are usually new visitors to your high traffic landing pages or your home page […]
By split testing your alternative value propositions with new visitors, you’ll reduce your exposure to existing customers or prospects who are already in the consideration phase. New prospects have a blank canvas for you to present your message variations and see what sticks.
Then, from the learning gained on landing pages, you can validate insights with other target audience groups and with your customers to leverage the learning company-wide.
Landing page testing can do more than just improve conversion rates on landing pages. When done strategically, it can deliver powerful, high-leverage marketing insights.”
Just because your landing pages are separate from your website, does not mean that your landing page optimization should be separate from your other optimization efforts. A landing page is just another zone, and you are free to (and should) use insights from one zone when testing on another zone.
4) Go deeper, explore further.
A lot of marketers talk about landing page design: how to build the right landing page, where to position each element, what color scheme and imagery to use, etc.
But when you dig into the why behind your test results, it’s like breaking into a piñata of possibilities, or opening a box of idea confetti.
Why do your 16-25 year old, mobile users respond so favorably to a one-minute video testimonial from a past-purchaser? Do they respond better to this indicator of social proof than another?
Why do your visitors prefer one landing page under normal circumstances, and a different version when external factors change (like a holiday, or a crisis)? Can you leverage this insight throughout your website?
Why does one type of urgency phrasing work, while slightly different wording decreases conversions on your page? Are your visitors sensitive to overly salesy copy? Why or why not?
For many marketers, personalized landing pages are becoming more normal. And personalization opens the door to even more potential customer insights. Assuming you already have visitor segments, you should test the personalized experiences on your landing pages.
For example, imagine you have started using your visitors’ first names in the hero banner of your landing page. Have you validated that this personalized experience is more effective than another, like moving a social proof indicator above the fold? Both can be deemed personalization, but they tap into very different motivations.
From psychological principles, to validating your personalized experiences, the possibilities for testing on your landing pages are endless.
Just keep testing, Dory-style
Your landing page(s) will never be “optimized”. That is the beauty and cruelty of optimization: we are always chasing unattainable perfection.
But your landing pages can definitely be better than they are now. Even if you have a high-converting page, even if your page is listed by Hubspot as one of the 16 best designed landing pages, even if you’ve followed all of the rules…your landing page can be better.
Because I’m not just talking about conversions, I’m talking about your entire customer experience. If you give them the opportunity, your new users will tell you what’s wrong with your page.
They’ll tell you where it is unclear and where it is distracting.
Regression testing is one of the most time-consuming tasks when developing a mobile Android app. Using myMail as a case study, I’d like to share my experience and advice on how to build a flexible and extensible automated testing system for Android smartphones — from scratch.
The team at myMail currently uses about 60 devices for regression testing. On average, we test roughly 20 builds daily. Approximately 600 UI tests and more than 3,500 unit tests are run on each build.
A type of hypothesis testing where multiple variables are tested simultaneously to determine how the variables and combinations of variables influence the output. If several different variables (factors) are believed to influence the results or output of a test, multivariate testing can be used to test all of these factors at once. Using an organized matrix which includes each potential combination of factors, the effects of each factor on the result can be determined. Also known as Design of Experiments (DOE), the first multivariate testing was performed in 1754 by Scottish physician James Lind as a means of identifying (or…
The one piece of information about your website that matters most is: how do your users interact with it? It’s easy to find out what they think about it: you can just ask them. But it’s very hard to find out how real users go about navigating your website. The thing is, that piece of knowledge is absolutely crucial. It’s the acid test for every design decision, because it’s what determines whether a visitor converts or bounces. So, we made Crazy Egg Recordings. Soon, Recordings will let you see multiple sessions of different visitors clicking, scrolling, and navigating throughout your…