You’re designing a landing page for your Real Estate client, and you turn to “best practice” advice articles to help guide the way.
But there’s a nagging voice at the back of your mind:
Does this “best practice” advice apply indiscriminately to my industry? Does this author really know anything about my audience at all?
“Best practices” become “better practices” when they are industry-specific.
When our design team was recently wireframing new landing page templates for the Unbounce builder, they set out to create industry-specific templates that addressed this truth: different audiences belonging to different industries behave differently. They have different pains, different motivators and different disincentives.
Firm believers that data needs to inform design, our design team sourced their research in two key areas:
Data from the Unbounce Conversion Benchmark Report: The report includes average conversion rates for 10 popular industries, as well as Machine Learning-powered recommendations around reading ease, page length, emotion and sentiment.
High-converting customer landing pages: Our designers looked at the top 10 highest-converting Unbounce landing pages in those industries, and analyzed common design and copy elements across the pages.
Our design team then combined insight from these two key areas of research to build out content and design requirements for the best possible landing page template for each of the 10 industries.
One of these industries was Real Estate, and now we want to share their findings with you.
See a breakdown of their process for designing the Real Estate page template at the bottom of this post, or read on for their key findings about what converts in the Real Estate industry.
Which copy elements convert best in the Real Estate industry?
The data scientists and conversion rate optimizers who put together the Unbounce Conversion Benchmark Report found that for Real Estate lead capture landing pages, short n’ sweet is better: overall, they saw 33% lower conversion rates for longer landing pages.
This chart shows how the word count relates to conversion rates for the Real Estate vertical. On the x-axis we have word count — on the y-axis, conversion rate.
This was consistent with what the design team saw across high-converting Unbounce customer landing pages in Real Estate: pages were relatively short with concise, to-the-point copy.
The Unbounce Convert Benchmark Report also revealed that in the Real Estate vertical, prospects want simple and accessible language. The predicted conversion rate for a landing page written with 6th grade level language was nearly double that of a page written at the university level.
This chart shows how conversion rates trend with changes to reading ease for the Real Estate Industry. On the x-axis we have the Flesch Reading Ease score — on the y-axis, conversion rate.
According to the Unbounce Conversion Benchmark Report, 41.6% of marketers in the Real Estate industry have at least one page that converts at less than 1.3% (in the 25th percentile for this industry). Download the report here to see the full data story on Real Estate and get recommendations for copy, sentiment, page length and more for nine additional industries.
The Unbounce Conversion Benchmark Report used an Emotion Lexicon and Machine Learning to determine whether words associated with eight basic emotions (anger, anticipation, disgust, fear, joy, sadness, surprise and trust) affected overall conversion rates.
While these emotions did not seem to dramatically correlate with conversion rate in the Real Estate vertical, fear-based language was the exception. We saw a slight negative trend for pages using more fear-inducing terms:
This chart shows how the percentage of copy that evokes fear is related to conversion rates for the Real Estate vertical. On the x-axis we have the percentage of copy that uses words related to fear — on the y-axis, conversion rate.
If more than half a percent of your copy evokes feelings of fear, you could be hurting your conversion rates.
Here are some words commonly associated with fear on Real Estate lead capture landing pages: highest, fire, problem, watch, change, confidence, mortgage, eviction, cash, risk…
When our designers looked at the top 10 highest-converting Unbounce customer landing pages in the Real Estate vertical, they took a close look at the calls to action and found that:
Every page provided a detailed description of the offer
Almost all had a “request a call back” or “call us” option (other CTAs included “get more info,” “apply now” and “get the pricelist”)
Most did an excellent job of including button copy that reinforces what prospects get by submitting the form
If you use a “call us” CTA on your landing pages, make sure you try out our CallRail integration. This will help you track which calls are a result of your paid spend and landing pages!
Here are some examples of the forms and calls to action on some of our highest-converting Real Estate lead capture landing pages:
The usual suspects (benefits, social proof, UVP…)
Without much exception, the pages featured a lot of the copywriting elements that one would expect to see on any high-converting landing page (regardless of vertical):
Detailed benefits listed as bullet points
A tagline that reinforces the unique value proposition or speaks to a pain point:
And not surprisingly, testimonials. One page went above and beyond with a video testimonial:
Which design elements convert best in the Real Estate industry?
The highest-converting Real Estate landing pages included lots of imagery:
Beautiful hero shots of the interior and exterior of properties
Full-width photography backgrounds
Our designers also studied other design features as basic guidelines for the template they were then going to create.
While these specifics are meant to be taken with a grain of salt (you may already have brand colors and fonts!) they could serve as a good starting point if you’re starting completely from scratch and want to know what others are up to.
Many of the high-converting pages had:
Palettes of deep navy and forest green
Orange (contrasting) call to action buttons
The highest-converting landing pages in the Real Estate industry sit at 11.2%. If your Real Estate page converts at over 8.7%, you’re beating 90% of your competitors’ pages. See the breakdown of median and top conversion rates (and where you stand!) via the Unbounce Conversion Benchmark Report.
Behold, the template our designers created
After synthesizing all that research, our Senior Art Director Cesar Martínez took to his studio (okay, his desk), and drafted up this beautiful Real Estate landing page template:
Not only is the template beautiful, it was created by analyzing actual data: what makes for a high-performing landing page in the Real Estate industry via the Unbounce Benchmark Report and high-converting customer pages.
Footnote: The design process
Curious about the process our designers used to develop this data-backed Real Estate landing page template? Here are the steps they followed:
For the 10 highest-converting customer landing pages, they analyzed all common elements (such as form, what type of information is collected, what type of offer, if there are any testimonials, etc). This allowed them to build their content requirements.
They referred to the word count recommendations in the Unbounce Conversion Benchmark Report and designed for that word count limit.
They referred to reading ease level recommendations for that specific industry from the Benchmark Report and shared the information with their copywriter.
They sketched out a rough idea of their potential landing page template.
They selected typography and colors relevant to the industry based on what was popular in the 10 examples.
They named their imaginary company in the industry and sketched out some potential logos. They picked photography built out a moodboard.
That helped them gather all the information they needed to build out their template!
There’s no lack of data to suggest how visual-oriented we are as humans. For instance, “90 percent of information transmitted to the brain is visual, and visuals are processed 60,000x faster in the brain than text.” Or this: “65 percent of people are visual learners, and one of the best ways to drive messages home is through visual content.” This data helps explain why visual marketing has really exploded recently, and visual-centric content such as infographics are so popular. Seeing is one of our primary senses by which we intake information and understand the world. Basically, it’s a big deal….
So you’ve started a business, and you’re trying to make sure everything goes perfectly. This obviously means you’ll want to be prepared to drive sales on your website so you can generate revenue. And for this, you’re going to need an effective sales funnel that will lay out a strategy for attracting, engaging, and converting your audience. Build a Strong Foundation on Audience Insights A good understanding of your audience is crucial if you want to know how to enhance their purchase journey and ensure that they turn into paying customers. That’s why even before you start building your sales…
I’m proud to announce the launch of next evolution of VWO. This version has been a long time coming, so the details of what we are launching are going to be long but exciting.
When VWO was launched 7 years ago, we changed the way marketers did A/B testing by making it stupidly simple to set up and launch A/B tests. We cut down on the time and effort required to launch a test, from weeks to minutes. Since then, more than a hundred thousand A/B testing campaigns have been launched in VWO by thousands of customers. We have numerous case studies where our customers have seen double-digit improvements in metrics that matter to their businesses (average order value, signup rate, and so on). The success of web testing (A/B, split-URL, and multivariate testing) for a range of brands has established that it works and any brand not doing testing is a dinosaur still stuck in subjective debates where the highest paid person’s opinion (HiPPO) usually wins.
While A/B testing works, the mystery of which A/B tests to do still remains unsolved. Yes, you can throw darts on the board randomly and some may strike at the center, giving you a significant improvement in your funnel metrics. But it’s a fact that successful A/B testing is still a dark art. Over the last 7 years, we’ve customers across the entire spectrum of maturity.
A/B Testing Maturity
Stage I – No A/B Tests
Stage II – Occasional A/B Testing
Stage III – Process-oriented A/B Testing
Stage IV – A/B Testing is a part of the company culture
VWO, when it first came out in 2010, caused online marketers to move from Stage I to Stage II. What we’re launching now will lead marketers to move from Stage II to Stage III. (Stage IV refers to companies which live and breathe A/B testing not only in web functioning but also in every other thing they do.)
What Is Process-Oriented A/B Testing, and Why Should You Care?
As the name says, process-oriented A/B testing means that A/B testing is no longer “let’s try to change button colors and see what happens.” Many companies new to A/B tests start by testing all low-hanging fruits and get quick wins. These early wins may seem great because these show the potential of A/B testing. However, many of these companies then hit a roadblock—what to do next? At this stage, either they abandon A/B testing completely or realize, through trial-and-error, that their next step should be to approach A/B testing in a process-oriented way.
Running your A/B testing program like a process means taking the following steps:
Set a target to increase the conversion rate by a specific percentage and time, and select the tests that are most likely to deliver.
Generate web testing ideas by doing a thorough research on your audience’s behavior and determining the causes of funnel drop-offs or abandonment.
Implement a proper prioritization framework to select tests that have best trade-offs of effort versus results.
Ensure that your testing velocity remains intact (or increases) week after week. (After all, most tests fail, so more testing equals more chances of success.)
The process-oriented A/B testing platform looks something like this:
This is a process that our most successful customers use. We observed that customers who followed a process for their A/B testing and conversion optimization efforts were 2x more likely to increase their conversion rate, compared to the ones who did only tactical testing.
When we compared why some customers approach A/B testing and conversion optimization as a process while others did on-and-off testing, we noted that:
Our customers cited lack of resources for doing tests frequently.
Many customers were bogged down by using multiple tools needed to support the entire process (for tracking funnels, spreadsheets or docs for managing test ideas, session recordings, surveys, and of course VWO for testing).
Many customers said that they weren’t aware that A/B testing could be approached in a process-oriented way. Except for a few digital agencies, nobody was showing them a better way.
After numerous conversations about what we had to do, it was clear. We had to build a platform that would empower our customers to run A/B tests efficiently, all at one place and through one integrated platform. The same platform will now provide a foundation for Stage II companies—that do A/B testing occasionally—to approach conversion optimization in a process-driven manner.
The New VWO: From a Web Testing Tool to a Conversion Optimization Platform
Here’s what we’re launching:
As you’d notice, it’s a pretty big leap from what we were earlier (web testing tool) to a conversion optimization platform. Let me unpack the capabilities of the new VWO and what it would mean for you.
The VWO Conversion Optimization platform brings everything you need to run conversion optimization in your organization on one platform. It does this by providing the following capabilities in an integrated fashion:
This capability allows you to measure funnels and goals on your (desktop and mobile) website, right there in the VWO dashboard.
This capability provides a single source of truth for your funnel conversion rate so that you can easily decide which part of the funnel you should “fix” next and for which segment of visitors should you be fixing the funnel.
This capability includes tools such as session recordings (see playback of visitors interacting with your website), on-site surveys, heatmaps, clickmaps, scrollmaps, and form analytics.
This capability lets you dig deeper into visitor behavior (from multiple angles) and find out what’s causing some visitors to abandon your website or landing pages.
This capability provides you and your team one dashboard to record and prioritize your observations, ideas, and hypotheses; and help you select ideas for your next web tests.
Imagine this to be like Trello or Jira for your conversion optimization.
This capability has been our forte, and we have enhanced it in numerous ways with this release.
This capability lets you deploy your winning tests or provide a specific experience to your target segment.
This capability also allows you to slice-and-dice your user segments and improve the conversion rate, for different segments by targeting a specific experience to them.
You would imagine that building all this technology would have been hard. Well, “hard is an understatement,” as I can imagine VWO’s engineering team saying. Some of the technology we got from our acquisition of Navilytics in 2015, but most of it is built in-house. All VWO teams worked hard for many months, because we have a deep conviction that one integrated platform has more advantages for marketers than juggling with different tools.
The value of an integrated platform is a bet we are taking and we are confident of demonstrating that value to marketers all over the world. When you take the new VWO for a test drive (see below, on how to get access), you will see the power of integration first hand but if you need a few examples of what this integration can do, see the following use cases:
You set up the funnel in the VWO Track capability and discover that for a specific segment, your checkout rate is much below average. What’s causing that drop? You immediately use the VWO Analyze capability to look at session recordings for that segment, look at form analytics, and fire up an on-site survey to ask why that specific user segment is abandoning sign-up.
You immediately use the VWO Analyze capability to look at session recordings for that segment, look at form analytics, and fire up an on-site survey to ask why that specific visitor segment is abandoning sign-up.
While looking at the form analytics in VWO Analyze, you realize that the checkout form accepts a US phone number only and the segment from UK has a low conversion rate. But then you see that you received a few responses from the survey that you ran by using VWO Analyze. The most common response is that your UK visitors are not sure whether you ship to UK or not. Now you have another insight here. But now your problem is which one to test. Plus, you also have many other such ideas from your last week’s idea brainstorm with your team. You are flooded with ideas, but don’t want to lose these 2 observations. You then remember the VWO Plan capability and use the form tool and survey to record the 2 observations in VWO Plan. (This is in addition to your team continuously using the VWO Chrome plug-in to record ideas from your and your competitors’ websites.)
You fire up VWO Plan and prioritize your ideas by attaching the expected effort versus expected results. When you are done, you realize that “Lack of shipping-to-UK information while we look like the US website” is the best among other ideas to test. From that observation, you get into the VWO’s Test capability where you set up a quick test that takes you 10 minutes. All you need to do is to mention “We ship to UK free of cost” when the visitor is from UK. (An added advantage of this link is that all your observations and notes regarding “Lack of shipping-to-UK information while we look like a US website” get attached to this test so that you can revisit this anytime in future. Your hypothesis was wrong. It will be great to have this context while looking at results, wouldn’t it?)
Your VWO test concludes, and you have a winner. You know that your IT team takes time to implement it at the back end, so you use VWO Target to make the winning version live for all UK visitors. After a week or so, you go back to VWO Track and see an increase in the checkout rate for UK visitors. You are happy, tell your colleagues and boss, and they are happy too.
You repeat this process over months and accumulate your wins, losses, and insights at one place, that is, the VWO CRO. As a result, your success rate increases with each passing month.
Early Feedback on the VWO Conversion Optimization Platform
We did not develop the new version of VWO in isolation. Various components of the VWO platform were co-developed and tested with customers. Their feedback was instrumental in developing the entire platform in an integrated fashion. Here’s some early feedback:
Wait, There’s More: VWO Services
For our customers who do not have sufficient resources for implementing and using the VWO platform, we have agency partners and an internal team who supplement the efforts of our customers’ teams. We provide design, development, and CRO consultants through our newest offering—VWO Services. So, you can depend on us from software to services for managing your conversion optimization and testing program end to end. Learn more about VWO Services.
FAQs: Conversion Optimization Platform
Q: How much does the VWO conversion optimization platform cost? A: We offer 3 plans for the VWO conversion optimization platform, with each plan tailored to your team’s needs and maturity of Testing. You can look at Pricing on our website or contact sales for a detailed demo and consultation.
Q: I’m an existing customer. How and when do I get access to the new version? A: As an existing customer, we have added a free 30-day evaluation in your account that you can activate at any point in time.
Q: I want to try out the new VWO Conversion Optimization platform. How do I get a trial version? A: If you are trying VWO for the first time, then you can register for a free trial on our website and get started.
Q. I am an existing customer. Does this mean prices have increased for me? A. Not exactly. We now have 2 products—Web Testing (A/B, Split URL, and Multivariate testing) and Conversion Optimization platform. As an existing customer, you can continue to use the Web Testing product and see absolutely no change in prices and your plans. However, if you wish to try out the new capabilities of the Conversion Optimization platform, then you would have to select one of the newer plans and pricing.
Q: How do I reach out to your sales team? A: You can reach us at email@example.com or visit our contact page for any sales related query.
If you have more questions, you can refer to the FAQs here or reach out to us at firstname.lastname@example.org.
Get Started with VWO Conversion Optimization now
The best way to get started with VWO Conversion Optimization is to explore our new website and take a tour of the platform here: VWO After you are convinced that VWO will help your team move faster with funnel and conversion optimization efforts, sign up for a free trial or give our sales team a shout-out.
For feedback, comments, or questions, you can also reach out to me (the CEO) at email@example.com.
Watch this video: Moz’s Whiteboard Friday – The Perfect Blog Post Length and Publishing Frequency is B?!!$#÷x Information has been floating around for a while that suggests you will rank better in search engines and / or get more traffic by: Blogging more frequently Writing longer blog posts (higher word counts) While we’ve noticed that both of these have seemed to be true in the past, at least anecdotally, this appears to no longer be the case. Myth busted? There’s Nothing Wrong With Posting Frequently You’re allowed to post as often as you’d like. There is certainly no rule against…
You’re going to start your very first influencer marketing campaign, and you want to make sure it’s a success. Or maybe you’ve executed a few campaigns before, and you want the next one to deliver better results. Either way, knowing how to manage your campaign effectively is crucial if you want influencer marketing to work for you. While it’s not always easy to manage influencer marketing campaigns, you’ll find it much easier if you remember the following steps: 1. Set Up a Goal You should always start with a defined goal, regardless of whether it’s influencer marketing or any other…
As a travel marketer or agency marketer servicing the travel industry, you have a tricky gig. You need to convince your prospects to spend thousands of dollars and precious vacation time.
Meanwhile, your prospects are increasingly wary of the legitimacy of your offers (thanks a lot, Fyre Fest).
Here’s to hoping your vacation is memorable, but not in a meme-worthy kind of way.
Your challenge then is to effectively convey trust on your travel landing pages. Doing so can help ease prospects’ conversion anxiety, resulting in more travel leads and sales for your business.
The importance of trust on your travel landing pages
We often talk about the importance of trust and credibility on your landing pages — this isn’t a new idea.
But for some industries, a lack of trust can have hugely detrimental effects on conversion rates.
In a recent analysis of 74,551,421 visitors to 64,284 lead generation landing pages created in the Unbounce platform, data scientists found that travel landing pages can realistically achieve conversion rates of at least 12%. Even more impressive is within the travel and tourism industry, the very best pages convert over 25% of their visitors (schwing!).
Notice the dramatic conversion rate difference between percentiles? If you’re part of that percentile getting 2.1% or lower conversion rates, your pages have lots of room for improvement. Image via the Unbounce Conversion Benchmark Report.
If you’re not hitting these benchmarks, it might be time to take a hard look at your marketing and ask yourself if you’ve done enough to make your prospects trust you.
And don’t worry if your answer is “No” or “I’m not sure.” We’ve compiled four data-backed ways to boost trust on your travel landing page. Use them as a jumping off point for your optimization efforts.
1. Bolster your copy with trust words
Using an Emotion Lexicon to analyze copy, Unbounce data scientists found evidence that visitors to travel landing pages have slight concerns about the legitimacy of the offers.
However, they also found that using at least 7% (and up to 10%) of your copy to establish trust could result in conversion rates that are up to 20% better.
(Keep in mind, though, that these words were generated by an algorithm and should still be applied using common sense. Just adding the word “spa” to your page — especially if you don’t offer spa services — is not going to increase your conversions.)
The travel experts at Nordic Visitor do a great job of using trust words to build confidence on their Iceland site. It’s not a landing page per se, but the same principles apply.
“Team,” “planning,” “provide” and “personal” are all words found to positively convey trustworthiness. Adding these and other trust words to your copy could be the subconscious nudge your prospects need to convert.
Take stock of the trust words you’re using in your marketing, and particularly on your landing pages. If they’re looking a little sparse, test out using confidence-building words to describe destinations in detail.
2. Cut copy that brings up emotions of fear and anger
Just as trust words can drastically improve your conversion rates, words that subconsciously trigger fear or anger will have a negative impact on travel landing page conversion rates.
In fact, Unbounce data scientists found that if even 1% of page copy reminds your visitors of feelings of anger or fear, you could be seeing up to 25% lower conversion rates.
“Feeling endless despair this Canadian winter? Warm yourself up with a limited-time-only vacation in the hot Mojave desert.”
“Escape the Canadian winter at a five-star award-winning vacation rental in sunny California.”
Get even more industry-specific emotion and sentiment copy suggestions
Download the Unbounce Conversion Benchmark Report to see how emotion and sentiment may be impacting conversion rates in your industry.
By entering your email you expressly consent to receive other resources to help you improve your conversion rates.
3. Leverage social proof to build visitor trust
Persuading your prospects to put their trust in you is tricky business, and it’s even trickier when it comes to travel, because they’re likely working with a tight budget and only a few weeks of precious vacation. They don’t want to take a leap of faith — they want a sure thing.
A proven strategy for easing prospect anxiety is to use social proof. It’s the “everybody’s doing it” mentality that helps convince your prospects to convert.
When you let your satisfied customers sing your praises, your credibility goes through the roof. Including testimonials on your travel landing page can have a positive impact on how trustworthy your prospects perceive you to be, but not all testimonials are created equal.
To best enhance your chance of conversion, heed the following testimonial commandments:
Include a photo of the person
Avoid hyperbole (i.e., This pedicure literally saved my life!)
Choose testimonials that demonstrate the transformative effect of your product or service on the lives of your users
Nordic Visitor takes it one step further with a video testimonial from several happy customers:
Don’t tell your prospects how great you are, show them with real live, happy customers.
Similar to testimonials, including reviews on your travel landing page can help convey trust to your prospects.
The luxury travel designers of Jacada Travel have embedded reviews from Trustpilot, a reputable online review community, directly into their landing page.
If you recall, the word “award” is associated with trust on travel landing pages. So if your company or client has won any reputable awards, be sure to flaunt ‘em.
Tour guide company Kensington Tours not only includes several trust seals on their travel landing page, they also mention in their Adwords ad that they’re a National Geographic award winner.
Highlight awards strategically to build confidence in your offers.
4. Security measures
Persuasive trust-infused copy and social proof are wonderful, but when you’re collecting travel leads and even money, you need to assure your prospects that their data and money is safe.
There are many ways to do this, but the two most impactful strategies are to enable SSL (Secure Sockets Layer) and to include trust seals.
SSL creates an encrypted link between your landing pages and your visitor’s browser. It’s identified by the little lock icon and the “https” (vs. http) in the top left-hand side of your browser search bar.
Enabling SSL on all your web properties (but especially on your lead gen and ecommerce landing pages) assures your visitors that they’re not at risk of being hacked.
Nordic Visitor nails it yet again with a trust seal from GeoTrust and a TripAdvisor Certificate of Excellence 2017, further reinforcing their credibility.
All aboard the Conversion Cruise
A lack of trust in any industry can hurt conversion rates, but in the travel industry the stakes are extra high.
Fortunately, this means the opportunities to improve your conversion rates are plenty. And if you nail the whole trust thing down, you could be seeing some of the highest conversion rates across any industry.
Leveraging a combo of effective copy, social proof and security measures, you can make your prospects forget about the stress associated with booking a vacation. Skip that trip to Poor Conversions-ville and instead put your feet up with a Mai Tai in hand on the Conversion Cruise.
For even more data-backed conversion insights in the travel industry, or for insights into industries such as health, finance, higher education and more, download the Unbounce Conversion Benchmark Report.
Get data-backed conversion insights across 10 popular industries
Download the Unbounce Conversion Benchmark Report to see how your conversion rates stack up against the competition — and how to improve them.
By entering your email you expressly consent to receive other resources to help you improve your conversion rates.
When it comes to increasing conversion rates, few strategies are more effective than the implementation of landing pages. Yet, these crucial linchpins to the optimization process are often rushed or overlooked completely in the grand scheme of marketing. Here at Crazy Egg, we believe it’s past time to give these hard-working pages a little more attention, which is why we’ve created this complete guide to landing page optimization. Even if you consider yourself a landing page pro, you’ll want to read this guide to make sure your pages are on track and converting as well as they should be. Why…
This Summer, WiderFunnel participated in several virtual events. And each one, from full-day summit to hour-long webinar, ended with a TON of great questions from all of you.
So, here is a compilation of 29 of your top conversion optimization questions. From how to get executive buy-in for experimentation, to the impact of CRO on SEO, to the power (or lack thereof) of personalization, you asked, and we answered.
As you’ll notice, many experts and thought-leaders weighed in on your questions, including:
Q: What do you see as the most common mistake people make that has a negative effect on website conversion?
Chris Goward: I think the most common mistake is a strategic one, where marketers don’t create or ensure they have a great process and team in place before starting experimentation.
I’ve seen many teams get really excited about conversion optimization and bring it into their company. But they are like kids in a candy store: they’re grabbing at a bunch of ideas, trying to get quick wins, and making mistakes along the way, getting inconclusive results, not tracking properly, and looking foolish in the end.
And this burns the organizational momentum you have. The most important resource you have in an organization is the support from your high-level executives. And you need to be very careful with that support because you can quickly destroy it by doing things the wrong way.
It’s important to first make sure you have all of the right building blocks in place: the right process, the right team, the ability to track and the right technology. And make sure you get a few wins, perhaps under the radar, so that you already have some support equity to work with.
Q: What are the most important questions to ask in the Explore phase?
Chris Goward: During Explore, we are looking for your visitors’ barriers to conversion. It’s a general research phase. (It’s called ‘Explore’ for a reason). In it, we are looking for insights about what questions to ask and validate. We are trying to identify…
What are the barriers to conversion?
What are the motivational triggers for your audience?
Why are people buying from you?
And answering those questions comes through the qualitative and quantitative research that’s involved in Explore. But it’s a very open-ended process. It’s an expansive process. So the questions are more about how to identify opportunities for testing.
Whereas Validate is a reductive process. During Validate, we know exactly what questions we are trying to answer, to determine whether the insights gained in Explore actually work.
Explore is one of two phases in the Infinity Optimization Process – our framework for conversion optimization. Read about the whole process, here.
Q: Is there such a thing as too much testing and / or optimizing?
Chris Goward: A lot of people think that if they’re A/B testing, and improving an experience or a landing page or a website…they can’t improve forever. The question many marketers have is, how do I know how long to do this? Is there going to be diminishing returns? By putting in the same effort will I get smaller and smaller results?
But we haven’t actually found this to be true. We have yet to find a company that we have over-A/B tested. And the reason is that visitor expectations continue to increase, your competitors don’t stop improving, and you continuously have new questions to ask about your business, business model, value proposition, etc.
So my answer is…yes, you will run out of opportunities to test, as soon as you run out of business questions. When you’ve answered all of the questions you have as a business, then you can safely stop testing.
Of course, you never really run out of questions. No business is perfect and understands everything. The role of experimentation is never done.
Case Study: DMV.org has been running an optimization program for 4+ years. Read about how they continue to double revenue year-over-year in this case study.
Q: Do you get better results with personalization or A/B testing or any other methods you have in mind?
Chris Goward: Personalization is a buzzword right now that a lot of marketers are really excited about. And personalization is important. But it’s not a new idea. It’s simply that technology and new tools are now available, and we have so much data that allows us to better personalize experiences.
I don’t believe that personalization and A/B testing are mutually exclusive. I think that personalization is a tactic that you can test and validate within all your experiences. But experimentation is more strategic.
At the highest level of your organization, having an experimentation ethos means that you’ll test anything. You could test personalization, you could test new product lines, or number of products, or types of value proposition messaging, etc. Everything is included under the umbrella of experimentation, if a company is oriented that way.
Personalization is really a tactic. And the goal of personalization is to create a more relevant experience, or a more relevant message. And that’s the only thing it does. And it does it very well.
Further Reading: Are you evaluating personalization at your company? Learn how to create the most effective personalization strategy with our 4-step roadmap.
Q: Is there such a thing as too much personalization? We have a client with over 40 personas, with a very complicated strategy, which makes reporting hard to justify.
Chris Goward: That’s an interesting question. Unlike experimentation, I believe there is a very real danger of too much personalization. Companies are often very excited about it. They’ll use all of the features of the personalization tools available to create (in your client’s case) 40 personas and a very complicated strategy. And they don’t realize that the maintenance cost of personalization is very high. It’s important to prove that a personalization strategy actually delivers enough business value to justify the increase in cost.
When you think about it, every time you come out with a new product, a new message, or a new campaign, you would have to create personalized experiences against 40 different personas. And that’s 40 times the effort of having a generic message. If you haven’t tested from the outset, to prove that all of those personas are accurate and useful, you could be wasting a lot of time and effort.
We always start a personalization strategy by asking, ‘what are the existing personas?’, and proving out whether those existing personas actually deliver distinct value apart from each other, or whether they should be grouped into a smaller number of personas that are more useful. And then, we test the messaging to see if there are messages that work better for each persona. It’s a step by step process that makes sure we are only creating overhead where it’s necessary and will create value.
Further Reading: Are you evaluating personalization at your company? Learn how to create the most effective personalization strategy with our 4-step roadmap.
Q: With the advance of personalization technology, will we see broader segments disappear? Will we go to 1:1 personalization, or will bigger segments remain relevant?
Chris Goward: Broad segments won’t disappear; they will remain valid. With things like multi-threaded personalization, you’ll be able to layer on some of the 1:1 information that you have, which may be product recommendations or behavioral targeting, on top of a broader segment. If a user falls into a broad segment, they may see that messaging in one area, and 1:1 messaging may appear in another area.
But if you try to eliminate broad segments and only create 1:1 personalization, you’ll create an infinite workload for yourself in trying to sustain all of those different content messaging segments. And it’s almost impossible for a marketing department practically to create infinite marketing messages.
Hudson Arnold: You are absolutely going to need both. I think there’s a different kind of opportunity, and a different kind of UX solution to those questions. Some media and commerce companies won’t have to struggle through that content production, because their natural output of 1:1 personalization will be showing a specific product or a certain article, which they don’t have to support from a content perspective.
What they will be missing out on is that notion of, what big segments are we missing? Are we not targeting moms? Newly married couples? CTOs vs. sales managers? Whatever the distinction is, that segment-level messaging is going to continue to be critical, for the foreseeable future. And the best personalization approach is going to balance both.
Q: How do you explain personalization to people who are still convinced that personalization is putting first and last name fields on landing pages?
A PANEL RESPONSE
André Morys: I compare it to the experience people have in a real store. If you go to a retail store, and you want to buy a TV, the salesperson will observe how you’re speaking, how you’re walking, how you’re dressed, and he will tailor his sales pitch to the type of person you are. He will notice if you’ve brought your family, if it’s your first time in a shop, or your 20th. He has all of these data points in his mind.
Personalization is the art of transporting this knowledge of how to talk to people on a 1:1 level to your website. And it’s not always easy, because you may not have all of the data. But you have to find out which data you can use. And if you can do personalization properly, you can get big uplift.
John Ekman: On the other hand, I heard a psychologist once say that people have more in common than what separates them. If you are looking for very powerful persuasion strategies, instead of thinking of the different individual traits and preferences that customers might have, it may be better to think about what they have in common. Because you’ll reach more people with your campaigns and landing pages. It will be interesting to see how the battle between general persuasion techniques and individual personalization techniques will result.
Chris Goward: It’s a good point. I tend to agree that the nirvana of 1:1 personalization may not be the right goal in some cases, because there are unintended consequences of that.
One is that it becomes more difficult to find generalized understanding of your positioning, of your value proposition, of your customers’ perspectives, if everything is personalized. There are no common threads.
The other is that there is significant maintenance cost in having really fine personalization. If you have 1:1 personalization with 1,000 people, and you update your product features, you have to think about how that message gets customized across 1,000 different messages rather than just updating one. So there is a cost to personalization. You have to validate that your approach to personalization pays off, and that is has enough benefit to balance out your cost and downside.
David Darmanin: [At Hotjar], we aren’t personalizing, actually. It’s a powerful thing to do, but there is a time to deploy it. If personalization adds too much complexity and slows you down, then obviously that can be a challenge. Like most things that can be complex, I think that they are the most valuable, when you have a high ticket price or very high value, where that touch of personalization has a big impact.
With Hotjar, we’re much more volume and lower price points, so it’s not yet a priority for us. Having said that, we have looked at it. But right now, we’re a startup, at the stage where speed is everything. And having many common threads is as important as possible, so we don’t want to add too much complexity now. But if you’re selling very expensive things, and you’re at a more advanced stage as a company, it would be crazy not to leverage personalization.
Q: How do you avoid harming organic SEO when doing conversion optimization?
Chris Goward: A common question! WiderFunnel was actually one of Google’s first authorized consultants for their testing tool, and Google told us is that they support optimization fully. They do not penalize companies for running A/B tests, if they are set up properly and the company is using a proper tool.
On top of that, what we’ve found is that the principles of conversion optimization parallel the principles of good SEO practice.
If you create a better experience for your users, and more of them convert, it actually sends a positive signal to Google that you have higher quality content.
Google looks at pogo-sticking, where people land on the SERP, find a result, and then return back to the SERP. Pogo-sticking signals to Google that this is not quality content. If a visitor lands on your page and converts, they are not going to come back to the SERP, which sends Google a positive signal. And we’ve actually never seen an example where SEO has been harmed by a conversion optimization program.
Q:When you are trying to solicit buy-in from leadership do you recommend going for big wins to share with the higher ups or smaller wins?
Chris Goward: Partly, it depends on how much equity you have to burn up front. If you are in a situation where you don’t have a lot of confidence from higher-ups about implementing an optimization program, I would recommend starting with more under the radar tests. Try to get momentum, get some early wins, and then share your success with the executives to show the potential. This will help you get more buy-in for more prominent areas.
This is actually one of the factors that you want to consider when prioritizing where to test. The “PIE Framework” shows you the three factors to help you prioritize.
One of them is Ease. Potential, Importance, and Ease. And one of the important aspects within Ease is political ease. So you want to look for areas that have political ease, which means there might not be as much sensitivity around them (so maybe not the homepage). Get those wins first, and create momentum, and then you can start sharing that throughout the organization to build that buy-in.
Q: Who would you say are the key stakeholders you need buy-in from, not only in senior leadership but critical members of the team?
Nick So: Besides the obvious senior leadership and key decision-makers as you mention, we find getting buy-in from related departments like branding, marketing, design, copywriters and content managers, etc., can be very helpful.
Having these teams on board can not only help with the overall approval process, but also helps ensure winning tests and strategies are aligned with your overall business and marketing strategy.
You should also consider involving more tangentially-related teams like customer support. This makes them a part of the process and testing culture, but your customer-facing teams can also be a great source for business insights and test ideas as well!
Q: Do you have any suggestions for success with lower traffic websites?
Nick So: In our testing experience, we find we get the most impactful results when we feel we have a strong understanding of the website’s visitors. In the Infinity Optimization Process, this understanding is gained through a balanced approach of Exploratory research, and Validated insights and results.
When a site’s traffic is low, the ability to Validate is decreased, and so we try to make up for it by increasing the time spent and work done in the Explore phase.
We take those yet-to-be-validated insights found in the Explore phase, and build a larger, more impactful single variation, and test the cluster of changes. (This variation is generally more drastic than we would create for a higher-traffic client, since we can validate those insights easily through multiple tests.)
Because of the more drastic changes, the variation should have a larger impact on conversion rate (and hopefully gain statistical significance with lower traffic). And because we have researched evidence to support these changes, there is a higher likelihood that they will perform better than a standard re-design.
If a site does not have enough overall primary conversions, but you definitely, absolutely MUST test, then I would look for a secondary metric further ‘upstream’ to optimize for. These should be goals that indicate or guide the primary conversion (e.g. clicks to form > form submission, add to cart > transaction). However with this strategy, stakeholders have to be aware that increases in this secondary goal may not be tied directly to increases of the primary goal at the same rate.
Q: What would you prioritize to test on a page that has lower traffic, in order to achieve statistical significance?
Chris Goward: The opportunities that are going to make the most impact really depend on the situation and the context. So if it’s a landing page or the homepage or a product page, they’ll have different opportunities.
But with any area, start by trying to understand your customers. If you have a low-traffic site, you’ll need to spend more time on the qualitative research side, really trying to understand: what are the opportunities, the barriers your visitors might be facing, and drilling into more of their perspective. Then you’ll have a more powerful test setup.
You’ll want to test dramatically. Test with fewer variations, make more dramatic changes with the variations, and be comfortable with your tests running longer. And while they are running and you are waiting for results, go talk to your customers. Go and run some more user testing, drill into your surveys, do post-purchase surveys, get on the phone and get the voice of customer. All of these things will enrich your ability to imagine their perspective and come up with more powerful insights.
In general, the things that are going to have the most impact are value proposition changes themselves. Trying to understand, do you have the right product-market fit, do you have the right description of your product, are you leading with the right value proposition point or angle?
Q: How far can I go with funnel optimization and testing when it comes to small local business?
A PANEL RESPONSE
David Darmanin: What do you mean by small local business? If you’re a startup just getting started, my advice would be to stop thinking about optimization and focus on failing fast. Get out there, change things, get some traction, get growth and you can optimize later. Whereas, if you’re a small but established local business, and you have traffic but it’s low, that’s different. In the end, conversion optimization is a traffic game. Small local business with a lot of traffic, maybe. But if traffic is low, focus on the qualitative, speak to your users, spend more time understanding what’s happening.
If you can’t test to significance, you should turn to qualitative research.
That would give you better results. If you don’t have the traffic to test against the last step in your funnel, you’ll end up testing at the beginning of your funnel. You’ll test for engagement or click through, and you’ll have to assume that people who don’t bounce and click through will convert. And that’s not always true. Instead, go start working with qualitative tools to see what the visitors you have are actually doing on your page and start optimizing from there.
André Morys: Testing with too small a sample size is really dangerous because it can lead to incorrect assumptions if you are not an expert in statistics. Even if you’re getting 10,000 to 20,000 orders per month, that is still a low number for A/B testing. Be aware of how the numbers work together. We’ve had people claiming 70% uplift, when the numbers are 64 versus 27 conversions. And this is really dangerous because that result is bull sh*t.
Jamie Elgie: It helps when you’ve had a screwup. When we started this process, we had not been successful with the radical design approach. But my advice for anyone championing optimization within an organization would be to focus on the overall objective.
For us, it was about getting our marketing spend to be more effective. If you can widen the funnel by making more people convert on your site, and then chase the people who convert (versus people who just land on your site) with your display media efforts, your social media efforts, your email efforts, and with all your paid efforts, you are going to be more effective. And that’s ultimately how we sold it.
It really sells itself though, once the process begins. It did not take long for us to see really impactful results that were helping our bottom line, as well as helping that overall strategy of making our display media spend, and all of our media spend more targeted.
Video Resource: Watch this webinar recording and discover how Jamie increased his company’s sales by more than 40% with evolutionary site redesign and conversion optimization.
Q: What has surprised you or stood out to you while doing CRO?
Jamie Elgie: There have been so many ‘A-ha!’s, and that’s the best part. We are always learning. Things that we are all convinced we should change on our website, or that we should change in our messaging in general, we’ll test them and actually find out.
We have one test running right now, and it’s failing, which is disappointing. But our entire emphasis as a team is changing, because we are learning something. And we are learning it without a huge amount of risk. And that, to me, has been the greatest thing about optimization. It’s not just the impact to your marketing funnel, it’s also teaching us. And it’s making us a better organization because we’re learning more.
One of the biggest benefits for me and my team has been how effective it is just to be able to say, ‘we can test that’.
If you have a salesperson who feels really strongly about something, and you feel really strongly that they’re wrong, the best recourse is to put it out on the table and say, ok, fine, we’ll go test that.
It enables conversations to happen that might not otherwise happen. It eliminates disputes that are not based on objective data, but on subjective opinion. It actually brings organizations together when people start to understand that they don’t need to be subjective about their viewpoints. Instead, you can bring your viewpoint to a test, and then you can learn from it. It’s transformational not just for a marketing organization, but for the entire company, if you can start to implement experimentation across all of your touch points.
Q: Do you have any tips for optimizing a website to conversion when the purchase cycle is longer, like 1.5 months?
Chris Goward: That’s a common challenge in B2B or with large ticket purchases for consumers. The best way to approach this is to
Track your leads and opportunities to the variation,
Then, track them through to the sale,
And then look at whether average order value changes between the variations, which implies the quality of the leads.
Because it’s easy to measure lead volume between variations. But if lead quality changes, then that makes a big impact.
We actually have a case study about this with Magento. We asked the question, “Which of these calls-to-action is actually generating the most valuable leads?”. And ran an experiment to try to find out. We tracked the leads all the way through to sale. This helped Magento optimize for the right calls-to-action going forward. And that’s an important question to ask near the beginning of your optimization program, which is, am I providing the right hook for my visitor?
Case Study: Discover how Magento increased lead volume and lead quality in the full case study.
Q: When you have a longer sales process, getting visitors to convert is the first step. We have softer conversions (eBooks) and urgent ones like demo requests. Do we need to pick ONE of these conversion options or can ‘any’ conversion be valued?
Nick So: Each test variation should be based on a single, primary hypothesis. And each hypothesis should be based on a single, primary conversion goal. This helps you keep your hypotheses and strategy focused and tactical, rather than taking a shotgun approach to just generally ‘improve the website’.
However, this focused approach doesn’t mean you should disregard all other business goals. Instead, count these as secondary goals and consider them in your post-test results analysis.
If a test increases demo requests by 50%, but cannibalizes ebook downloads by 75%, then, depending on the goal values of the two, a calculation has to be made to see if the overall net benefit of this tradeoff is positive or negative.
Different test hypotheses can also have different primary conversion goals. One test can focus on demos, but the next test can be focused on ebook downloads. You just have to track any other revenue-driving goals to ensure you aren’t cannibalizing conversions and having a net negative impact for each test.
Q: You’ve mainly covered websites that have a particular conversion goal, for example, purchasing a product, or making a donation. What would you say can be a conversion metric for a customer support website?
Nick So: When we help a client determine conversion metrics…
…we always suggest following the money.
Find the true impact that customer support might have on your company’s bottom line, and then determine a measurable KPI that can be tracked.
For example, would increasing the usefulness of the online support decrease costs required to maintain phone or email support lines (conversion goal: reduction in support calls/submissions)? Or, would it result in higher customer satisfaction and thus greater customer lifetime value (conversion goal: higher NPS responses via website poll)?
Q: Do you find that results from one client apply to other clients? Are you learning universal information, or information more specific to each audience?
Chris Goward: That question really gets at the nub of where we have found our biggest opportunity. When I started WiderFunnel in 2007, I thought that we would specialize in an industry, because that’s what everyone was telling us to do. They said, you need to specialize, you need to focus and become an expert in an industry. But I just sort of took opportunities as they came, with all kinds of different industries. And what I found is the exact opposite.
We’ve specialized in the process of optimization and personalization and creating powerful test design, but the insights apply to all industries.
What we’ve found is people are people, regardless of whether they’re shopping for a server, or shopping for socks, or donating to third-world countries, they go through the same mental process in each case.
The tactics are a bit different, sometimes. But often, we’re discovering breakthrough insights because we’re able to apply principles from one industry to another. For example, taking an e-commerce principle and identifying where on a B2B lead generation website we can apply that principle because someone is going through the same step in the process.
Most marketers spend most of their time thinking about their near-field competitors rather than in different industries, because it’s overwhelming to look at all of the other opportunities. But we are often able to look at an experience in a completely different way, because we are able to look at it through the lens of a different industry. That is very powerful.
Q: For companies that are not strictly e-commerce and have multiple business units with different goals, can you speak to any challenges with trying to optimize a visible page like the homepage so that it pleases all stakeholders? Is personalization the best approach?
Nick So: At WiderFunnel, we often work with organizations that have various departments with various business goals and agendas. We find the best way to manage this is to clearly quantify the monetary value of the #1 conversion goal of each stakeholder and/or business unit, and identify areas of the site that have the biggest potential impact for each conversion goal.
In most cases, the most impactful test area for one conversion goal will be different for another conversion goal (e.g. brand awareness on the homepage versus checkout for e-commerce conversions).
When there is a need to consider two different hypotheses with differing conversion goals on a single test area (like the homepage), teams can weigh the quantifiable impact + the internal company benefits in their decision and make that negotiation of prioritization and scheduling between teams.
I would not recommend personalization for this purpose, as that would be a stop-gap compromise that would limit the creativity and strategy of hypotheses, as well as create a disjointed experience for visitors, which would generally have a negative impact overall.
If you HAVE to run opposing strategies simultaneously on an area of the site, you could run multiple variations for different teams and measure different goals. Or, run mutually exclusive tests (keeping in mind these tactics would reduce test velocity, and would require more coordination between teams).
Q: Do you find testing strategies differ cross-culturally? Do conversion rates vary drastically across different countries / languages when using these strategies?
Chris Goward: We have run tests for many clients outside of the USA, such as in Israel, Sweden, Australia, UK, Canada, Japan, Korea, Spain, Italy and for the Olympics store, which is itself a global e-commerce experience in one site!
There are certainly cultural considerations and interesting differences in tactics. Some countries don’t have widespread credit card use, for example, and retailers there are accustomed to using alternative payment methods. Website design preferences in many Asian countries would seem very busy and overly colorful to a Western European visitor. At WiderFunnel, we specialize in English-speaking and Western-European conversion optimization and work with partner optimization companies around the world to serve our global and international clients.
Q: How do you recommend balancing the velocity of experimentation with quality, or more isolated design?
Chris Goward: This is where the art of the optimization strategist comes into play. And it’s where we spend the majority of our effort – in creating experiment plans. We look at all of the different options we could be testing, and ruthlessly narrow them down to the things that are going to maximize the potential growth and the potential insights.
And there are frameworks we use to do that. Its all about prioritization. There are hundreds of ideas that we could be testing, so we need to prioritize with as much data as we can. So, we’ve developed some frameworks to do that. The PIE Framework allows you to prioritize ideas and test areas based on the potential, importance, and ease. The potential for improvement, the importance to the business, and the ease of implementation. And sometimes these are a little subjective, but the more data you can have to back these up, the better your focus and effort will be in delivering results.
Q: I notice that you often have multiple success metrics, rather than just one? Does this ever lead to cherry-picking a metric to make sure that the test you wanted to win seem like it’s the winner?
Chris Goward: Good question! We actually look for one primary metric that tells us what the business value of a winning test is. But we also track secondary metrics. The goal is to learn from the other metrics, but not use them for decision-making. In most cases, we’re looking for a revenue-driving primary metric. Revenue-per-visitor, for example, is a common metric we’ll use. But the other metrics, whether conversion rate or average order value or downloads, will tell us more about user behavior, and lead to further insights.
There are two steps in our optimization process that pair with each other in the Validate phase. One is design of experiments, and the other is results analysis. And if the results analysis is done correctly, all of the metrics that you’re looking at in terms of variation performance, will tell you more about the variations. And if the design of experiments has been done properly, then you’ll gather insights from all of the different data.
But you should be looking at one metric to tell you whether or not a test won.
Q: When do you make the call for A/B tests for statistical significance? We run into the issue of varying test results depending on part of the week we’re running a test. Sometimes, we even have to run a test multiple times.
Chris Goward: It sounds like you may be ending your tests or trying to analyze results too early. You certainly don’t want to be running into day-of-the-week seasonality. You should be running your tests over at least a week, and ideally two weekends to iron out that seasonality effect, because your test will be in a different context on different days of the week, depending on your industry.
So, run your tests a little bit longer and aim for statistical significance. And you want to use tools that calculate statistical significance reliably, and help answer the real questions that you’re trying to ask with optimization. You should aim for that high level of statistical significance, and iron out that seasonality. And sometimes you’ll want to look at monthly seasonality as well, and retest questionable things within high and low urgency periods. That, of course, will be more relevant depending on your industry and whether or not seasonality is a strong factor.
Q: Is there a way to conclusively tell why a test lost or was inconclusive? To know what the hidden gold is?
Chris Goward: Developing powerful hypotheses is dependent on having workable theories. Seeking to determine the “Why” behind the results is some of the most interesting part of the work.
The only way to tell conclusively is to infer a potential reason, then test again with new ways to validate that inference. Eventually, you can form conversion optimization theories and then test based on those theories. While you can never really know definitively the “why” behind the “what”, when you have theories and frameworks that work to predict results, they become just as useful.
As an example, I was reviewing a recent test for one of our clients and it didn’t make sense based on our LIFT Model. One of the variations was showing under-performance against another variation, but I believed strongly that it should have over-performed. I struggled for some time to align this performance with our existing theories and eventually discovered the conversion rate listed was a typo! The real result aligned perfectly with our existing framework, which allowed me to sleep at night again!
Q: How many visits do you need to get to statistically relevant data from any individual test?
Chris Goward: The number of visits is just one of the variables that determines statistical significance. The conversion rate of the Control and conversion rate delta between the variations are also part of the calculation. Statistical significance is achieved when there is enough traffic (i.e. sample size), enough conversions, and the conversion rate delta is great enough.
Here’s a handy Excel test duration calculator. Fortunately, today’s testing tools calculate statistical significance automatically, which simplifies the conversion champion’s decision-making (and saves hours of manual calculation!)
When planning tests, it’s helpful to estimate the test duration, but it isn’t an exact science. As a rule-of-thumb, you should plan for smaller isolation tests to run longer, as the impact on conversion rate may be less. The test may require more conversions to potentially achieve confidence.
Larger, more drastic cluster changes would typically run for a shorter period of time, as they have more potential to have a greater impact. However, we have seen that isolations CAN have the potential to have big impact. If the evidence is strong enough, test duration shouldn’t hinder you from trying smaller, more isolated changes as they can lead to some of the biggest insights.
Often, people that are new to testing become frustrated with tests that never seem to finish. If you’ve run a test with more than 30,000 to 50,000 visitors and one variation is still not statistically significant over another, then your test may not ever yield a clear winner and you should revise your test plan or reduce the number of variations being tested.
Q: We are new to optimization (had a few quick wins with A/B testing and working toward a geo targeting project). Looking at your Infinity Optimization Process, I feel like we are doing a decent job with exploration and validation – for this being a new program to us. Our struggle seems to be your orange dot… putting the two sides together – any advice?
Chris Goward: If you’re getting insights from your Exploratory research, those insights should tie into the Validate tests that you’re running. You should be validating the insights that you’re getting from your Explore phase. If you started with valid insights, the results that you get really should be generating growth, and they should be generating insights.
Part of it is your Design of Experiments (DOE). DOE is how you structure your hypotheses and how you structure your variations to generate both growth and insights, and those are the two goals of your tests.
If you’re not generating growth, or you’re not generating insights, then your DOE may be weak, and you need to go back to your strategy and ask, why am I testing this variation? Is it just a random idea? Or, am I really isolating it against another variation that’s going to teach me something as well as generate lift? If you’re not getting the orange dot right, then you probably need to look at researching more about Design of Experiments.
Q: When test results are insignificant after lots impressions, how do you know when to ‘call it a tie’ and stop that test and move on?
Chris Goward: That’s a question that requires a large portion of “it depends.” It depends on whether:
You have other tests ready to run with the same traffic sources
The test results are showing high volatility or have stabilized
The test insights will be important for the organization
There’s an opportunity cost to every test. You could always be testing something else and need to constantly be asking whether this is the best test to be running now vs. the cost and potential benefit of the next test in your conversion strategy.
Q: There are tools meant to increase testing velocity with pre-built widgets and pre-built test variations, even – what are your thoughts on this approach?
A PANEL RESPONSE
John Ekman: Pre-built templates provide a way to get quick wins and uplift. But you won’t understand why it created an uplift. You won’t understand what’s going on in the brain of your users. For someone who believes that experimentation is a way to look in the minds of whoever is in front of the screen, I think these methods are quite dangerous.
Chris Goward: I’ll take a slightly different stance. As much as I talk about understanding the mind of the customer, asking why, and testing based on hypotheses, there is a tradeoff. A tradeoff between understanding the why and just getting growth. If you want to understand the why infinitely, you’ll do multivariate testing and isolate every potential variable. But in practice, that can’t happen. Very few have enough traffic to multivariate test everything.
But if you don’t have tons of traffic and you want to get faster results, maybe you don’t want to know the why about anything, and you just want to get lift.
There might be a time to do both. Maybe your website performance is really bad, or you just want to try a left-field variation, just to see if it works…if you get a 20% lift in your revenue, that’s not a failure. That’s not a bad thing to do. But then, you can go back and isolate all of the things to ask yourself: Well, I wonder why that won, and start from there.
The approach we usually take at WiderFunnel is to reserve 10% of the variations for ‘left-field’ variations. As in, we don’t know why this will work, but we’re just going to test something crazy and see if it sticks.
David Darmanin: I agree, and disagree. We’re living in an era when technology has become so cheap, that I think it’s dangerous for any company to try to automate certain things, because they’re going to just become one of many.
Creating a unique customer experience is going to become more and more important.
If you are using tools like a platform, where you are picking and choosing what to use so that it serves your strategy and the way you want to try to build a business, that makes sense to me. But I think it’s very dangerous to leave that to be completely automated.
Some software companies out there are trying to build a completely automated conversion rate optimization platform that does everything. But that’s insane. If many sites are all aligned in the same way, if it’s pure AI, they’re all going to end up looking the same. And who’s going to win? The other company that pops up out of nowhere, and does everything differently. That isn’t fully ‘optimized’ and is more human.
Optimization, in itself, if it’s too optimized, there is a danger. If we eliminate the human aspect, we’re kind of screwed.
You’ve probably heard time and time again that content marketing is important in growing your business’s online presence. When done correctly, it can increase brand awareness, advocacy, and lead to more sales. The problem is many businesses are doing it wrong. Many business owners think they need to be creating X new pieces of content per week, which simply isn’t true. Producing content for the sake of producing content is not a productive use of time and will not rank you higher in Google or drive more sales. Sure the more quality content you can create, the more traffic you…