Tag Archives: alternative

How to do server-side testing for single page app optimization

Reading Time: 5 minutes

Gettin’ technical.

We talk a lot about marketing strategy on this blog. But today, we are getting technical.

In this post, I team up with WiderFunnel front-end developer, Thomas Davis, to cover the basics of server-side testing from a web development perspective.

The alternative to server-side testing is client-side testing, which has arguably been the dominant testing method for many marketing teams, due to ease and speed.

But modern web applications are becoming more dynamic and technically complex. And testing within these applications is becoming more technically complex.

Server-side testing is a solution to this increased complexity. It also allows you to test much deeper. Rather than being limited to testing images or buttons on your website, you can test algorithms, architectures, and re-brands.

Simply put: If you want to test on an application, you should consider server-side testing.

Let’s dig in!

Note: Server-side testing is a tactic that is linked to single page applications (SPAs). Throughout this post, I will refer to web pages and web content within the context of a SPA. Applications such as Facebook, Airbnb, Slack, BBC, CodeAcademy, eBay, and Instagram are SPAs.


Defining server-side and client-side rendering

In web development terms, “server-side” refers to “occurring on the server side of a client-server system.”

The client refers to the browser, and client-side rendering occurs when:

  1. A user requests a web page,
  2. The server finds the page and sends it to the user’s browser,
  3. The page is rendered on the user’s browser, and any scripts run during or after the page is displayed.
Static app server
A basic representation of server-client communication.

The server is where the web page and other content live. With server-side rendering, the requested web page is sent to the user’s browser in final form:

  1. A user requests a web page,
  2. The server interprets the script in the page, and creates or changes the page content to suit the situation
  3. The page is sent to the user in final form and then cannot be changed using server-side scripting.

To talk about server-side rendering, we also have to talk a little bit about JavaScript. JavaScript is a scripting language that adds functionality to web pages, such as a drop-down menu or an image carousel.

Traditionally, JavaScript has been executed on the client side, within the user’s browser. However, with the emergence of Node.js, JavaScript can be run on the server side. All JavaScript executing on the server is running through Node.js.

*Node.js is an open-source, cross-platform JavaScript runtime environment, used to execute JavaScript code server-side. It uses the Chrome V8 JavaScript engine.

In laymen’s (ish) terms:

When you visit a SPA web application, the content you are seeing is either being rendered in your browser (client-side), or on the server (server-side).

If the content is rendered client-side, JavaScript builds the application HTML content within the browser, and requests any missing data from the server to fill in the blanks.

Basically, the page is incomplete upon arrival, and is completed within the browser.

If the content is being rendered server-side, your browser receives the application HTML, pre-built by the server. It doesn’t have to fill in any blanks.

Why do SPAs use server-side rendering?

There are benefits to both client-side rendering and server-side rendering, but render performance and page load time are two huge pro’s for the server side.

(A 1 second delay in page load time can result in a 7% reduction in conversions, according to Kissmetrics.)

Server-side rendering also enables search engine crawlers to find web content, improving SEO; and social crawlers (like the crawlers used by Facebook) do not evaluate JavaScript, making server-side rendering beneficial for social searching.

With client-side rendering, the user’s browser must download all of the application JavaScript, and wait for a response from the server with all of the application data. Then, it has to build the application, and finally, show the complete HTML content to the user.

All of which to say, with a complex application, client-side rendering can lead to sloooow initial load times. And, because client-side rendering relies on each individual user’s browser, the developer only has so much control over load time.

Which explains why some developers are choosing to render their SPAs on the server side.

But, server-side rendering can disrupt your testing efforts, if you are using a framework like Angular or React.js. (And the majority of SPAs use these frameworks).

The disruption occurs because the version of your application that exists on the server becomes out of sync with the changes being made by your test scripts on the browser.

NOTE: If your web application uses Angular, React, or a similar framework, you may have already run into client-side testing obstacles. For more on how to overcome these obstacles, and successfully test on AngularJS apps, read this blog post.


Testing on the server side vs. the client side

Client-side testing involves making changes (the variation) within the browser by injecting Javascript after the original page has already loaded.

The original page loads, the content is hidden, the necessary elements are changed in the background, and the ‘new’ version is shown to the user post-change. (Because the page is hidden while these changes are being made, the user is none-the-wiser.)

As I mentioned earlier, the advantages of client-side testing are ease and speed. With a client-side testing tool like VWO, a marketer can set up and execute a simple test using a WYSIWYG editor without involving a developer.

But for complex applications, client-side testing may not be the best option: Layering more JavaScript on top of an already-bulky application means even slower load time, and an even more cumbersome user experience.

A Quick Hack

There is a workaround if you are determined to do client-side testing on a SPA application. Web developers can take advantage of features like Optimizely’s conditional activation mode to make sure that testing scripts are only executed when the application reaches a desired state.

However, this can be difficult as developers will have to take many variables into account, like location changes performed by the $routeProvider, or triggering interaction based goals.

To avoid flicker, you may need to hide content until the front-end application has initialized in the browser, voiding the performance benefits of using server-side rendering in the first place.

WiderFunnel - client side testing activation mode
Activation Mode waits until the framework has loaded before executing your test.



When you do server-side testing, there are no modifications being made at the browser level. Rather, the parameters of the experiment variation (‘User 1 sees Variation A’) are determined at the server route level, and hooked straight into the javascript application through a service provider.

Here is an example where we are testing a pricing change:

“Ok, so, if I want to do server-side testing, do I have to involve my web development team?”

Yep.

But, this means that testing gets folded into your development team’s work flow. And, it means that it will be easier to integrate winning variations into your code base in the end.

If yours is a SPA, server-side testing may be the better choice, despite the work involved. Not only does server-side testing embed testing into your development workflow, it also broadens the scope of what you can actually test.

Rather than being limited to testing page elements, you can begin testing core components of your application’s usability like search algorithms and pricing changes.

A server-side test example!

For web developers who want to do server-side testing on a SPA, Tom has put together a basic example using Optimizely SDK. This example is an illustration, and is not functional.

In it, we are running a simple experiment that changes the color of a button. The example is built using Angular Universal and express JS. A global service provider is being used to fetch the user variation from the Optimizely SDK.

Here, we have simply hard-coded the user ID. However, Optimizely requires that each user have a unique ID. Therefore, you may want to use the user ID that already exists in your database, or store a cookie through express’ Cookie middleware.

Are you currently doing server-side testing?

Or, are you client-side testing on a SPA application? What challenges (if any) have you faced? How have you handled them? Do you have any specific questions? Let us know in the comments!

The post How to do server-side testing for single page app optimization appeared first on WiderFunnel Conversion Optimization.

Continue reading – 

How to do server-side testing for single page app optimization

How to get evergreen results from your landing page optimization

Reading Time: 7 minutes

Landing page optimization is old news.

Seriously. A quick google will show you that Unbounce, QuickSprout, Moz, Qualaroo, Hubspot, Wordstream, Optimizely, CrazyEgg, VWO (and countless others), have been writing tips and guides on how to optimize your landing pages for years.

Not to mention the several posts we have already published on the WiderFunnel blog since 2008.

And yet. This conversation is so not over.

Warning: If your landing page optimization goals are short-term, or completely focused on conversion rate lift, this post may be a waste of your time. If your goal is to continuously have the best-performing landing pages on the internet, keep reading.



Marketers are funnelling more and more money into paid advertising, especially as Google allocates more and more SERP space to ads.

In fact, as an industry, we are spending upwards of $92 billion annually on paid search advertising alone.

landing-page-optimization-SERP-space
The prime real estate on a Google search results page often goes to paid.

And it’s not just search advertising that is seeing an uptick in spend, but social media advertising too.

It makes sense that marketers are still obsessing over their landing page conversion rates: this traffic is costly and curated. These are visitors that you have sought out, that share characteristics with your target market. It is extremely important that these visitors convert!

But, there comes a time in every optimizer’s life, when they face the cruel reality of diminishing returns. You’ve tested your landing page hero image. You’ve tested your value proposition. You’ve tested your form placement. And now, you’ve hit a plateau.

So, what next? What’s beyond the tips and guides? What is beyond the optimization basics?

1) Put on your customer’s shoes.

First things first: Let’s do a quick sanity check.

When you test your hero image, or your form placement, are you testing based on tips and recommended best practices? Or, are you testing based on a specific theory you have about your page visitors?

landing-page-optimization-customer-shoes
Put on your customer’s shoes.

Tips and best practices are a fine place to start, but the insight behind why those tactics work (or don’t work) for your visitors is where you find longevity.

The best way to improve experiences for your visitors is to think from their perspective. And the best way to do that is to use frameworks, and framework thinking, to get robust insights about your customers.

– Chris Goward, Founder & CEO, WiderFunnel

Laying the foundation

It’s very difficult to think from a different perspective. This is true in marketing as much as it is in life. And it’s why conversion optimization and A/B testing have become so vital: We no longer have to guess at what our visitors want, but can test instead!

That said, a test requires a hypothesis. And a legitimate hypothesis requires a legitimate attempt to understand your visitor’s unique perspective.

To respond to this need for understanding, WiderFunnel developed the LIFT Model® in 2008: our foundational framework for identifying potential barriers to conversion on a page from the perspective of the page visitor.

Get optimization ideas with the LIFT poster!

Get the LIFT Model poster, and challenge yourself to keep your visitor’s perspective in mind at all times. Use the six conversion factors to analyze your pages, and get optimization ideas!



By entering your email, you’ll receive bi-weekly WiderFunnel Blog updates and other resources to help you become an optimization champion.


The LIFT Model attempts to capture the idea of competing forces in communication, narrowing them down to the most salient aspects of communication that marketers should consider.

I wanted to apply the principles of Relevance, Clarity, Distraction, Urgency and Anxiety to what we were delivering to the industry and not just to our clients. And the LIFT Model is a part of that: making something as simple as possible but no simpler.

– Chris Goward

When you look at your page through a lens like the LIFT Model, you are forced to question your assumptions about what your visitors want when they land on your page.

landing-page-optimization-LIFT-Model
View your landing pages through a framework lens.

You may love an interactive element, but is it distracting your visitors? You may think that your copy creates urgency, but is it really creating anxiety?

If you are an experienced optimizer, you may have already incorporated a framework like the LIFT Model into your optimization program. But, after you have analyzed the same page multiple times, how do you continue to come up with new ideas?

Here are a few tips from the WiderFunnel Strategy team:

  1. Bring in fresh eyes from another team to look at and use your page
  2. User test, to watch and record how actual users are using your page
  3. Sneak a peek at your competitors’ landing pages: Is there something they’re doing that might be worth testing on your site?
  4. Do your page analyses as a team: many heads are better than one
  5. Brainstorm totally new, outside-the-box ideas…and test one!

You should always err on the side of “This customer experience could be better.” After all, it’s a customer-centric world, and we’re just marketing in it.

2) Look past the conversion rate.

“Landing page optimization”, like “conversion rate optimization”, is a limiting term. Yes, on-page optimization is key, but mature organizations view “landing page optimization” as the optimization of the entire experience, from first to last customer touchpoint.

Landing pages are only one element of a stellar, high-converting marketing campaign. And focusing all of your attention on optimizing only one element is just foolish.

From testing your featured ads, to tracking click-through rates of Thank You emails, to tracking returns and refunds, to tracking leads through the rest of the funnel, a better-performing landing page is about much more than on-page conversion rate lift.

landing-page-optimization-big-picture
On-page optimization is just one part of the whole picture.

An example is worth 1,000 words

One of our clients is a company that provides an online consumer information service—visitors type in a question and get an Expert answer. One of the first zones (areas on their website) that we focused on was a particular landing page funnel.

Visitors come from an ad, and land on page where they can ask their question. They then enter a 4-step funnel: Step 1: Ask the question > Step 2: Add more information > Step 3: Pick an Expert > Step 4: Get an answer (aka the checkout page)

Our primary goal was to increase transactions, meaning we had to move visitors all the way through the funnel. But we were also tracking refunds and chargebacks, as well as revenue per visitor.

More than pushing a visitor to ‘convert’, we wanted to make sure those visitors went on to be happy, satisfied customers.

In this experiment, we focused on the value proposition statements. The control landing page exclaimed, “A new question is answered every 9 seconds!“. Our Strategy team had determined (through user testing) that “speed of answers” was the 8th most valuable element of the service for customers, and that “peace of mind / reassurance” was the most important.

So, they tested two variations, featuring two different value proposition statements meant to create more peace of mind for visitors:

  • “Join 6,152,585 satisfied customers who got professional answers…”
  • “Connect One on One with an Expert who will answer your question”

Both of these variations ultimately increased transactions, by 6% and 9.4% respectively. But! We also saw large decreases in refunds and chargebacks with both variations, and large increases in net revenue per visitor for both variations.

By following visitors past the actual conversion, we were able to confirm that these initial statements set an impactful tone: visitors were more satisfied with their purchases, and comfortable investing more in their expert responses.

3) Consider the big picture.

As you think of landing page optimization as the optimization of a complete digital experience, you should also think of landing page optimization as part of your overall digital optimization strategy.

When you discover an insight about visitors to your product page, feed it into a test on your landing page. When you discover an insight about visitor behavior on your landing page, feed it into a test on your website.

It’s true that your landing pages most likely cater to specific visitor segments, who may behave totally differently than your organic visitors. But, it is also true that landing page wins may be overall wins.

Plus, landing page insights can be very valuable, because they are often new visitor insights. And now, a little more advice from Chris Goward, optimization guru:

“Your best opportunities for testing your value proposition are with first impression visitors. These are usually new visitors to your high traffic landing pages or your home page […]

By split testing your alternative value propositions with new visitors, you’ll reduce your exposure to existing customers or prospects who are already in the consideration phase. New prospects have a blank canvas for you to present your message variations and see what sticks.

Then, from the learning gained on landing pages, you can validate insights with other target audience groups and with your customers to leverage the learning company-wide.

Landing page testing can do more than just improve conversion rates on landing pages. When done strategically, it can deliver powerful, high-leverage marketing insights.”



Just because your landing pages are separate from your website, does not mean that your landing page optimization should be separate from your other optimization efforts. A landing page is just another zone, and you are free to (and should) use insights from one zone when testing on another zone.

4) Go deeper, explore further.

A lot of marketers talk about landing page design: how to build the right landing page, where to position each element, what color scheme and imagery to use, etc.

But when you dig into the why behind your test results, it’s like breaking into a piñata of possibilities, or opening a box of idea confetti.

landing-page-optimization-ideas
Discovering the reason behind the result is like opening a box of idea confetti!

Why do your 16-25 year old, mobile users respond so favorably to a one-minute video testimonial from a past-purchaser? Do they respond better to this indicator of social proof than another?

Why do your visitors prefer one landing page under normal circumstances, and a different version when external factors change (like a holiday, or a crisis)? Can you leverage this insight throughout your website?

Why does one type of urgency phrasing work, while slightly different wording decreases conversions on your page? Are your visitors sensitive to overly salesy copy? Why or why not?

Not only are there hundreds of psychological principles to explore within your landing page testing, but landing page optimization is also intertwined with your personalization strategy.

For many marketers, personalized landing pages are becoming more normal. And personalization opens the door to even more potential customer insights. Assuming you already have visitor segments, you should test the personalized experiences on your landing pages.

For example, imagine you have started using your visitors’ first names in the hero banner of your landing page. Have you validated that this personalized experience is more effective than another, like moving a social proof indicator above the fold? Both can be deemed personalization, but they tap into very different motivations.

From psychological principles, to validating your personalized experiences, the possibilities for testing on your landing pages are endless.

Just keep testing, Dory-style

Your landing page(s) will never be “optimized”. That is the beauty and cruelty of optimization: we are always chasing unattainable perfection.

But your landing pages can definitely be better than they are now. Even if you have a high-converting page, even if your page is listed by Hubspot as one of the 16 best designed landing pages, even if you’ve followed all of the rules…your landing page can be better.

Because I’m not just talking about conversions, I’m talking about your entire customer experience. If you give them the opportunity, your new users will tell you what’s wrong with your page.

They’ll tell you where it is unclear and where it is distracting.

They’ll tell you what motivates them.

They’ll tell you how personal you should get.

They’ll tell you how to set expectations so that they can become satisfied customers or clients.

A well-designed landing page is just the beginning of landing page optimization.

The post How to get evergreen results from your landing page optimization appeared first on WiderFunnel Conversion Optimization.

More: 

How to get evergreen results from your landing page optimization

“The more tests, the better!” and other A/B testing myths, debunked

Reading Time: 8 minutes

Will the real A/B testing success metrics please stand up?

It’s 2017, and most marketers understand the importance of A/B testing. The strategy of applying the scientific method to marketing to prove whether an idea will have a positive impact on your bottom-line is no longer novel.

But, while the practice of A/B testing has become more and more common, too many marketers still buy into pervasive A/B testing myths. #AlternativeFacts.

This has been going on for years, but the myths continue to evolve. Other bloggers have already addressed myths like “A/B testing and conversion optimization are the same thing”, and “you should A/B test everything”.

As more A/B testing ‘experts’ pop up, A/B testing myths have become more specific. Driven by best practices and tips and tricks, these myths represent ideas about A/B testing that will derail your marketing optimization efforts if left unaddressed.

Avoid the pitfalls of ad-hoc A/B testing…

Get this guide, and learn how to build an optimization machine at your company. Discover how to use A/B testing as part of your bigger marketing optimization strategy!



By entering your email, you’ll receive bi-weekly WiderFunnel Blog updates and other resources to help you become an optimization champion.



But never fear! With the help of WiderFunnel Optimization Strategist, Dennis Pavlina, I’m going to rebut four A/B testing myths that we hear over and over again. Because there is such a thing as a successful, sustainable A/B testing program…

Into the light, we go!

Myth #1: The more tests, the better!

A lot of marketers equate A/B testing success with A/B testing velocity. And I get it. The more tests you run, the faster you run them, the more likely you are to get a win, and prove the value of A/B testing in general…right?

Not so much. Obsessing over velocity is not going to get you the wins you’re hoping for in the long run.

Mike St Laurent

The key to sustainable A/B testing output, is to find a balance between short-term (maximum testing speed), and long-term (testing for data-collection and insights).

Michael St Laurent, Senior Optimization Strategist, WiderFunnel

When you focus solely on speed, you spend less time structuring your tests, and you will miss out on insights.

With every experiment, you must ensure that it directly addresses the hypothesis. You must track all of the most relevant goals to generate maximum insights, and QA all variations to ensure bugs won’t skew your data.

Dennis Pavlina

An emphasis on velocity can create mistakes that are easily avoided when you spend more time on preparation.

Dennis Pavlina, Optimization Strategist, WiderFunnel

Another problem: If you decide to test many ideas, quickly, you are sacrificing your ability to really validate and leverage an idea. One winning A/B test may mean quick conversion rate lift, but it doesn’t mean you’ve explored the full potential of that idea.

You can often apply the insights gained from one experiment, when building out the strategy for another experiment. Plus, those insights provide additional evidence for testing a particular concept. Lining up a huge list of experiments at once without taking into account these past insights can result in your testing program being more scattershot than evidence-based.

While you can make some noise with an ‘as-many-tests-as-possible’ strategy, you won’t see the big business impact that comes from a properly structured A/B testing strategy.

Myth #2: Statistical significance is the end-all, be-all

A quick definition

Statistical significance: The probability that a certain result is not due to chance. At WiderFunnel, we use a 95% confidence level. In other words, we can say that there is a 95% chance that the observed result is because of changes in our variation (and a 5% chance it is due to…well…chance).

If a test has a confidence level of less than 95% (positive or negative), it is inconclusive and does not have our official recommendation. The insights are deemed directional and subject to change.

Ok, here’s the thing about statistical significance: It is important, but marketers often talk about it as if it is the only determinant for completing an A/B test. In actuality, you cannot view it within a silo.

For example, a recent experiment we ran reached statistical significance three hours after it went live. Because statistical significance is viewed as the end-all, be-all, a result like this can be exciting! But, in three hours, we had not gathered a representative sample size.

Claire Vignon Keser

You should not wait for a test to be significant (because it may never happen) or stop a test as soon as it is significant. Instead, you need to wait for the calculated sample size to be reached before stopping a test. Use a test duration calculator to understand better when to stop a test.

After 24 hours, the same experiment had dropped to a confidence level of 88%, meaning that there was now only an 88% likelihood that the difference in conversion rates was not due to chance – i.e. statistically significant.

Traffic behaves differently over time for all businesses, so you should always run a test for full business cycles, even if you have reached statistical significance. This way, your experiment has taken into account all of the regular fluctuations in traffic that impact your business.

For an e-commerce business, a full business cycle is typically a one-week period; for subscription-based businesses, this might be one month or longer.

Myth #2, Part II: You have to run a test until reaches statistical significance

As Claire pointed out, this may never happen. And it doesn’t mean you should walk away from an A/B test, completely.

As I said above, anything below 95% confidence is deemed subject to change. But, with testing experience, an expert understanding of your testing tool, and by observing the factors I’m about to outline, you can discover actionable insights that are directional (directionally true or false).

  • Results stability: Is the conversion rate difference stable over time, or does it fluctuate? Stability is a positive indicator.
ab testing results stability
Check your graphs! Are conversion rates crossing? Are the lines smooth and flat, or are there spikes and valleys?
  • Experiment timeline: Did I run this experiment for at least a full business cycle? Did conversion rate stability last throughout that cycle?
  • Relativity: If my testing tool uses t-test to determine significance, am I looking at the hard numbers of actual conversions in addition to conversion rate? Does the calculated lift make sense?
  • LIFT & ROI: Is there still potential for the experiment to achieve X% lift? If so, you should let it run as long as it is viable, especially when considering the ROI.
  • Impact on other elements: If elements outside the experiment are unstable (social shares, average order value, etc.) the observed conversion rate may also be unstable.

You can use these factors to make the decision that makes the most sense for your business: implement the variation based on the observed trends, abandon the variation based on observed trends, and/or create a follow-up test!

Myth #3: An A/B test is only as good as its effect on conversion rates

Well, if conversion rate is the only success metric you are tracking, this may be true. But you’re underestimating the true growth potential of A/B testing if that’s how you structure your tests!

To clarify: Your main success metric should always be linked to your biggest revenue driver.

But, that doesn’t mean you shouldn’t track other relevant metrics! At WiderFunnel, we set up as many relevant secondary goals (clicks, visits, field completions, etc.) as possible for each experiment.

Dennis Pavlina

This ensures that we aren’t just gaining insights about the impact a variation has on conversion rate, but also the impact it’s having on visitor behavior.

– Dennis Pavlina

When you observe secondary goal metrics, your A/B testing becomes exponentially more valuable because every experiment generates a wide range of secondary insights. These can be used to create follow up experiments, identify pain points, and create a better understanding of how visitors move through your site.

An example

One of our clients provides an online consumer information service — users type in a question and get an Expert answer. This client has a 4-step funnel. With every test we run, we aim to increase transactions: the final, and most important conversion.

But, we also track secondary goals, like click-through-rates, and refunds/chargebacks, so that we can observe how a variation influences visitor behavior.

In one experiment, we made a change to step one of the funnel (the landing page). Our goal was to set clearer visitor expectations at the beginning of the purchasing experience. We tested 3 variations against the original, and all 3 won resulted in increased transactions (hooray!).

The secondary goals revealed important insights about visitor behavior, though! Firstly, each variation resulted in substantial drop-offs from step 1 to step 2…fewer people were entering the funnel. But, from there, we saw gradual increases in clicks to steps 3 and 4.

Our variations seemed to be filtering out visitors without strong purchasing intent. We also saw an interesting pattern with one of our variations: It increased clicks from step 3 to step 4 by almost 12% (a huge increase), but decreased actual conversions by -1.6%. This result was evidence that the call-to-action on step 4 was extremely weak (which led to a follow-up test!)

ab testing funnel analysis
You can see how each variation fared against the Control in this funnel analysis.

We also saw large decreases in refunds and chargebacks for this client, which further supported the idea that the right visitors (i.e. the wrong visitors) were the ones who were dropping off.

This is just a taste of what every A/B test could be worth to your business. The right goal tracking can unlock piles of insights about your target visitors.

Myth #4: A/B testing takes little to no thought or planning

Believe it or not, marketers still think this way. They still view A/B testing on a small scale, in simple terms.

But A/B testing is part of a greater whole—it’s one piece of your marketing optimization program—and you must build your tests accordingly. A one-off, ad-hoc test may yield short-term results, but the power of A/B testing lies in iteration, and in planning.

ab testing infinity optimization process
A/B testing is just a part of the marketing optimization machine.

At WiderFunnel, a significant amount of research goes into developing ideas for a single A/B test. Even tests that may seem intuitive, or common-sensical, are the result of research.

ab testing planning
The WiderFunnel strategy team gathers to share and discuss A/B testing insights.

Because, with any test, you want to make sure that you are addressing areas within your digital experiences that are the most in need of improvement. And you should always have evidence to support your use of resources when you decide to test an idea. Any idea.

So, what does a revenue-driving A/B testing program actually look like?

Today, tools and technology allow you to track almost any marketing metric. Meaning, you have an endless sea of evidence that you can use to generate ideas on how to improve your digital experiences.

Which makes A/B testing more important than ever.

An A/B test shows you, objectively, whether or not one of your many ideas will actually increase conversion rates and revenue. And, it shows you when an idea doesn’t align with your user expectations and will hurt your conversion rates.

And marketers recognize the value of A/B testing. We are firmly in the era of the data-driven CMO: Marketing ideas must be proven, and backed by sound data.

But results-driving A/B testing happens when you acknowledge that it is just one piece of a much larger puzzle.

One of our favorite A/B testing success stories is that of DMV.org, a non-government content website. If you want to see what a truly successful A/B testing strategy looks like, check out this case study. Here are the high level details:

We’ve been testing with DMV.org for almost four years. In fact, we just launched our 100th test with them. For DMV.org, A/B testing is a step within their optimization program.

Continuous user research and data gathering informs hypotheses that are prioritized and created into A/B tests (that are structured using proper Design of Experiments). Each A/B test delivers business growth and/or insights, and these insights are fed back into the data gathering. It’s a cycle of continuous improvement.

And here’s the kicker: Since DMV.org began A/B testing strategically, they have doubled their revenue year over year, and have seen an over 280% conversion rate increase. Those numbers kinda speak for themselves, huh?

What do you think?

Do you agree with the myths above? What are some misconceptions around A/B testing that you would like to see debunked? Let us know in the comments!

The post “The more tests, the better!” and other A/B testing myths, debunked appeared first on WiderFunnel Conversion Optimization.

Excerpt from:

“The more tests, the better!” and other A/B testing myths, debunked