Tag Archives: model

Eating Our Own Dogfood – How To Optimize For Revenue As A SaaS Business

It wouldn’t be an exaggeration to say that we at VWO are very passionate about experimentation.

Not only have we built a product around A/B testing and conversion optimization, but we are always looking for ways to run experiments on our website.

Recently, we got our entire team to actively research and contribute ideas for optimization on our website and ran multiple tests. This post is a narrative of what we did after.

Who Is This Post for?

This post will help SaaS growth-hackers, marketers, and optimization experts to predict the business value from a test.

The aim of this post is to not only share the tests we ran on our website, but also introduce a revenue-based framework that predicts the business impact of an A/B test and prioritizing on the basis of it.

Revenue-Based Optimization

Need for a Model

After we propelled our team to suggest ideas for testing, we had more than 30 hypotheses looking at us, but no distinct way of knowing which of these to take up first. Of course, there is a range of prioritizing frameworks available, but we particularly wanted to look at the ones that would directly impact our revenue.

This framework helped us project the potential impact on the revenue from each test. Here’s what we did:

Step 1

We decided to identify high-impact pages and winnow the pages that were not as important for our business, that is, pages where no goal conversions take place. We looked at Google Analytics for pages with the:

  • Highest Amount of Traffic
    (We used “New Users” to nullify visits by existing customers.)
  • Highest Number of Goal Conversions
    (Goal conversion, which contributes to your overall business goal, is the main goal for your website. In our case, this meant all qualified lead-generating forms. A free trial or request a demo qualifies a visitor as a lead with a genuine interest in our product; or, as the industry popularly refers to it, a Marketing Qualified Lead.)

This gave us a list of pages which were high-value in terms of, either traffic generation or last touch before conversions.

We identified the following key pages:

  • Free-trial page
  • Request-a-demo page
  • Homepage
  • Pricing page
  • Features page
  • Blog pages (All)
  • Contact-us page

Step 2

Our main objective was to project an estimated increase in the revenue due to a particular test. If your test increases the conversion rate by say 20%, what would this mean for your business and, in turn, the revenue?

This is how our marketing funnel looked like:

VWO Marketing Funnel

Note: You should use data from the recent 3–6 months, and the average (mean) of each step. This is to accurately reflect what to expect from your testing and be relevant to your business.

For each of the “Key Pages” we identified in the first step, we also dug out the corresponding numbers at each funnel stage. We’ve explained each stage of the funnel and how it is calculated:

a) Key Page Traffic: The total number of pageviews per Key Page (new users in our case). You can find the data in Google Analytics.

b) Total Conversions: The total number of leads generated from each particular page. If there is an additional qualification your company follows, source this data from your preferred CRM or Marketing Automation software. For example, at VWO, we use Clearbit to qualify our leads in Salesforce.

c) Opportunities: The total number of opportunities generated for your sales team. This data will be available in your CRM; make sure to count qualified opportunities only.

d) Customers:  The total number of customers created in a month.

e) MRR (New): Or monthly recurring revenue, means revenue booked on a monthly basis; you can use this to estimate annual recurring revenue, or ARR, as well.

Step 3

Now that we had all the numbers needed in our arsenal, I decided to calculate some more internal benchmarks. This gave us the performance of our marketing and/or sales funnel.

  1. We computed the conversion rate of a particular page, using the following formula:
    Existing conversion rate = (Total Conversions Key Page Traffic); this is represented as %
  2. The conversion of your leads into opportunities:
    (Opportunities ÷ Total conversions) × 100, represented as %
  3.  The conversion rate of opportunities into customers:
    (Customers ÷ Opportunities) × 100, represented as %
  4.  The average revenue per user or ARPU:
    Total MRR  ÷ Total number of paying customers

Now all you have to do is to impute these numbers in this template.
Revenue-based Testing Model
The model uses all of that data and projects how much revenue increase or decrease you can estimate based on your test results. This estimate can give you a good idea of where to begin or prioritize your testing.

Step 4 (Optional)

This is where it may get tricky. At VWO, we sell both Enterprise plans and Standard plans. So to be fair, we must estimate each cohort with separate data and individual conversion rates.

For example, Opportunity creation % for an Enterprise plan may be lower, but a Standard plan is easier to convert. You may want to decide what type of plan do you want to focus on.

We, for instance, used website traffic and Alexa rank as the benchmark for lead qualification. We attributed more value to the leads that came in through key pages and prioritized them.

This led us to the next step, which is the qualification rate of the said lead of high value. This rate may be in the range 30–50%, depending on your definition.

It was interesting to note that each page had a different qualification rate. For example, we get better quality leads from our Request a demo page than we do from our free trial or blog post page.

Tests Conducted:

After we had the model in place, we played around with the increase or decrease in our conversion rates. This was to identify what would be our best optimization opportunities?

The free trial pages and the home page were among the high-priority pages, in terms of the impact of revenue. (Unfortunately, I can’t share the exact numbers with you.) We first looked at the hypotheses on the free trial page:

Test 1 – Free Trial Page

Our hypothesis was “Illustrating VWO features and social proof on the free trial page will compel users to sign up for the free trial.”

Here is a screenshot of what it looks like in VWO.
hypothesis-free-trial

Bonus tip: VWO has recently launched a new capability called PLAN that lets you manage and prioritize your testing hypotheses. To learn more about this capability, visit the VWO evolution page.

This is what the control looked like:

Free Trial Control

Our heatmap data also showed a lot of users clicking the features page after accessing the free trial page.

Screenshot of heatmap data:

Heatmap Screenshot for test

We created a variation which included the features we offer to solve this issue. Here’s a screenshot of the same.

This is our current free trial page:

Free Trial Page(New)(Variation)

We ran the test for over 2 months. The result was an increase of 6% in our conversion rate, which led to increased revenues.

Test 2 – Request a Demo CTA (A/B Test)

The main CTA on the homepage has been the free trial CTA. The headline on the homepage was “A/B Testing Software for Marketers.”

The hypothesis for the test was “We will get more qualified leads through a request a demo CTA on the homepage.”

This is what the control looked like:

Homepage Control

We came up with a more targeted copy and changed the existing CTA to Request A Demo. Here is what the variation looked like:

Homepage variation

We also wanted to change our positioning due to our foray into Conversion Optimization. The results from this test were that our variation beat the control and had more than 31% improvement in the conversion rate.

Based on the first example, we have already implemented the new free-trial page as our main free-trial page now. Based on the second test, we updated our current home page.

All in all, this model helped us correctly predict the best optimization opportunities, make our testing better, and more strategically aligned to business goals.

Let me know your experience with this model and how you go about testing.

Would love to hear your feedback on this!

Free-trial CTA

The post Eating Our Own Dogfood – How To Optimize For Revenue As A SaaS Business appeared first on VWO Blog.

Read more – 

Eating Our Own Dogfood – How To Optimize For Revenue As A SaaS Business

How to get evergreen results from your landing page optimization

Reading Time: 7 minutes

Landing page optimization is old news.

Seriously. A quick google will show you that Unbounce, QuickSprout, Moz, Qualaroo, Hubspot, Wordstream, Optimizely, CrazyEgg, VWO (and countless others), have been writing tips and guides on how to optimize your landing pages for years.

Not to mention the several posts we have already published on the WiderFunnel blog since 2008.

And yet. This conversation is so not over.

Warning: If your landing page optimization goals are short-term, or completely focused on conversion rate lift, this post may be a waste of your time. If your goal is to continuously have the best-performing landing pages on the internet, keep reading.



Marketers are funnelling more and more money into paid advertising, especially as Google allocates more and more SERP space to ads.

In fact, as an industry, we are spending upwards of $92 billion annually on paid search advertising alone.

landing-page-optimization-SERP-space
The prime real estate on a Google search results page often goes to paid.

And it’s not just search advertising that is seeing an uptick in spend, but social media advertising too.

It makes sense that marketers are still obsessing over their landing page conversion rates: this traffic is costly and curated. These are visitors that you have sought out, that share characteristics with your target market. It is extremely important that these visitors convert!

But, there comes a time in every optimizer’s life, when they face the cruel reality of diminishing returns. You’ve tested your landing page hero image. You’ve tested your value proposition. You’ve tested your form placement. And now, you’ve hit a plateau.

So, what next? What’s beyond the tips and guides? What is beyond the optimization basics?

1) Put on your customer’s shoes.

First things first: Let’s do a quick sanity check.

When you test your hero image, or your form placement, are you testing based on tips and recommended best practices? Or, are you testing based on a specific theory you have about your page visitors?

landing-page-optimization-customer-shoes
Put on your customer’s shoes.

Tips and best practices are a fine place to start, but the insight behind why those tactics work (or don’t work) for your visitors is where you find longevity.

The best way to improve experiences for your visitors is to think from their perspective. And the best way to do that is to use frameworks, and framework thinking, to get robust insights about your customers.

– Chris Goward, Founder & CEO, WiderFunnel

Laying the foundation

It’s very difficult to think from a different perspective. This is true in marketing as much as it is in life. And it’s why conversion optimization and A/B testing have become so vital: We no longer have to guess at what our visitors want, but can test instead!

That said, a test requires a hypothesis. And a legitimate hypothesis requires a legitimate attempt to understand your visitor’s unique perspective.

To respond to this need for understanding, WiderFunnel developed the LIFT Model® in 2008: our foundational framework for identifying potential barriers to conversion on a page from the perspective of the page visitor.

Get optimization ideas with the LIFT poster!

Get the LIFT Model poster, and challenge yourself to keep your visitor’s perspective in mind at all times. Use the six conversion factors to analyze your pages, and get optimization ideas!



By entering your email, you’ll receive bi-weekly WiderFunnel Blog updates and other resources to help you become an optimization champion.


The LIFT Model attempts to capture the idea of competing forces in communication, narrowing them down to the most salient aspects of communication that marketers should consider.

I wanted to apply the principles of Relevance, Clarity, Distraction, Urgency and Anxiety to what we were delivering to the industry and not just to our clients. And the LIFT Model is a part of that: making something as simple as possible but no simpler.

– Chris Goward

When you look at your page through a lens like the LIFT Model, you are forced to question your assumptions about what your visitors want when they land on your page.

landing-page-optimization-LIFT-Model
View your landing pages through a framework lens.

You may love an interactive element, but is it distracting your visitors? You may think that your copy creates urgency, but is it really creating anxiety?

If you are an experienced optimizer, you may have already incorporated a framework like the LIFT Model into your optimization program. But, after you have analyzed the same page multiple times, how do you continue to come up with new ideas?

Here are a few tips from the WiderFunnel Strategy team:

  1. Bring in fresh eyes from another team to look at and use your page
  2. User test, to watch and record how actual users are using your page
  3. Sneak a peek at your competitors’ landing pages: Is there something they’re doing that might be worth testing on your site?
  4. Do your page analyses as a team: many heads are better than one
  5. Brainstorm totally new, outside-the-box ideas…and test one!

You should always err on the side of “This customer experience could be better.” After all, it’s a customer-centric world, and we’re just marketing in it.

2) Look past the conversion rate.

“Landing page optimization”, like “conversion rate optimization”, is a limiting term. Yes, on-page optimization is key, but mature organizations view “landing page optimization” as the optimization of the entire experience, from first to last customer touchpoint.

Landing pages are only one element of a stellar, high-converting marketing campaign. And focusing all of your attention on optimizing only one element is just foolish.

From testing your featured ads, to tracking click-through rates of Thank You emails, to tracking returns and refunds, to tracking leads through the rest of the funnel, a better-performing landing page is about much more than on-page conversion rate lift.

landing-page-optimization-big-picture
On-page optimization is just one part of the whole picture.

An example is worth 1,000 words

One of our clients is a company that provides an online consumer information service—visitors type in a question and get an Expert answer. One of the first zones (areas on their website) that we focused on was a particular landing page funnel.

Visitors come from an ad, and land on page where they can ask their question. They then enter a 4-step funnel: Step 1: Ask the question > Step 2: Add more information > Step 3: Pick an Expert > Step 4: Get an answer (aka the checkout page)

Our primary goal was to increase transactions, meaning we had to move visitors all the way through the funnel. But we were also tracking refunds and chargebacks, as well as revenue per visitor.

More than pushing a visitor to ‘convert’, we wanted to make sure those visitors went on to be happy, satisfied customers.

In this experiment, we focused on the value proposition statements. The control landing page exclaimed, “A new question is answered every 9 seconds!“. Our Strategy team had determined (through user testing) that “speed of answers” was the 8th most valuable element of the service for customers, and that “peace of mind / reassurance” was the most important.

So, they tested two variations, featuring two different value proposition statements meant to create more peace of mind for visitors:

  • “Join 6,152,585 satisfied customers who got professional answers…”
  • “Connect One on One with an Expert who will answer your question”

Both of these variations ultimately increased transactions, by 6% and 9.4% respectively. But! We also saw large decreases in refunds and chargebacks with both variations, and large increases in net revenue per visitor for both variations.

By following visitors past the actual conversion, we were able to confirm that these initial statements set an impactful tone: visitors were more satisfied with their purchases, and comfortable investing more in their expert responses.

3) Consider the big picture.

As you think of landing page optimization as the optimization of a complete digital experience, you should also think of landing page optimization as part of your overall digital optimization strategy.

When you discover an insight about visitors to your product page, feed it into a test on your landing page. When you discover an insight about visitor behavior on your landing page, feed it into a test on your website.

It’s true that your landing pages most likely cater to specific visitor segments, who may behave totally differently than your organic visitors. But, it is also true that landing page wins may be overall wins.

Plus, landing page insights can be very valuable, because they are often new visitor insights. And now, a little more advice from Chris Goward, optimization guru:

“Your best opportunities for testing your value proposition are with first impression visitors. These are usually new visitors to your high traffic landing pages or your home page […]

By split testing your alternative value propositions with new visitors, you’ll reduce your exposure to existing customers or prospects who are already in the consideration phase. New prospects have a blank canvas for you to present your message variations and see what sticks.

Then, from the learning gained on landing pages, you can validate insights with other target audience groups and with your customers to leverage the learning company-wide.

Landing page testing can do more than just improve conversion rates on landing pages. When done strategically, it can deliver powerful, high-leverage marketing insights.”



Just because your landing pages are separate from your website, does not mean that your landing page optimization should be separate from your other optimization efforts. A landing page is just another zone, and you are free to (and should) use insights from one zone when testing on another zone.

4) Go deeper, explore further.

A lot of marketers talk about landing page design: how to build the right landing page, where to position each element, what color scheme and imagery to use, etc.

But when you dig into the why behind your test results, it’s like breaking into a piñata of possibilities, or opening a box of idea confetti.

landing-page-optimization-ideas
Discovering the reason behind the result is like opening a box of idea confetti!

Why do your 16-25 year old, mobile users respond so favorably to a one-minute video testimonial from a past-purchaser? Do they respond better to this indicator of social proof than another?

Why do your visitors prefer one landing page under normal circumstances, and a different version when external factors change (like a holiday, or a crisis)? Can you leverage this insight throughout your website?

Why does one type of urgency phrasing work, while slightly different wording decreases conversions on your page? Are your visitors sensitive to overly salesy copy? Why or why not?

Not only are there hundreds of psychological principles to explore within your landing page testing, but landing page optimization is also intertwined with your personalization strategy.

For many marketers, personalized landing pages are becoming more normal. And personalization opens the door to even more potential customer insights. Assuming you already have visitor segments, you should test the personalized experiences on your landing pages.

For example, imagine you have started using your visitors’ first names in the hero banner of your landing page. Have you validated that this personalized experience is more effective than another, like moving a social proof indicator above the fold? Both can be deemed personalization, but they tap into very different motivations.

From psychological principles, to validating your personalized experiences, the possibilities for testing on your landing pages are endless.

Just keep testing, Dory-style

Your landing page(s) will never be “optimized”. That is the beauty and cruelty of optimization: we are always chasing unattainable perfection.

But your landing pages can definitely be better than they are now. Even if you have a high-converting page, even if your page is listed by Hubspot as one of the 16 best designed landing pages, even if you’ve followed all of the rules…your landing page can be better.

Because I’m not just talking about conversions, I’m talking about your entire customer experience. If you give them the opportunity, your new users will tell you what’s wrong with your page.

They’ll tell you where it is unclear and where it is distracting.

They’ll tell you what motivates them.

They’ll tell you how personal you should get.

They’ll tell you how to set expectations so that they can become satisfied customers or clients.

A well-designed landing page is just the beginning of landing page optimization.

The post How to get evergreen results from your landing page optimization appeared first on WiderFunnel Conversion Optimization.

More: 

How to get evergreen results from your landing page optimization

Tips and tactics for A/B testing on AngularJS apps

Reading Time: 8 minutes

Alright, folks, this week we’re getting technical.

This post is geared toward Web Developers who’re working in conversion optimization, specifically those who are testing on AngularJS (or who are trying to test on AngularJS).

Angular, while allowing for more dynamic web applications, presents a problem for optimization on the development side.

It basically throws a wrench in the whole “I’m trying to show you a variation instead of the original webpage without you knowing it’s a variation”-thing for reasons I’ll get into in a minute.

At WiderFunnel, our Dev team has to tackle technical obstacles daily: many different clients means many different frameworks and tools to master.

Recently, the topic of How the heck do you test on Angular came up and Tom Davis, WiderFunnel Front End Developer, was like, “I can help with that.”

So here we go. Here are the tips, tricks, and workarounds we use to test on AngularJS.

Let’s start with the basics:

What is AngularJS?

Angular acts as a Javascript extension to HTML, running in most cases on the client-side (through the browser). Because HTML isn’t a scripting language (it doesn’t run code), it’s limited. Angular allows for more functionality that HTML doesn’t have. It provides a framework to develop apps that are maintainable and extendable, while allowing for features such as single page navigation, rich content, and dynamic functionality.

Note: You can mimic Angular with plain Javascript, however, Angular provides a lot of functionality that a Developer would otherwise have to build themselves.

Why is AngularJS popular?

The real question here is why are JS front-end frameworks and libraries popular? Angular isn’t the only framework you can use, of course: there’s EmberJS, React.js, BackBone etc., and different Developers prefer different frameworks.

But frameworks, in general, are popular because they offer a means of providing a rich user experience that is both responsive and dynamic. Without Angular, a user clicks a button or submits a form on your site, the browser communicates with the server, and the server provides entirely new HTML content that then loads in the browser.

When you’re using Angular, however, a user clicks a button or submits a form and the browser is able to build that content itself, while simultaneously performing server tasks (like database submissions) in the background.

For example, let’s think about form validations.

No Angular:

A user submits a form to create an account on a site. The browser talks to the server and the server says, “There’s a problem. We can’t validate this form because this username already exists.” The server then has to serve up entirely new HTML content and the browser re-renders all of that new content.

This can lead to a laggy, cumbersome user experience, where changes only happen on full page reloads.

With Angular:

A user submits a form to create an account on a site. The browser talks to the server via JSON (a collection of data) and the server says, “There’s a problem. We can’t validate this form because this username already exists.” The browser has already loaded the necessary HTML (on the first load) and then simply fills in the blanks with the data it gets back from the server.

Disclaimer: If you don’t have a basic understanding of web development, the rest of this post may be tough to decipher. There is a Glossary at the end of this post, if you need a quick refresher on certain terms.

Why it can be tricky to test on Angular apps

As mentioned above, Angular acts as an HTML extension. This means that the normal behaviors of the DOM* are being manipulated.

Angular manipulates the DOM using two-way data binding. This means that the content in the DOM is bound to a model. Take a look at the example below:

Testing on Angular_2-way-data-binding

The class “ng-binding” indicates that the H1 element is bound to a model, in this case $scope.helloWorld. In Angular, model data is referred to in an object called $scope. Any changes to the input field value will change helloWorld in the $scope object. This value is then propagated down to the H1 text.

This means that, if you make any changes to the H1 element through jQuery or native JS, they will essentially be overridden by $scope. This is not good in a test environment: you cannot guarantee that your changes will show up when you intend them to, without breaking the original code.

Laymen’s terms: $scope.helloWorld is bound to the H1 tag, meaning if anything in the variable helloWorld changes, the H1 element will change and vice versa. That’s the power of Angular.

Typically, when you’re testing, you’re making changes to the DOM by injecting Javascript after all of the other content has already loaded.

A developer will wait until the page has loaded, hide the content, change elements in the background, and show everything to the user post-change. (Because the page is hidden while these changes are being made, the user is none-the-wiser.)

Tom-Davis

We’re trying to do this switcheroo without anyone seeing it.

– Thomas Davis, Front End Developer, WiderFunnel

In Angular apps, there’s no way to guarantee that all of the content has been rendered before that extra Javascript is injected. At this point, Angular has already initialized the app, meaning any code running after this is outside of Angular’s execution context. This makes it complicated to try to figure out when and how to run the changes that make up your test.

When you’re running a test, the changes that make up Variation A (or B or C) are loaded when the page loads. You can only manipulate what’s in the DOM already. If you can’t guarantee that the content is loaded, how do you ensure that your added Javascript runs at the right time and how do you do this without breaking the code and functionality?

Tom explained that, as a dev trying to do conversion optimization on an Angular application, you find yourself constantly trying to answer this question:

How can I make this change without directly affecting my (or my client’s) built-in functionality? In other words, how can I make sure I don’t break this app?

How to influence Angular through the DOM

Angular makes for a complicated testing environment, but there are ways to test on Angular. Here are a few that we use at WiderFunnel (straight from Tom’s mouth to your eyeballs).

Note: In the examples below, we are working in the Inspector. This is just to prove that the changes are happening outside the context of the app and, therefore, an external script would be able to render the same results.

1. Use CSS wherever possible

When you’re running a test on Angular, use CSS whenever possible to make styling changes.

CSS is simply a set of styling rules that the browser applies to matching elements. Styling will always be applied on repaints regardless of how the DOM is bound to Angular. Everytime something changes within the browser, the browser goes through its list of styling rules and reapplies them to the correct element.

Let’s say, in a variation, you want to hide a banner. You can find the element you want to hide and add a styling tag that has an attribute of display none. CSS will always apply this styling and that element will never be displayed.

Of course, you can’t rely on CSS all of the time. It isn’t a scripting language, so you can’t do logic. For instance, CSS can’t say “If [blank] is true, make the element color green. If [blank] is false, make the element color red.”

In other cases, you may want to try $apply.

2. Using $scope/$apply in the DOM

We’ve established that Angular’s two-way data binding makes it difficult to develop consistent page changes outside of the context of Angular. Difficult…but not impossible.

Say you want to change the value of $scope.helloWorld. You need a way to tell Angular, “Hey, a value has changed — you need to propagate this change throughout the app.”

Angular checks $scope variables for changes whenever an event happens. An event attribute like ng-click or ng-model will force Angular to run the Digest Loop*, where a process called dirty checking* is used to update the whole of the app with any new values.

If you want to change the value of $scope.helloWorld and have it propagated throughout the app, you need to trick Angular into thinking an event has occurred.

But, how?

First step: You’ll need to access the model in the $scope object. You can do this simply by querying it in the DOM.

Testing on Angular_$scope

In this example, you’re looking at the $scope object containing all models available to the H1 element. You’re looking at the helloWorld variable exposed.

Once you have access to helloWorld, you can reassign it. But wait! You’ve probably noticed that the text hasn’t changed in the window… That’s because your code is running outside the context of Angular — Angular doesn’t know that a change has actually been made. You need to tell Angular to run the digest loop, which will apply the change within it’s context.

Fortunately, Angular comes equipped with an $apply function, that can force a $digest, as shown below.

Testing on Angular_$apply

3. Watch for changes

This workaround is a little manual, but very important. If the source code changes a variable or calls a function bound to $scope, you’ll need to be able to detect this change in order to keep your test functional.

That’s where Angular’s $watch function comes in. You can use $watch to listen to $scope and provide a callback when changes happen.

In the example below, $watch is listening to $scope.helloWorld. If helloWorld changes, Angular will run a callback that provides the new value and the old value of helloWorld as parameters.

Testing on Angular_$watch

Custom directives and dependency injection

It’s important that you don’t default to writing jQuery when testing on Angular apps. Remember, you have access to all the functionality of Angular, so use it. For complex experiments, you can use custom directives to manage code structure and make it easy to debug.

To do this, you can implement an injector to apply components in the context of the app that you’re testing on. Here’s a simple example that will alert you if your helloWorld variable changes:

For more details on how to use an injector, click here.

—–

These are just a few of the tactics that the WiderFunnel Dev team uses to run successful conversion optimization on Angular apps. That said, we would love to hear from all of you about how you do CRO on Angular!

Do you use the same tactics described here? Do you know of other workarounds not mentioned here? How do you test successfully on Angular apps? Let us know in the comments!

Glossary

DOM: The Document Object Model (DOM) is a cross-platform and language-independent convention for representing and interacting with objects in HTML, XHTML, and XML documents

$scope: Scope is an object that refers to the application model. It is an execution context for expressions. Scopes are arranged in hierarchical structure which mimic the DOM structure of the application. Scopes can watch expressions and propagate events.

$apply: Apply is used to execute an expression in Angular from outside of the Angular framework. (For example from browser DOM events, setTimeout, XHR or third party libraries).

JSON: (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition – December 1999

Two-way data binding: Data-binding in Angular apps is the automatic synchronization of data between the model and view components. The way that Angular implements data-binding allows you to treat the model as the single source of truth in your application.

Digest Loop: There is an internal cycle called $digest that runs through the application and executes watch expressions and compares the value returned with the previous value and if the values do not match then a listener is fired. This $digest cycle keeps looping until no more listeners are fired.

Dirty Checking: Dirty checking is a simple process that boils down to a very basic concept: It checks whether a value has changed that hasn’t yet been synchronized across the app

The post Tips and tactics for A/B testing on AngularJS apps appeared first on WiderFunnel Conversion Optimization.

Visit source: 

Tips and tactics for A/B testing on AngularJS apps

How to succeed at segmentation and personalization

I’m terrible at remembering names.

I was in high school when I first read How to Win Friends and Influence People by Dale Carnegie. One of the six ways he said to make people like you is “Remember that a person’s name is, to that person, the sweetest and most important sound in any language.”

win friends and influence people
According to Dale Carnegie, people love to hear their own names.

It’s true. People love the sound of their own name.

So, I’ve tried. And I’ll keep trying. In the meantime, please forgive me if I ask you to remind me of your name.

Luckily, marketing personalization is much more reliable than my brain. And it can be powerful.

Personalization, segmentation, and the broader concept of web customization, can have a great impact on web sales.

Understandably, it’s a hot topic and becoming more popular among retailers and lead generators. From simple message segmentation to programmatic ad buying and individual-level website customization, the combination of big data and ad technology is transforming the possibilities of personalization.

With so much content swirling around, marketers are scrambling to create messages that are more relevant to consumers. Segmentation and personalization can be seen as saviors in a content shocked world.

More and more often, WiderFunnel is testing and implementing personalization and segmentation methods for our clients too. It’s certainly a method that we always test.

Does your webpage deliver on what your prospects expect? Does the content match their specific needs and feelings? Segmentation and personalization allow you to deliver more relevant messaging to each of your prospects.

Without taking a strategic approach, though, many marketers are wasting a lot of time and effort (and cost on unnecessary tools) trying to find the holy grail of personalization. There are good reasons why segmentation without a proper strategy can hurt your business. I’ve also shown previously the 8 steps to creating and testing segmentation and personalization.

3 ways to create personalization and segmentation hypotheses to test

Today, I’m going to share with you the 3 ways WiderFunnel creates segmentation hypotheses. Together, these tiers form a framework you can use to guide your segmentation and personalization efforts.

First: what are personalization and segmentation?

Personalization and segmentation are often used interchangeably, and are arguably similar. Both use information gathered about the shopper to customize their experience.

I define segmentation in the context of conversion optimization as the process of putting structures in place to deliver appropriate messages to audiences with distinct needs and expectations. While segmentation attempts to bucket prospects into similar aggregate groups, personalization dynamically changes the prospects’ experience based on behavior, and usually uses individual details like the person’s name.

You can think of them as points along a spectrum of customized messaging.

Customized marketing spectrum
The customized marketing spectrum.

As you customize your mass marketing messages based on target audience groups, they become segmented. As more and more data allow you to create smaller segments, they eventually become a personalized experience with a single person segment.

Why should you use web personalization?

Unfortunately, too many people customize their websites for the wrong reasons, like…

  • More and more tools enable segmentation and personalization, so why not?
  • It sounds complex (which is attractive to marketers looking to be on the cutting edge)
  • ‘Segmentation’ and ‘personalization’ are powerful buzzwords in the industry
  • It gives marketers something to do with their personas

Those aren’t good reasons to use segmentation. But there is one good reason: segmentation helps solve a problem. The Relevance problem, to be specific.

Segmentation and personalization create Relevance

Ultimately, segmentation and personalization are aimed at improving Relevance (one of the 6 factors in the LIFT Model®).

Relevance - LIFT Model
Relevance is 1 of the 6 conversion factors detailed on the LIFT Model.

Barriers to Relevance can often be seen by looking at landing page bounce rates. The average bounce rate of landing pages is between 70% and 90%. For various reasons, your bounce rate doesn’t matter all that much, except when it comes to your target audience. For your target audience, bounce rates are vital.

The question then becomes:

How do I make experiences on my website sticky for each of my target audience segments?

This brings us back to Relevance. Segmentation and personalization are one set of tactics to improve the Relevance of what you’re presenting, provided you know how to communicate in a meaningful way for your visitors.

At this point, most marketers begin to think, “Great, what tools should I use to do that?” That’s a good question, but it’s not the only question you should ask.

Watch out for the tool-centric approach to segmentation

Technology is getting smarter and smarter. Most marketing automation tools boast some sort of segmentation capability. The leading A/B testing tool, Optimizely, recently launched their Personalization tool. And while it’s true that tools enable segmentation, they can’t design a great segmentation plan.

Personalization requires the same inputs and workflow as testing; sound technical implementation, research-driven ideation, a clear methodology for translating concepts into test hypotheses, and tight technical execution. In this sense, personalization is really just an extension of a/b testing and normal CRO activities. What’s more, successful personalization campaigns are the result of testing and iteration.

– Hudson Arnold, Strategy Consultant, Optimizely

In recent years, marketers have put a lot of trust in tools and algorithms to create marketing success. But, computers aren’t creative: they can detect patterns and facilitate solutions, but they can’t create new ideas.

Trusting your segmentation and personalization efforts to a machine means risking variation diarrhea, wherein the computer must test everything. When you have too many variations, spurious decisions are based on small data points – this is non-significance decision-making.

Some tool vendors use a lot of buzzwords and hand-waving while over-promising what the tool alone can deliver.

Buzzword marketing
Buzzwords > Other words = Fluff Marketing

As a rule of thumb, when buzzwords outnumber other words, there’s probably a lot of fluff marketing at work.

Just as importantly, computers can’t conduct insight-seeking results analysis. They can make small improvements using automated optimization algorithms, but they can’t glean the why behind the what.

Machine learning isn’t real learning. Real learning happens when you dig into the data, unearth insights and embed those insights into hypotheses, then test and validate the insights.

So, how do you do segmentation effectively? Below are 3 methods you can start using right now.

Best-guess segmentation

As you might’ve guessed, best-guess segmentation starts with your assumptions >> but it doesn’t end there. In best-guess segmentation, you feed those intuitive ideas into experiments that either validate or disprove your segmentation approach.

Start with how do I believe my customers should respond and how should my segments work? Ideally, you should have some data or rationale to back up your intuition. Then, test your segments with a segmentation split test.

Step 1. Maintain a non-segmented control group that sees your mass messaging.

Step 2. Create test groups based on various segmentation and personalization methods.

For example, here are some common methods for creating customized experiences:

  • Traffic source to landing pages
  • Inbound ad group
  • Browsing history
  • Cart contents leading to cross sells
  • Device and OS combination
  • Customer demographics and/or personas
  • Returning or new visitors
  • Time of day, week, or season

Step 3. Create your segmentation approaches, your actual segments, and specific messages and campaigns. It’s important to be very selective about the segments you test. Adding just 2-3 data sources can create many segment groups, which all need customized messages. The KISS principle applies.

Step 4. Test and track conversion rates across the segmentation hypothesis, not just across the segments. In other words, rather than just testing different messages for your segments, you should test different segmentation approaches to see which type of data shows response rate elasticity.

Step 5. Determine whether or not your hypothesis is validated (with 95% significance) and whether or not this hypothesis justifies supporting these segments on an on-going basis.

While best-guess segmentation begins with assumptions, iterative testing is the only way to be sure that your segmentation is actually effective and a worthwhile pursuit.

Be warned, the more you segment, the longer it takes to validate your test results. Try to find the largest possible relevant segments that can be aggregated. Read 6 reasons your over-segmentation is hurting your marketing for more on this risk.

User-selected segmentation

With user-selected segmentation, you’re asking your users to self-identify, segmenting themselves, which triggers specific messaging based on how they self-identified.

Here’s an example to help you visualize what I mean. WiderFunnel worked with a healthcare company for a few years and they used user-selected segmentation to drive more relevant content and offers. When visitors landed on any page throughout the website, they could self-identify in the following ways:

  • I am a physician treating [the disease].
  • I work at a hospital treating [the disease].
  • I manage [disease] while working.
  • I have late stage [disease].
  • I have early stage [disease].
user-selected segmentation
This healthcare company asked users to segment themselves.

Once a user self-identified, the offers and messaging that were featured on the page were adjusted accordingly.

You might be thinking, “He’s going to start saying ‘but you should test that’ pretty soon…” And you’re right. In user-selected segmentation, there are two big places you can test:

1) What are the best segments?
2) Is this the best messaging for each segment?

For this healthcare company, we didn’t simply assume that those 5 segments were the best segments, or that the messages and offers triggered were the best messages and offers. Instead, we tested both.

A series of A/B tests within their segmentation and personalization efforts resulted in a doubling of this company’s conversion rate.

Post-test segmentation

If you’re already optimizing your site, chances are you’ve seen segments naturally emerge through A/B testing. I call this post-test segmentation.

Post-test segmentation is driven by insights from your existing A/B test data. As you test, you discover insights that point you toward segmentation hypotheses.

Here’s an example from one of WiderFunnel’s e-commerce clients that manufactures and sells weather technology products. We ran a test on this client’s product pages, pitting the original page against our variation, which emphasized visual clarity.

Our variation lost to the original, decreasing order completions by 6.8%.

WiderFunnel Optimization Strategist, Nick So, was perplexed by the result and he dug into the data to find an insight. He found that the original page had more pages per session while our variation saw a 7.4% higher average time on page. This could imply that shoppers on the original page were browsing more, while shoppers on our variation spent more time on fewer pages.

Nick recalled an article published by the Nielsen Norman Group describing teen-targeted websites: research suggests that younger users enjoy searching and are impatient while older users enjoy searching but are also much more patient when browsing.

teen usability
Originally published by the Neilsen Norman Group.

So, Nick dug in further and found that our variation actually won for older users to our client’s site, but lost among younger users. So, what’s the takeaway? There are potentially new ways of customizing the shopping experience for different age segments, such as:

  1. Reducing distractions and adding clarity for older visitors
  2. Providing multiple products in multiple tabs for younger visitors

This client can use these insights to inform their age-group segmentation efforts across their site.

The best strategic testing process builds segmentation on insights

Always think of the bigger picture.

At WiderFunnel, segmentation and personalization are just a part of the 2 phases within the Infinity Optimization Process™: Explore and Validate. Insights gained from A/B testing inform future segments and personalized messaging, while insights derived from segmentation and personalization inform future A/B testing hypotheses. And on and on.

Don’t assume that insights gained during segmentation testing are only valid for those segments. Segmentation wins may be overall wins. The best practice when it comes to segmentation is to take the insights you validate within your segmentation test and use them to inform your hypotheses in your general optimization strategy.

The post How to succeed at segmentation and personalization appeared first on WiderFunnel Conversion Optimization.

Continue reading: 

How to succeed at segmentation and personalization