Tag Archives: ecommerce conversion rate

Sort and Measure Method: Doing User Research from Product Reviews for eCommerce A/B Tests

 Note: This is a guest article written by Devesh Khanal, Founder of Growthrock.co. Any and all opinions expressed in the post are Devesh’s.

It’s well established now that elite conversion optimization firms and experts do extensive user research to generate high-quality test ideas. Without this research, companies will resort to just “gut instinct” and guesses, instead of having a list of evidence-backed hypotheses.

But the typical suite of conversion optimization “user research” tactics falls into a small set of familiar categories:

  • Use heatmaps/scrollmaps.
  • Survey users on site.
  • Watch session recordings.
  • Do live user testing.

These are perfectly fine methods of learning about your user. This article, however, will focus on a treasure trove of user feedback that most ecommerce sites are underutilizing despite having abundant amounts of it: product reviews.

Product reviews give you a window into the psychology of the most qualified customers ever, those who purchased.

You can learn:

  • What features or benefits do they value?
  • What features or benefits do they not value?
  • Why did they purchase?
  • What words and phrases do they use to describe the products and benefits?

It’s also free and fast; the reviews already exist on your site; and there is no need to pay for participants or wait to collect data.

But the key to successfully using product reviews is in having a method to analyze them quantitatively. So you can extract actionable, numerical takeaways, not just vague high-level ideas.

Here, we’re going to tell a story about a quantitative method of analyzing open-ended feedback like product reviews. We call this method the Sort and Measure method. The information you can extract from it helps uncover which benefits are most valued by customers which can help shape copy and site design choices. The ideas you extract from applying this method can lead to better, more data-backed AB test hypotheses.

Analyzing Product Reviews to Improve Product Page Copy

In this case study, we analyzed hundreds of product reviews for the site Amerisleep.com, a multimillion dollar online mattress company in the United States.


Using our observation, we were able to quantitatively rank how important different benefits were to the users—comfort, pain, relative temperature regulation, environmental manufacturing, cost, returns, shipping, and more.

Knowing how important these benefits were allowed us to craft detailed, long form product detail pages with confidence. Instead of guessing, we had real data that told us which benefits mattered and which did not.

If you have a 7- or 8-figure ecommerce brand, it’s possible that you can get the same results with this technique. Let’s meet the company.

Meet Amerisleep: An Online Mattress Leader with Multiple Product Benefits

Online mattresses is an ultra-competitive space right now. Amerisleep is a leading player in this space. It has a high-end mattress that it feels has several advantages:

  • (Comfort) Specially designed memory foam
  • (Options) Available in 5 different firmness levels
  • (Environmental) Manufactured in an environmentally friendly process
  • (Temperature) Designed to avoid overheating
  • (Cost) More affordable than retail memory foam mattresses
  • (Returns) They have a great “sleep trial” policy

Having lots of benefits sounds great, but it leads to unanticipated site design challenges.

The Problem: Which Benefit to Emphasize?

Imagine you are the VP of Ecommerce for Amerisleep and also in charge of AB testing new product detail page copy to improve sales.

It’s nice to have a lot of product benefits, but it means that you have some predicaments when improving your product detail pages:

  • Which of these benefits should you emphasize?
  • If you list all of them, will it overwhelm the shopper?
  • Do shoppers even care about [insert benefit]?
  • Which benefits should you put at the top of the page, and which down below?

How People Solve This: On-Site Surveys

How would you solve this problem and understand which benefits are more important than others?

The most typical ecommerce user research method for extracting this information is ask users in an on-site survey. This is a useful method and can work. You use a Hotjar, Qualaroo or VWO style poll and ask shoppers which benefits they care about.

  • You could ask an open-ended question: What are you looking for in a mattress?


  • You could ask a multiple-choice question: Which of these benefits of a mattress is most important to you?

This is a solid method of starting to understand your users. If you do this consistently, in our experience, you’d be ahead of a lot of competitors.

But this method has a key drawback—you don’t know how qualified the respondents are.

This is especially an issue for high-end brands. What if you get lot of responses about the price being too high, but the respondents weren’t your target customers and weren’t ever going to buy from you anyways?

This is where product reviews become extremely useful.

Product Reviews: You Already Have the Answers on Your Site

A great way to solve this problem is to survey only users who have bought the mattress. You can do this on the success page by asking: What were the key reasons you bought your mattress from us today? But the vast majority of successful ecommerce brands already have these survey results on their sites: these are in the form of product reviews.

Product reviews are exactly what benefits users consider important: it’s a survey of only customers who bought, and it’s open-ended and in their own words.

Note: The case study in this article used a buyer verification process for leaving reviews. If such a process doesn’t exist, it is possible that non-buyers could leave reviews. But in our experience, most stores have too few reviews, not too many.

Reviews also have a tendency of being brutally honest (something many store owners know too well), as many will include criticism of what they didn’t like and what could be improved, even though they bought anyways.

Sort and Measure Method: Extract Quantitative Answers to Make Decisions

But how are you going to turn product reviews into actionable information to inform your product page redesign or any other AB test on your site?

Enter the Sort and Measure Method. Here’s what it is, in a nutshell.

Casual Browsing: A Recipe for Personal Biases

Typically, what happens with survey data at an ecommerce company is that it’s emailed around and discussed a bit, and people then interpret it how they want.

One person says: “I told you, most of the reviewers were talking about back pain relief!”

Another employee says: “Well, what really struck me were the ones saying our environmental manufacturing was important.”

The debate can be never-ending. Most companies end up going with decisions based on either who is the loudest or who is the most senior.

For example, even before we used this technique, we did a large redesign and rewrite of their product detail pages. Using VWO to A/B test the redesign, we got a great, 14% lift in checkouts – a lift that’s worth 7 figures annually.

Although this was a great result, we had email exchanges like this between me and the copywriting consultant we partnered with, debating where to put emphasis. Here is an email I wrote to him about a certain benefit I felt wasn’t that important:


Here is his email to me:


Like I said, these debates are never-ending and get you nowhere. That’s why, while this test was running, we started using the Sort and Measure Method. Before the test was even over, we had a quantitative understanding of which benefits did and did not matter to Amerisleep customers (not prospects, but customers who actually bought).

Now, we don’t have to spend additional time and resources AB testing features and benefits that we know very few customers care about, and we don’t need to have endless email debates about what matters—we have data.

The Sort and Measure Method requires some manual effort, but it can be outsourced and the results are worth it. Here’s how it works.

Step 1: Sort

You list the product reviews in a spreadsheet:


Then you go through them one by one, create categories for common themes, and mark which categories they fall into.


It’s totally fine for a product review to be “sorted” into multiple categories.

As you work your way through the list, you’ll notice new categories and refine the list a bit, that’s okay. You’ll need to take a second pass to get it right.

Step 2: Measure

Now you just total up how many times certain categories are mentioned and you have your results, which any spreadsheet can do for you easily.

Here are the results for our analysis of Amerisleep’s product reviews, except that we’ve removed the benefit, to maintain data confidentiality.


(Yes, reviews are public information; and you can do this analysis for Amerisleep yourself, so it’s not really confidential…but will you? Probably not.)

Think about how useful this is. For the two smallest bars, we were able to clearly agree that this was not something happy customers talked about often.


That means:

  • We don’t need to have mentions of those benefits take up a lot of space, or be above the fold.
  • We should not prioritize AB testing to optimize sections that talk about these benefits.
  • We should not prioritize testing these benefits in the ad copy.

Benefits of the Sort and Measure Method

We were able to use this method to understand which benefits were valuable to customers and which were not. This understanding has allowed us to focus our testing efforts on the copy that emphasizes the benefits that matter to our customers. Thus, the Sort and Measure Method has become a useful tool in our user research arsenal.

Try this method yourself on your own store. Doing so will give you information on improving site functionality, help you identify friction points that could be hurting sales, and help you create better data-backed AB test hypotheses.


0 ratings

How will you rate this content?

Please choose a rating

The post Sort and Measure Method: Doing User Research from Product Reviews for eCommerce A/B Tests appeared first on VWO Blog.

Originally from – 

Sort and Measure Method: Doing User Research from Product Reviews for eCommerce A/B Tests

How To Increase Website Conversions With The Right Messaging

 Note: This is a guest article written by Josh Mendelsohn, VP of Marketing at Privy. Any and all opinions expressed in the post are Josh’s.

Let’s cut right to it. We all suck at conversion. According to E-marketer, 98% of online traffic leaves a site without filling out a form or completing a purchase. That means you have missed a chance to start building a relationship with potential customers. While it’s easy to shrug off a low on-site conversion rate, imagine if you owned a physical store and 100 people walked in… and 98 walked out without interacting with a represented or making a purchase. You’d be pretty sad, right? Yet, that’s what most of us are doing in our online stores and are not able to increase website conversions.

Why Do We Do This To Ourselves?

For starters, most organizations are thinking their product far more than they are thinking about conversion. If you’re a publisher, that might be the articles you are producing. If you’re an online store, it’s literally the products you are sourcing, merchandising, and selling. If you’re a non-profit, it’s the services you are providing to the world.

They are also likely thinking about how to drive site traffic. Whether that is through building a social media presence, paid search, radio, or even print ads.

And they may have even hired someone to think about the customer or member experience and how to keep those people engaged and generating word of mouth. But they often forget the middle, critical piece of the funnel, which is on-site conversion.

For the (much) smaller group of organizations who are actively trying to drive conversion, most fall into one of two camps. They either take a very passive approach because they don’t want to be too salesy. Or they take an overly aggressive approach with forms coming at a visitor from all angles, blocking a site’s core content.  But that’s not what good salespeople do. They take what they know about a prospect (in this case, a site visitor) and they use that to craft a message.

What We Know About Site Visitors

Through the magic of digital marketing, we know a lot about a site visitor without having to ask. While some people may find this creepy, for marketers it is an untapped goldmine of messaging opportunity.  For example, we can usually answer the question:

  • Where did they come from?
  • Is this their first visit?
  • What page are they on?
  • How many pages have they looked at?
  • What language do they speak?
  • What device are they on?
  • How much is in their cart?

What Do You Do With That Information?

While most organizations who have started thinking about conversion might have a simple opt-in form pop-up for visitors to their site, those who are focused on it can use the information we know to their advantage to create a more targeted experience for visitors to their site by crafting different messages based on who they are and what they have done. For the example below, I am going to imagine an E-Commerce company selling women’s clothing and I want to offer a 10% discount to new customers who sign up for my email list. While you probably wouldn’t want to hit someone with ALL of these messages, you can see how your core message might change based on what you know about a visitor.

Question What we know Messaging Strategy
Where did they come from? The visitor clicked on an Instagram ad featuring a specific blue swimsuit . Try featuring the product that they already expressed interest in within your message. “Looking for a new swimsuit? Get 10% off your first purchase by entering your email below.”
Is this their first visit? They have visited before but have never bought anything from you. Don’t treat them like a stranger! “Welcome back to my store! We’ve just launched a new product line. Sign up below to get 10% off your first purchase.”
What page are they on? They are on the “About” page of your site and not actually shopping. Try a “stay in touch” message over a discount. “Sign up to hear about new products and special offers.”
How many pages have they looked at?


How much is in their cart?

They have looked at 7 different pages in your store without adding anything to their cart, which means they are browsing but are not yet sold. “Having trouble finding what you are looking for? Sign up and we’ll let you know when we launch new products and give you a 10% discount for your first purchase.”
What language do they speak? The visitor’s primary language on their browser is spanish. “¡Bienvenidos a mi tienda! Regístrese abajo para obtener un 10% de descuento en su primera compra.”
What device category are they on? The visitor is on a mobile device, which is a great cue to slim down your text. “Sign up today for 10% off your first purchase.”

How To Deliver The Message

There are two things that you need to think about when delivering the message to your site visitors: timing and format. Let’s look at the format first :

1- Targeted displays – There are three categories of display types that drive the most on-site conversions.

– Popups: Popups, also known as lightboxes, typically display in the center of the website, or sometimes as “fly outs” in the corner.

– Bars: A full width bar that typically sits either on top of your site, or at the bottom.

– Banners: A more subtle interaction that sits at the top or bottom of a site, but starts in a “hidden” state until triggered, then rolls into sight at the desired time.

Pop-Ups for Increasing Website Conversions
Pop-Up for Targeted Display

More and more often, successful online stores are investing in automated and live chat to help reduce the anxiety that consumers feel before making a purchase from a new retailer. In fact, the availability of a “live” person on your site accomplishes two important goals:

– It allows people to ask any questions ahead of completing a purchase. Especially for larger ticket items, this inspires confidence that they are making the right decision

– It tells them that if something goes wrong with an order, there is a real person they can reach out to for help. The combination of those two factors makes shoppers more likely to hit the buy button.

Chat for Increasing Website Conversions
Engaging Visitors through Chat

3- Video
The third way of delivering the message that can have a huge impact on conversion is the use of video. Unlike static images and text, video helps bring your products to life and gives you the chance to both explain why someone should buy and put the product in a real life context. Or in some cases, lets you tell a broader story of how the product came to be in the first place.  Here’s an example of one I love (and am desperate to own.)

Product Videos for Increasing Website Conversions
Product Videos for Capturing Visitor Attention

Triggering Your Messages

The second consideration is deciding when to trigger each of your messages. There are four primary ways you can trigger a campaign to your desired audience.

  • Timer: The time trigger simply enables you to determine when to display your campaign, based on how long a visitor has been on your site. It could show immediately when a visitor lands, 10 seconds later, etc.
  • Exit intent: This trigger is growing in popularity. Exit intent tracks your visitors mouse movement, and if the visitor appears to be leaving or “exiting” your site, you can use that as a trigger for your campaign.
  • Scroll percentage: Show your campaign once a visitor has scrolled down your page a certain percentage.
  • Tabs: Tabs, or other visual calls to action can be customized to fit in with your site layout, and when clicked, trigger your campaign to display.

Which Converts Best?

Ultimately any combination of targeted messaging delivered through displays, videos, and chats will improve your conversion rate. We’ve looked at thousands of campaigns and found that each of the display types and triggers can be effective.  Because investing in video can take significant resources (time and money), I recommend starting with display and chat to deliver the right message at the right time. Once you have videos on hand, you can embed them on your product pages to level up your product content and add them into your displays to get them in front of shoppers as they navigate your site.

In terms of display types, banners are actually the highest converting format largely because they are less subtle than a simple “bar” but less frustrating to visitors than pop-ups that interrupt the browsing experience before a visitor has had a chance to consumer any of your content. In addition, we find that triggering a campaign in less than thirty seconds from the time a visitor lands on your site (or a specific page) is most effective in driving conversion.

Setting that data aside for a second, recent trends are showing that among the most impactful things you can do if you operate an online store is actually combining a pop-up with an exit intent trigger that serves as a “cart saver.” Simply put, if someone is visiting your store and attempts to leave by closing the browser tab or clicking the back button, you can show a message with a special offer that gets them to sign up and/or keep shopping while giving you permission to market to them in the future.

Exit Intent Pop-Ups to Increase Website Conversions
Exit Intent Pop-Ups

Walk. Jog. Run.

So, where do you get started? You don’t need to craft custom messages for every audience and every page on your site right out of the gate. We suggest thinking about one or two of your most common audiences and creating targeted offers and messages just for them that you can track, test, and adapt before rolling out a full on-site conversion program.


0 ratings

How will you rate this content?

Please choose a rating

The post How To Increase Website Conversions With The Right Messaging appeared first on VWO Blog.

Link to original: 

How To Increase Website Conversions With The Right Messaging

A Holistic Customer Retention Plan for eCommerce

While eCommerce enterprises realize the impact of customer retention on revenue, there are a number of key challenges that keep them from holistically implementing customer retention strategies.

One such challenge is the data complexity involved with calculating certain key metrics such as retention rate. As a result, businesses are making decisions per the metrics that might not be critical to long-term success. 2016 Retention Marketing report provides statistical evidence to why eCommerce organizations need clarity in their retention rate metrics.

Declining customer loyalty is affecting customer retention in a big way too. A (2015) research conducted by Accenture reports, “Just over one-quarter of U.S. consumers (28 percent) feel very loyal toward their providers and only about one in three (31 percent) are willing to recommend them to others.”

Competition demands eCommerce establishments to outgrow mediocrity and scale from average to elite. In this blog post, we highlight how optimizing their customer retention strategy (infused with actionable tips) can help eCommerce enterprises do exactly that!

Focus On Your High-Value Customers

To allocate marketing budget sensibly, eCommerce establishments need to understand where and what they are putting money and efforts into. Even before investing in a customer retention strategy, enterprises must focus on identifying their high-value customers. Centering retention efforts around high-value customers, as opposed to average customers, can yield considerably higher value in terms of RoI.

Digging into metrics such as customer lifetime value, average order value, purchase frequency, price sensitivity, and so on helps identify high-value customers. This post by eConsultancy talks about how you can do that.

Tracking and analyzing on-site behavior of high-value customers is also critical to optimizing the customer retention process. Tools like heatmap and visitor recordings can help understand collective and individual on-site user behavior respectively.

It is also important to understand the implication and usage of ‘churn rate’ correctly for your high-value customers. Instead of viewing it as a transactional metric, eCommerce enterprises should consider it to be a behavioral indicator. Harvard Business Review points out talks that enterprises are fixated with churn as a number instead of looking at it as a measure for improvement. The article also details on other mistakes that businesses make in using churn rate.

Actionable Tip: Use Net Promoter Score surveys to find out who your promoters, passives, and detractors are. Think of promoters as people who can drive more conversion and positive influence for your business. In this way, they are your high-value customers. Focusing efforts around retaining them is likely to lift your business value.

Net Promoter Score

Solicit Feedback And Act On It

Feedback helps fill the gap between what businesses offer and what their users actually want, in terms of products, services, and/or experiences. MarketingSherpa Ecommerce Benchmark Study 2014 establishes that companies who make changes to their website per customer feedback had a success score of almost 8/10.

Displaying encouraging reviews on the website, for example, can help eCommerce establishments increase not just conversions but also credibility—something that is extremely important for retaining customers long term. The Deloitte Consumer Review Consumer Review Report 2014 provides some interesting statistics in a graph that we have borrowed for the purpose of this post.

Deloitte Consumer Review Report

Testimonials and surveys are the two other indispensable feedback mechanisms. Surveying your repeat visitors on their satisfaction with your product/services, suggestions for improving service quality, and so on can help you identify areas of improvement for your business.

On the other hand, on-page surveys (OPS)/voice-of-customer surveys trigger questions real-time and typically aim at addressing pain points on websites. Nevertheless, choosing the right trigger for your OPS is critical to soliciting feedback. Some common examples of OPS triggers are listed below:

  1. Time spent on a page
  2. Number of pages browsed
  3. Exit intent on “adding items to cart”

Qualitative feedback obtained by OPS can also be of great value for optimizing purchase experiences, which can eventually increase retention. Here’s a VWO post – 10 tips on optimizing website surveys for soliciting better response.  

Actionable Tip: Negative reviews don’t hit a business as hard as not responding to them can. Moreover, customers value businesses that are quick in accepting and amending their mistakes. While not all negative feedback is authentic, genuine resentments provide valuable insights into the business or product aspects lacking in your business. Check out this article  for a complete approach on tackling negative reviews/feedback—right from digging into the problem to solving the problem to requesting customers for updating their review.

Continuously Scale Up On User Experience

Data about user behavior/activity as well as feedback lays the foundation for businesses to continuously improve user experience. However, it’s easier said than done. eCommerce enterprises not only need to continuously upgrade their technology, but also must provide consistent experiences to users across devices.

According to Mckinsey customer-experience survey 2014, “measuring satisfaction on customer journeys is 30 percent more predictive of overall customer satisfaction than measuring happiness for each individual interaction, establishing that consistency on the most common customer journeys is an important predictor of overall customer experience and loyalty.”

A customer journey can be considered as the sum of all experiences that customers go through while interacting with a website. It is an aggregate measure – good and bad – across different touchpoints.

That said, customer ‘satisfaction’ is not enough. Enterprises, in a digital world that is rapidly transforming, need to focus on customer delight and this requires them to be not just customer-centric but customer-obsessed. The Warehouse does this exceptionally well. By focusing on what really works for their customers, it has been able to increase their online sales from $18.8 million in 2011 to $149.2 million in 2015.

Most importantly, creating an exceptional user experience cannot be possible without personalization. eCommerce enterprises who want to move from being average to elite should focus on providing personalized services across human and digital touch points. The power of personalization can also be harnessed to form an emotional connect with customers, ultimately driving retention and long-term customer relationships.This would require identifying emotional motivators that customers associate with and then crafting personalized messages around those motivators.

Consider ‘convenience’ to be an emotional motivator. eCommerce enterprises can use personalized messages on their website focused around this emotion that propels people to buy. Warby Parker does exactly this by offering it’s customers a chance to try what they like before they buy.

Warby Parker

Actionable Tip: eCommerce establishments must focus on the 3 Es of experience – ease, efficiency, and emotion, as identified by Forrester Research. Storytelling is an effective tactic that can be used for establishing emotional connect with customers. Consider Everlane. The brand takes the story telling approach to communicate who and what they are, and what they do. They are able to establish trust and transparency – two values with which they establish an emotional connect with their customers.


Foster Relevance In Post-Sale Messages

Creating relevant post-sale messaging requires eCommerce enterprises to understand customer purchase history and behavior. It would be too soon to send out ‘We miss you here at…’ to someone who made his/her last purchase from you only a week back. What would be relevant instead is a cross-sell.

Here’s an example: A customer bought a boho necklace from your website. Now within a week of purchase, you could send an email to her/him, advertising the earrings available on your website that she could easily and perfectly pair with the necklace.

Moreover, not everyone who revisits or re-engages with your eCommerce website would have an intent to make a new purchase. You can give those sales emails a break. Let the goal be engagement.

This eMail from Boden is an excellent example on how to keep customers engaged even after a purchase.

boden thank you email

Your post-sale messaging strategy can include sending out newsletters, articles, and product usage tips.

Sold a peppy scarf recently? Your THANK YOU email can contain a video on 20 trendy ways to wrap a scarf in less than 5 minutes.

Actionable Tip: Another way of retaining your customers through post-sale messages is to keep sending them replenishment reminders (in case your store sells replenishable items). This would require you to keep track of the standard purchase cycle for the product and the customer’s average frequency of the product or order. Read some interesting examples of replenishment emails on this article on Ometria.

To Conclude

eCommerce enterprises must steer their efforts towards retaining those customers who can create more value for the business. An effective and optimized customer retention strategy that comprises understanding and analyzing user behavior, soliciting feedback, improving customer experience, and crafting relevant messaging, can help businesses retain such customers.

What are you doing to retain your valuable customers? Drop us your suggestions or tips in the comments section below.

VWO Cart Abandonment Report 2016

The post A Holistic Customer Retention Plan for eCommerce appeared first on VWO Blog.

View original article: 

A Holistic Customer Retention Plan for eCommerce

Creating Better A/B Tests for eCommerce | Visitor Behavior Analysis Use Cases

alt : http://feedproxy.google.com/~r/ILoveSplitTesting/~5/LClAaoFWRhc/vba-session-recording02.mp4http://feedproxy.google.com/~r/ILoveSplitTesting/~5/LClAaoFWRhc/vba-session-recording02.mp4

According to a recent report, the total eCommerce sales in the U.S. soared from $60.6 billion to $69.7 billion — by almost 15%.

As an eCommerce business owner, you would agree that these numbers are promising. However, where on one hand there is tremendous growth in sales, on the other, there are an increasing number of challenges.

With growing number of market players competing to win the largest chunk of eCommerce sales, the fundamental challenge for them is to reduce Customer Acquisition Cost (CAC) and increase Customer Lifetime Value (LTV).

According to the 2015 Benchmark report series on eCommerce buyer behavior, the cost of acquiring new visitors through channels such as PPC, social media, and search engines is rapidly surging. The best way for eCommerce marketers to deal with that is by converting more of their existing visitors into customers. They should create a seamless and friction-free experience for their visitors, not only making more visitors convert into customers but also retaining the existing customers.

Creating such an experience requires eCommerce businesses to understand how their visitors behave — what exactly are users looking for, what are their pain points, and what makes them buy.

Understanding User Behavior with Visitor Behavior Analysis Tools

While website analytics tools such as Google Analytics tell who your users are and what they are doing on your website, visitor behavior analysis tools such as Heatmaps, Visitor Recordings and Form Analysis help analyze all interactions users have with specific pages.

For example, from Google Analytics you might get to know that a user bounced off from the product page, but Visitor Recordings will show you the exact drop-off point from the page, i.e., product menu, search bar, etc. In the process of uncovering ‘how users behave,’ these tools dig out the pain points that users face while browsing, searching, or paying.

Understanding user pain points can help eCommerce businesses provide a user-friendly experience, which can ultimately lead to more conversions.

More often than not, eCommerce marketers try to address the website usability issue by running random A/B tests. Only a few smart ones have realized the power of visitor behavior analysis.

The post walks you through multiple use cases of visitor behavior analysis tools, spun out of eCommerce A/B testing case studies. It will help you understand how those tools help in creating smarter, data-backed hypotheses for A/B tests quickly.

Use Cases for Heatmaps

Use Case 1

Company: nameOn (Scandinavia’s leading supplier of personalized gifts with embroidery)

Objective: Increase conversions from the “add-to-cart page” to “checkout page.”

Hypothesis for A/B test: On observing a 31.7% drop-off rate between the add-to-cart page and checkout page, nameOn thought that reducing the number of CTAs on the page will reduce distractions for visitors, and allow them to focus on “continue to checkout” button.

The Test: On reviewing the add-to-cart page, nameOn noticed that it contained too many CTAs. Because nameOn wanted visitors to focus only on “continue to checkout,” the decision was made to retain only two CTAs — “continue to checkout” and “welcome bonus.” The test was run for 44 days for 621 visitors. You can read the entire case study here. Images of control and variation are given below:

Minimizing number of CTAs and reducing distractions - Case Study
Minimizing number of CTAs and reducing distractions - Case Study (2)

Running an Analysis with Heatmaps

With the help of Heatmaps, nameOn could have studied the number of clicks that each CTA on the page receives. With actual user insights derived from the analysis of Heatmaps and Clickmap (a type of Heatmaps), nameOn could have done the following:

  1. Study which areas on the page get the most attention.
  2. Discover which CTAs/hyperlinks on the page get the most/least clicks.

For example, nameOn could have studied clicks on all CTAs and only removed the ones that were considerably cannibalizing the primary CTAs.

Use Case 2

Company: Veggietales.com (eCommerce site for VeggieTales, a TV series for children)

Objective: To make people aware of the immense size of their followers on social media.

Hypothesis: The free shipping CTA overshadowed the social icons on the homepage. Increasing the size of the social icons and featuring the large follower count will act as a social proof, and increase the trust visitors have with the brand.

The Test: The control version of the homepage displayed a huge CTA for free shipping with social sharing buttons right below. Based on the hypothesis, VeggieTales replaced the free-shipping CTA with social sharing buttons along with the number of followers that VeggieTales had on Twitter and Facebook. You can read the entire case study here. Images of control and variation are given below:

Removing Free Shipping Banner - CTA
Removing Free Shipping Banner - CTA (2)

Running an Analysis with Heatmaps

VeggieTales could have studied the number of clicks on the free-shipping CTA on the header as well as the free-shipping CTA on the control page, using a Clickmap analysis.

Next, they could have decided whether doing away entirely with the free shipping CTA makes sense, or would it be wiser to simply shift the text below on the homepage. This way, they could have worked out another version that could get them more social engagement as well as retain the conversions that might have been lost because of completely removing the free shipping text on the right.

Use Case 3

Company: Buyakilt.com (an online kilt and Scottish highland dress retailer)

Objective: To improve click-throughs and engagement on the product category page.

Hypothesis: Adding a product filter on the page would increase conversions.

The Test: Buyakilt.com has a number of category pages that further have four sub-category pages. They added a product filter that gave visitors an option to shop by different kilt types, kilt patterns, etc. You can read the case study here. Images of control and variation are given below:

Removing sub-categories from product page and adding filter criterias - Case Study
Removing sub-categories from product page and adding filter criterias - Case Study

Running an Analysis with Heatmaps

In this case, apart from adding a product filter for simplifying the search process, Buyakilt could have used element list analysis (explained below) to study the number of clicks on each element under the categories.

An element list analysis is a type of Clickmap analysis which gives the actual number of clicks on both visible as well as hidden elements in a list (e.g., a drop-down list). This data can be further used for optimizing the category menu. For example, elements that receive maximum clicks can be placed on the top of the list, and those with lesser clicks can be placed lower on the list.

Use Cases for Visitor Recordings

Use Case 1

Company: Drukwerkdeal (an online printing shop based out of the Netherlands)

Objective: To push more checkout actions from the product pages.

Hypothesis: Drukwerkdeal hypothesized that the cross-sell messages were doing more harm than good. Upon reviewing a number of its product pages, Drukwerkdeal realized that their cross-selling messages were not very convincing.

The Test: Drukwerkdeal decided to remove these messages from some of the page categories and test if this would have any impact on the sales. 14,000 visitors participated in this test that was run for two weeks. Read the entire case study here. Images of control and variation are given below:

Cross-sell message removed from product page - Case Study
Cross-sell message removed from product page - Case Study

Running a Visitor Recording Analysis

Although upsell and cross-sell help boost eCommerce conversions, at times these strategies might backfire. Using Visitor Recordings, businesses can find out whether their upsell or cross-sell campaigns are getting them the desired results, or are proving to be a distraction. These recordings give accurate data about users’ entire journey on a website — where exactly they dropped-off, where they spent maximum time, from which point on a page they bounced off, etc. Based on this data, website owners can tailor an improved experience for users.

Here’s an example of Visitor Recordings:

In Drukwerkdeal’s case, before removing the cross-sell messages from the product page, they could first analyze how people interact with the product pages. Using Visitor Recordings, they could have played back the entire session of visitors who interacted with the product page. They could then validate whether users who clicked on a link in the cross-sell message returned to the original product page or not. It could then be established whether actually the cross-sell messages were proving to be a distraction on the product page.

Use Case 2

Company: Yuppiechef (a leading online store selling premium kitchen tools)

Objective/Conversion Goal: Increase the rate of wedding registry signups on one of their landing pages.

Hypothesis: Removing navigation menu would increase the rate of wedding registry signups.

The Test: Yuppiechef created a variation of their landing page without a navigation bar and tested it against the control. You can read the entire case study here. Here’s how the control and variation for Yuppiechef looked like:

Removing navigation bar from eCommerce landing page - Case Study
Removing navigation bar from eCommerce landing page - Case Study (2)

Running a Visitor Recording Analysis

By playing back the Visitor Recordings of users who visited the landing page, Yuppiechef would be able to playback the individual recordings of those visitors who browsed and clicked on the navigation menu but did not register.

They could also playback recordings of those who clicked on an item in the navigation bar but returned to register. Studying different kinds of interactions by different users, Yuppiechef could have used this research to validate that the navigation bar was serving as a distraction on the website.

Use Case 3

Company: UKToolCenter (an online store that offers tools & products for trade professionals and DIY experts)

Objective: To increase user engagement on a specific category of products on the website.

Hypothesis: By removing the filtering option on a product search, users will convert more.

The Test: UKToolCenter hypothesized that the filter menu was unnecessarily adding extra options for the users, and removing it could help increase engagement for a specific product category page (Cuprinol, a brand of woodcare products). They created a variation in which the product filter for Cuprinol products was removed. You can read the entire case study here. Take a look at their control and variation page:

Removing navigation bar from eCommerce landing page - Case Study
Removing navigation bar from eCommerce landing page - Case Study (2)

Running a Visitor Recording Analysis

While removing the filter from that specific product category increased user engagement on that product page by 27%, we have another case study where adding a product filter led to an increase in conversions.

Using Visitor Recordings, UKToolCenter could have studied user engagement on the product filter for Cuprinol. By playing back Visitor Recordings they could have identified visitors dropping-off from the Cupinol category page. By playing individual recordings, they could have validated whether the filter on the page was causing visitors to drop-off without browsing Cuprinol products at all.

Use Case for Form Analysis

Use Case 1

Company: Blivakker.no (Norway’s leading online beauty shop)

Objective: To increase form conversions on the registration form.

Hypothesis: Reducing number of form fields will increase conversions.

The Test: The original registration form on Blivakker.no consisted of 17 form fields. The company wanted to test if making a small change in the form would increase conversions on the form. Three form fields — phone number, account number and phone number (evening) — were removed from the web form. Another variation that was tested was a completely stripped down form with fewer fields and less navigational elements. You can read the entire case study here

Reduce form-fields - Case study
Removing three form-fields

Running a Form Analysis

While the test showed that removing unnecessary fields from the form increased the number of registrations, using Form Analysis could have shown the actual fields on the form where users dropped off or hesitated to fill in the information. Here’s a screenshot from VWO’s Form Analysis feature:

Screen Shot 2016-05-19 at 1.41.19 PM

Rather than relying on expert opinions, which can only be contextually right, using Form Analysis could have helped Blivakker.no to remove the fields that created most confusion and retain all the other ones, aiming to fetch more conversions and valuable user information, at the same time.

To Wrap Up

eCommerce businesses cannot rely on A/B testing random ideas for improving conversions. The need of the hour is to understand what their users are really looking for and what pain-points they’re facing.

Any eCommerce business that is able to identify what appeals or repels its users has an edge above its competitors. Using actual user behavior data, businesses can create smoother, friction-free experience for their users, and ultimately push overall conversions higher.

Let us know what you think about visitor behavior analysis tools. Post your comments below.

VWO free-trial CTA

The post Creating Better A/B Tests for eCommerce | Visitor Behavior Analysis Use Cases appeared first on VWO Blog.

Continue reading here: 

Creating Better A/B Tests for eCommerce | Visitor Behavior Analysis Use Cases

Just another WordPress site