Tag Archives: interview

Learn from the Best: an Interview with Content Marketing Rock Star Andy Crestodina

I first had the pleasure of working with Andy on a Kissmetrics blog post five years ago. A few months after his post was published, I looked at our traffic in Google Analytics and said: Whatever Andy touches becomes magic. The electric sparks that shoot off his finger tips as he types turn into thousands of social shares, ten of thousands of pageviews, and more importantly – unbelievable wisdom that his readers consume. Let’s get inside his head for a moment and learn a few new things! 1. In the current state of inbound marketing, are people getting suffocated by…

The post Learn from the Best: an Interview with Content Marketing Rock Star Andy Crestodina appeared first on The Daily Egg.

From:  

Learn from the Best: an Interview with Content Marketing Rock Star Andy Crestodina

Learn From The Best: An Interview With Product Designer Michael Wong

mizko portrait

Here at Crazy Egg, we’re infatuated with design. Graphic design, product design, UX, UI – all of it. And of course, there are a handful of designers that we really admire. Michael Wong is one of them. You can see Michael’s skill right away when you come face-to-face with his work. And since Michael is a product designer, you can bet your you-know-what he’s released some of his own products. Check out bukketapp.com We decided to reach out to Michael to ask him a few questions. What’s the best skill to have as a UX designer in today’s world? How…

The post Learn From The Best: An Interview With Product Designer Michael Wong appeared first on The Daily Egg.

Continued here: 

Learn From The Best: An Interview With Product Designer Michael Wong

Filling Up Your Tank, Or How To Justify User Research Sample Size And Data

Jen is presenting her research report to a client, who runs an e-commerce website. She conducted interviews with 12 potential users. Her goal was to understand the conditions under which users choose to shop online versus in store. The client asks Jen why they should trust her research when she has spoken to only 12 people. Jen explains her process to the client. She shares how she determined the sample size and collected and analyzed her data through the lens of data saturation.

See original article: 

Filling Up Your Tank, Or How To Justify User Research Sample Size And Data

How Agencies Should Approach Conversion Optimization for eCommerce | An Interview with AWA Digital

Going ahead with our interview series, this time we are in conversation with Johann Van Tonder from AWA Digital.

Johann, is the COO at AWA Digital, a leading international Conversion Optimization (CRO) agency, specializing in eCommerce.

He is also the coauthor of the book E-commerce Optimization, out in January 2017. He speaks about how they practice conversion optimization for different verticals of their eCommerce clients.

After reading this post, agencies will learn the nuances of CRO when applied to different eCommerce verticals such as Fashion, Homeware, and Consumer Electronics. They’ll learn CRO strategies that will help them make a stronger case for adopting CRO for their prospective eCommerce clients.

Introduction

1) How important do you think Conversion Optimization (CRO) is for eCommerce enterprises? Why?

CRO is one of the best growth strategies available to eCommerce firms. Turnover is not something you influence directly. It is the outcome of activities performed in other areas. The rate at which people buy from you and how much they spend when they buy, are within your control. In turn, these will increase the revenue and ultimately profit. This is what CRO is about.

On average, for every £92 spent on getting traffic to UK websites, only £1 is spent on improving the conversion rate. If you improve the ability of your site to generate money, your acquisition dollars stretch further as more of the visitors are converted to buyers.

2) Is there a difference in your approach to CRO for different eCommerce verticals? If yes, how?

Not really. We always follow an evidence-led approach informed by research, data analysis, and testing. That said, our implementation will not be the same on two projects as we are guided by the opportunities specific to that particular website.

As long as you follow the scientific method, which we outline in our book E-commerce Optimization (Kogan Page), the same approach can generally be applied across different verticals. Broadly speaking, it’s a system of generating and prioritizing relevant ideas, and a mechanism by which to test those ideas.

3) Which are the major eCommerce verticals that you have worked with?

We have extensive experience in the fashion retail industry, having worked with top clothing and footwear brands from different countries. Furniture and homeware are two other categories we are well-known for.

Other big verticals for us include consumer electronics, flowers, gardening, gifting, health products, and outdoor travel gear. Our entire portfolio ranges from bathroom fittings to wearable technology.

Conversion Optimization for Different eCommerce Verticals

4) Do your CRO goals (micro and macro) differ for Fashion, Homeware and Consumer electronics based eCommerce businesses?

Our philosophy is to optimize for revenue, so in almost all cases, the primary metric is Revenue Per Visitor (RPV). If it’s an eCommerce business, Conversion Rate simply doesn’t give you the complete picture.

Secondary metrics, aligned with micro goals, vary widely. These are typically determined by the context of the experiment, rather than the vertical. For example, on a product detail page (PDP), you might want to track clicks on “add to basket” and engagement with product images. It helps to interpret the outcome of the test.

Sometimes we track key performance indicators (KPIs) outside of the testing environment. For example, experimenting with free delivery for a fashion client, we tracked product returns and married this data manually with test results.

5) What are the main “conversion funnels” for these different eCommerce websites? Do you see a difference in major traffic sources for the websites?

It’s not uncommon to see organic search being the major source of traffic for known brands. Often, the lion’s share of that is branded search terms, so in a way, it’s an extension of direct traffic. When a business is still establishing its brand, you’d expect to see more from paid search and other channels.

Many agencies limit optimization efforts to the website, which is a mistake. Social is an exciting area for some businesses, often rich with opportunities. Email consistently delivers good results for many of our clients and therefore, any gains in this arena can have a significant impact on overall business results.

Omni-channel, where we have a lot of experience, adds different dynamics. Not only do you see more direct traffic at the top of the funnel, but a large group of website visitors tend to browse with the intention to buy in-store. Or they may buy online, but only after visiting a store to “touch and feel” the product.

It’s important for the optimizer to take into consideration the entire journey, mapping out how the various touch points contribute to the purchase decision.

6)  Which persuasion principles (scarcity, social proof, reciprocity, etc.) do you use in optimizing different eCommerce vertical websites?

We regularly use social proof, liking, authority, and scarcity. It depends entirely on the situation. We don’t actively look for ways to plug them in. Instead, we examine the data and use a principle if it seems like a relevant solution. For example, one of our clients sells plants by catalogue and online. A common sales objection was whether the flowers would look as good in the customer’s garden as they are in the product images. This prompted us to invite customers to submit pictures of products in their gardens, invoking the social proof principle.

Once we’ve decided to use a principle, we may run a few tests to find the best implementation.

If a principle is already present on the website, there could be ways of making it more persuasive. In some cases, a message can be distracting in one part of the funnel yet very effective in another area of the site.

7) Which are the common conversion killers for these different eCommerce enterprises?

Some are universal, for example, delivery. Not only do consumers generally resist paying for shipping, but long waiting periods put them off. If you charge for it, you have to treat it like a product with its own compelling value proposition.

In the fashion industry, it’s size and fitting. Will these boots fit me? How will this shirt hang on me? Is your size 8 the same as another manufacturer’s size 8? These are the common themes. Typical concerns in the furniture and homeware space are material composition, dimensions, and perspective.

Sometimes we’re surprised by what we uncover. One of our clients, a gifting site, had a great returns policy. Obviously this was messaged clearly on the website. However, we discovered that it actually turned out to be a conversion killer for them. Why? Many of the buyers were grandparents who didn’t want to contemplate the prospect of their grandchildren returning their gifts.

8) The buying cycle for each eCommerce vertical website varies. Does your CRO strategy take this into account?

Definitely. The buying cycle is something we map out carefully.

For us, it is crucial to get under the skin of the customer. We want to understand exactly what goes into a sale being made or lost.

It can also inform our approach with testing. Normally we’d run an experiment for two weeks. However, if the purchase cycle is longer than that for the majority of customers, we may extend the test duration.

9) Does seasonality have an effect on your CRO strategy for different eCommerce verticals?

Clearly, some verticals are affected by it more than others. Seasonality is partly a reflection of customer needs. It is easier to deal with if you have a solid understanding of the core needs. In some verticals like gardening, it might be a good idea to conduct user research in low and high seasons.

Some clients are loath to run tests during peak trading periods like Christmas sales. Our view is that it is essential to optimize the site for those periods, especially if they contribute to annual turnover in any significant way.

10) On which eCommerce websites do you employ upsell/cross-sell strategies mostly?

Because our primary metric is usually Revenue Per Visitor rather than Conversion Rate, a driving question for us is how to increase Average Order Value. Upselling and cross-selling strategies are, therefore, almost always on our radar. We have had great success, for example, by optimizing the presentation and algorithms of popular product recommendation tools.

Upselling and cross-selling may be thought of as “tricking” the customer into spending more money. However, we’ve seen how frustrated customers become, having to hunt for items related to the one they are considering. It actually improves user experience, which is then reflected in an increase in revenue.

11) What CRO strategies do you apply on product pages of different eCommerce vertical websites (for instance, on product descriptions, product images, etc.)?

On most eCommerce sites, the product detail pages, or PDPs, have the highest drop-off rates on the site.

Exit rates in the region of 90%, and even higher are not uncommon. It is where the visitor is asked to make the decision. This is where questions about the product itself, as well as the buying process, are often most prominent.

We don’t have a checklist of tactics to apply to PDPs. Our test ideas emerge from research and analysis. If you understand the customer and what goes into the purchase decision, you’ll know what to test. Think of it as optimizing the sales conversation. It’s all about how you influence what plays out in the visitor’s mind.

  • Product description

If the visitors engage with product description, they may be closer to making a buying decision. Often this decision is based on the image, and reading the copy serves only to rationalize the purchase. Perhaps they are checking a few details or looking to answer a question about the product. The starting point for writing a good copy is knowing the customers and understanding their motivations and sales objections in relation to the product.

  • Product images

Likely to be the focus of most attention on the PDP. Often a substitute for reading product descriptions, so make sure you have a good selection of images that will answer key questions. On a lantern page, customers might wonder about the light patterns on their wall. Show them! Images appeal to System 1 thinking, which means purchase decisions are made instantly without thinking it over. Good images help the customer imagine themselves using the item, which can be quite persuasive.

Over to You

Do you have anything to add or suggest to this interview? Share with us what you think in the comments below.

CTA CRO program

The post How Agencies Should Approach Conversion Optimization for eCommerce | An Interview with AWA Digital appeared first on VWO Blog.

This article is from: 

How Agencies Should Approach Conversion Optimization for eCommerce | An Interview with AWA Digital

Not An Imposter: Fighting Front-End Fatigue

I recently spoke with a back-end developer friend about how many hours I spend coding or learning about code outside of work. He showed me a passage from an Uncle Bob book, “Clean Code”, which compares the hours musicians spend with their instruments in preparation for a concert to developers rehearsing code to perform at work.
I like the analogy but I’m not sure I fully subscribe to it; it’s that type of thinking that can cause burnout in the first place.

Continued here:  

Not An Imposter: Fighting Front-End Fatigue

All You Need to Know About eCommerce Conversion Optimization | An Interview with Tomasz Mazur

The following is an interview with Tomasz Mazur, a Conversion Rate Optimization expert, currently working as a consultant with Peaks & Pies.

Tomasz has several years of experience working on eCommerce conversion optimization and UX. He has previously worked with Zalando, Europe’s leading online fashion website.

Tomasz Mazur, Conversion Rate Optimization expert at Peaks & Pies

Tomasz answers our questions about conversion rate optimization from the perspective of eCommerce professionals.

Regarding the CRO Team and Sponsors

1) What does your ideal eCommerce CRO team consist of?

The team composition varies with the company. However, there are a few key members that every CRO team consists of (or should consist of):

  • A CRO manager who looks over the entire program—from strategy creation to website analysis and A/B testing
  • A design professional
  • A developer who can create functional test variations

The scale of the work of a CRO team mainly depends on website traffic. The objective is to make use of the entire website traffic always. Therefore, the greater the traffic, the larger a CRO team needs to be. Zalando, for instance, had a sizeable CRO team. I cannot disclose the exact number but there was a large number of CRO managers working on the entire website traffic. The web analytics team was also huge, working  on the minutest of details on the website such as product sorting algorithms. There was also a user research team that was in charge of qualitative research; the team used methods like usability labs, prototyping, focus groups, user interviews, and so on. As Zalando was operating in 15 markets and had a dedicated mobile website, the large size of the CRO team was justified.

2) Who should be the owner of a CRO program in an eCommerce organization?

The ownership of a CRO program should remain with someone from the higher management who understands the business impact of CRO. This person is the sponsor of the program. With understanding of the business impact of CRO, the sponsor doesn’t necessarily have to exhibit knowledge of all the nuances of CRO. The sponsor could be the VP of Marketing, or the VP of Product.

Sponsorship is necessary to ensure proper delivery of resources to a CRO team.

Regarding Coordination with Other Teams

3) Does a CRO team require help from other teams in an eCommerce organization?

One team that works closely with a CRO team is Marketing (especially, the customer acquisition department). The KPIs of the marketing and CRO teams are often aligned. The input of the marketing team is quite valuable—they know the product and the behavior of the customers.

The customer support team also plays a crucial role in highlighting pain points across a website. This team acts as a channel for customer feedback and helps a great deal in developing hypotheses.

A CRO team also needs to have the product team on its side. The product ownership mostly lies with the product team and their buy-in is essential. I know a few organizations where the product team has a “Conversion Lead” who acts as a bridge with the CRO team.

Regarding the CRO Process

4) What is the CRO process that you follow?

The process typically consists of the following steps:

  • Studying the quantitative and qualitative data of a website
  • Identifying problem areas on the website
  • Developing hypotheses that aim to address the problem areas
  • Creating variations per the hypotheses
  • A/B testing the variations
  • Analyzing the test results and sharing it with the team

Here is a graphic illustrating the process we follow in Peaks & Pies:

Screen Shot 2016-08-31 at 7.51.58 pm copy

5) How do you build a hypothesis?

I use website analytics tools such as Google Analytics to look for optimization opportunities across a website, for example catalog pages, product pages, and the checkout page.

When certain webpages are identified for optimization, these undergo rigorous analysis. I first check if a webpage has the essential eCommerce features (such as a recommendation engine on the home page) or not. If a certain feature is missing, we have got an opportunity for optimization. I use heatmaps and click-tracking tools to find elements or a functionality that require optimization. Gathering user feedback, taking help of usability testing labs, and checking out competitors’ websites are other ways of creating strong hypotheses.

I first check if a webpage has the essential eCommerce features (such as a recommendation engine on the home page) or not.

6) How do you prioritize A/B testing hypotheses?

I think a simple prioritization model like PIE—Potential, Importance, and then Ease—can work well for beginners. I personally like the model that is based on Potential, Confidence, and then Ease. This model gives a chance to the CRO team to take into account its past experience. A more sophisticated model can also include “political impact” as a criterion.

Moreover, prioritization needs to be a collaborative task to be successful. To estimate the “potential” of an A/B test, you might require the help of your marketing team. Similarly, the developers and designers can help you estimate the “ease” part.

7) What is the next step after hypothesis creation?

The next step is the design and development of a variation per the hypothesis.

However, before the variation is tested against a control, it must go through a thorough “quality assurance.” I think quality assurance is crucial for highly effective testing, but it is not emphasized enough in the CRO industry. You must make sure that all variations of an A/B test are free of bugs and issues.

Consider this: You are trying to improve a page that is already very optimized. You aim to achieve a humble 5% improvement in the conversion rate. If the variation doesn’t work for 5% of your traffic (because you forgot to optimize for mobile users or Firefox users), your test will invariably fail.

I think quality assurance is crucial for highly effective testing, but it is not emphasized enough in the CRO industry.

8) Is sales or revenue always the primary goal of your tests? Or do you look at micro conversions?

I would say the metrics or KPIs you are tracking should always answer your hypothesis.

In some cases, it is quite simple to track the revenue metrics (for instance, while adding new payment options on the checkout page). But in most cases, you have to track a combination of micro and macro metrics.

9) How do you analyze your A/B test results?

To derive valuable learning from a test, we need to conduct a thorough analysis.

I look at both micro and macro goals to get a better context of the results. I also dig deeper by analyzing the test results for different traffic segments. For instance, I compare test results for new visitors versus returning visitors. You need to deal with a concept called novelty effect. Your returning visitors, when encountering a test variation, will recognize the changes on the page and might hold strong feelings about it (similar to how people strongly respond to major changes happening on Facebook or Instagram). However, new visitors will be unbiased with your test variation and would interact without any prejudices. Another set of segments that is relevant to eCommerce is mobile and desktop visitors. The behavior of both kinds of users can vary significantly.

I dig deeper by analyzing the test results for different traffic segments.

An analysis is always followed by summarizing the test results and recommending a plan of action. You check if the hypothesis was valid and whether you need to implement the winning variation or run a follow-up test.

10) If you find that conversions increased for new visitors but decreased for returning visitors, what do you do?

You need to derive learning from the test and realize why the difference in user behavior exists. For example, this could be happening because of an offer for your new visitors such as “10% off for new visitors.” While the new visitors would be encouraged to shop on the website, the returning visitors might feel like they are not being offered the best deal and feel cheated. Your next step would be to set up a system to identify new visitors and display the “10% off on first order” deal exclusively for them. You can additionally have a loyalty program in place for your returning visitors.

The goal of CRO is to not just have winning A/B tests, but also gain more knowledge about your users so that you can provide them a superior experience. Think about the bigger picture—the entire eCommerce ecosystem. If you find that your close-up product pictures work better than zoomed-out ones, you can communicate this idea to your display marketing team and help them create better ads. If you find that the “free delivery” offer works better than a “10$ free voucher,” you can share the knowledge with other teams such as customer support and product.

11) How important is a long-term A/B testing calendar for high-traffic eCommerce websites?

I think a long-term A/B testing calendar is essential for any kind of website. You need to have a prudent approach; clearly define all the tests that you can conduct, along with the time that each test is going to take.

Think of time as a resource. If you are not testing all the time, you are wasting opportunities to optimize your website (the same time that your competitors might capitalize to move ahead of you). With a long-term calendar, you can easily identify time slots in which you can fit quick A/B tests and utilize all your resources effectively.

Here is a sample template that I would use as my CRO roadmap (you can access the template here):

Example Testing Roadmap Peaks Pies Google Sheets

12) How important is a knowledge repository of past A/B tests?

A knowledge repository is crucial to make your CRO program effective. It helps you know what works for your users (and what doesn’t) and helps you create better tests in the future. It is also used to introduce newcomers to the testing culture of an organization.

With every test you perform, it is important to document the details. You can start with a simple Google doc, and later get a sophisticated and comprehensive spreadsheet. Share the document with the entire CRO team so that everyone is on the same page and can avoid repeating any mistake. When the number of stakeholders is large, you can even think of a weekly/daily newsletter.

I usually archive my test results for every quarter.

Regarding the Nuances of eCommerce Conversion Optimization

13) Which eCommerce webpages do you think are key to a CRO program?

All of them. It’s important to go through the complete customer journey and find optimization opportunities. The most common customer journey path is from the home page to the catalog page to the product page to the cart page, and finally to the checkout page. Of course, there are other customer journey paths as well.

It’s important to go through the complete customer journey and find optimization opportunities.

However, a good CRO manager should always be able to identify low hanging fruits. I have seen that home page is the most tested page of eCommerce websites—mostly because it attracts the largest amount of traffic. Next in the list are product pages. These pages have a lot of traffic from channels such as affiliate partners and display ads. Third in the list are the register or login pages of websites.

14) With a large amount of traffic, how long do you run a test?

It is necessary to run an A/B test until it delivers a statistically significant result. However, with eCommerce websites, it is also important to consider the business cycle. For instance, the business cycle for fashion eCommerce is about one to two weeks. So even when a test delivers results in a few days, you need to run it for an entire business cycle to derive useful learning.

15) How many A/B tests do large eCommerce enterprises roughly run on a monthly basis?

That completely depends on the scale at which an eCommerce website is operating.

Large eCommerce enterprises like Zalando easily generate a traffic of millions of visitors in a single day. For example, Zalando had traffic from numerous markets and it could allow even 100 tests to run at an instance.

16) How does a festival season or a “sale period” affect a CRO process?

I have a good amount of experience working with fashion eCommerce. I’d say that the sale period definitely has an effect on their CRO process. One of the main goals in a sale period is to clear out the inventory. This is when the principle of “urgency” is deployed heavily in CRO campaigns.

This is when the principle of “urgency” is deployed heavily in CRO campaigns.

For other eCommerce websites, the heavy-traffic period can be markedly different. For example, I am working with an eCommerce enterprise that sells premium alcohol. The on-season period for them is winters when people tend to stay at home more and buy alcohol (for personal consumption or as a gift). As a result, this enterprise is focused on optimization activities more during the winter season.

Your Turn

What do you think about this interview? Do you have any question of your own that you would like to ask Tomasz? Post them in the comments section below.

VWO Free trial CTA

The post All You Need to Know About eCommerce Conversion Optimization | An Interview with Tomasz Mazur appeared first on VWO Blog.

View article:  

All You Need to Know About eCommerce Conversion Optimization | An Interview with Tomasz Mazur

Learn From The Best: An Interview With Basecamp Product Designer Jonas Downey

Believe it or not, a lot of your success when you’re running an online business comes down to how you approach design. Design is one of those subtle things that really make or break the experience you give your customers. We reached out to one of our favorite designers – Jonas Downey – to ask him a few questions. Sometimes getting into the head of a great designer can be the ticket to your next product breakthrough! 1. How do you define UX/design? UX has become such a nebulous, overused term — it can mean many different things in different…

The post Learn From The Best: An Interview With Basecamp Product Designer Jonas Downey appeared first on The Daily Egg.

Visit site – 

Learn From The Best: An Interview With Basecamp Product Designer Jonas Downey

What Does a CRO Program Consist of? | An Interview with Avast’s Michal Parizek

The following is an interview with Michal Parizek, Senior eCommerce & Optimization Specialist at Avast (a leading antivirus software company). Michal is a Conversion Rate Optimization (CRO) expert, having over seven years of experience across multiple industries.

Michal has created the popular Conversion Rate Optimization Maturity Model, where he illustrates the core assets of a successful CRO program for organizations operating at different scales. Using the model, organizations can understand the current state of their CRO efforts and identify ways to improve their programs.

This is how the CRO maturity model looks like:

Conversion Rate Optimization Maturity ModelThe questions in this interview aim to bring out actionable tips for organizations trying to bring structure to their CRO program.

Let’s begin.

Regarding the People and Culture Required for CRO

1) What are the essential skills or capabilities needed in a CRO team?

CRO is a complex discipline. In my opinion, it is a combination of a few, quite different skills.

  • One of them is analytics. To have a successful CRO program, you need to understand your data and derive insights.
  • Another skill is the user experience design. Being able to prototype great user experience is important.
  • Then marketing and copywriting—working on pricing strategies, composing an effective value proposition, and other relevant activities.
  • Other useful skills include statistics, consumer psychology, email marketing, and so on.

CRO can benefit from a range of skill areas. Therefore, it is very difficult to find great CRO consultants.

When you are building a CRO team, you should make sure you essentially hire an analyst, a UX designer, and a marketer/copywriter. These three I see as the key to driving an effective CRO program and results.

2) What does Avast do to create and spread a culture of CRO within the organization?

CRO has a good position in Avast culture. We are keen on A/B testing every major change in our sales flows or website. Data-driven decisions outbalance the gut-driven ones, even though there is a room for improvement. When I think about the reasons, I think there are mainly three things:

  • No data silos: Everyone can have access to pretty much every data in the company.
  • Sharing knowledge: We have been practicing A/B testing in the company for over four years, and the practice is now deeply rooted in our eCommerce department. Senior employees share their knowledge with the newcomers and help to spread the CRO culture.
  • Effectiveness: Particularly when Avast was much smaller than today, we counted literally every penny we spent. (Was the initiative worth the cost? How much did the $1,000 investment in that partnership return us?) Being small is an advantage since you need to monitor your spend and earning closely, and it naturally forces you to be more data-driven.

Regarding the Importance of a Sponsor

3) How does the absence of a sponsor affect an organization’s CRO program?

It makes things more difficult. It is not just about the budget, but also about time and resources. You are fighting two enemies at the same time, low conversion rate and your boss. It is not very easy to practice CRO in such circumstances. In the end, to be successful, you need to get the buy-in from the sponsor—either your boss or the management.

4) How can you convince the top management to back your CRO program?

From my experience, the strongest argument for a CRO program was always the “results.”

Speak in terms of dollars. Prove how much money your organization gained because of the CRO efforts. And explain how much money your company can gain in the future if you get more resources, time, or money. Don’t forget that tests with negative results are equally powerful since you can argue how much money you can lose if you did not practice A/B testing.

Regarding the Research Methodology in CRO

5) What is the importance of pre-test analysis or research?

It is the absolute key. If you just throw ideas to your A/B testing tool, your success rate will suck and you will waste time and resources. Arriving at hypotheses, scientifically, is essential. (In my previous job at Liberty Global, we did not pay a lot of attention to research and we were not very successful in our CRO activities.)

Also, the pre-test analysis can help you identify the test feasibility—if you are able to get results in a meaningful time and/or if you know how many variants you can afford to have.

6) What are the essential tools required for the research?

You can segregate them as quantitative and qualitative:

  • The quantitative tools include analytics, reports, heatmaps, and session recordings. They help you identify where the problem is.
  • On the other hand, the qualitative tools help you find out why people take certain actions on a website. These include usability testing, card sorting, surveys, interviews, focus groups, and so on.

7) What are some of the basic mistakes people often make with pre-test analysis?

The biggest (and a common) mistake is the absence of a pre-test analysis altogether. Then there are, I believe, the other common analysis mistakes: wrong interpretation of metrics, sampling issue ignorance, common sense absence, no test feasibility analysis, and more.

Every test specification should contain the research part—explaining why a particular test should be executed and what insights led you to the test idea.

From my experience, tests which have a solid research in the background have a higher success rate than the tests without any research.

Regarding A/B Testing Practices

8) How important is it to keep a long-term calendar for testing experiments?

A test calendar helps to focus on important tests being launched on time. It is also vital for resource planning and for bringing all stakeholders in the loop. We usually do a quarterly overview of what tests we’d like to run and then we specify and add details on a monthly basis.

9) How do you prioritize tests?

I often use a rather simple formula. First, I list all the possible tests we could run in a certain timeframe. Then, to every test I add two estimates:

  1. Effort: How many hours/days are required to execute the test? (It’s even better if you can translate that into monetary values.)
  2. Impact: How much money will be returned if the test is successful. Rather than thinking if the test can increase conversion rate by 10% or 15%, pay attention to where the test is running and what element you are changing. Do a pre-test analysis and calculate how many conversions a particular page generates. By using research or common sense, identify the importance of the changes. For example, in most cases, changing prices will have a bigger impact than changing button colors on a homepage. After you have executed several A/B tests, you will get an idea what matters and what does not. And this will help you set better expectations.

When all your tests are listed with effort and impact estimates, congratulate yourself. Now, it is easy. You execute the tests with the highest impact and least effort first. Then in the long term, you focus on the tests with a high impact, but also with great effort. In the meantime, you can execute the tests that don’t have a huge impact, but are easy to launch. And, you avoid ideas that require a lot of effort, but have almost no impact.

10) Can failed/inconclusive tests still provide value?

Yes, they can! If the test is designed and executed correctly (variants differ in one element, flawless measurement, sufficient data, no bugs in all variants, and so on), it provides great value regardless of the results. As long as you can learn from a test, it is a good test. Failed tests help you see which way you should not go. Inconclusive tests (again, if the tests are done correctly) tell you that perhaps the testing element does not matter much and you should test something else.

I really like what Ron Kohavi says,

“A valuable test is when the real results differ from your expectations. You learn the most in these cases and the learning matters the most in the long term.”

11) What are the common post-test analysis mistakes?

One of the common mistakes is not making sure if the test results are trustworthy or statistically significant. We sometimes tend not to analyze these thoroughly. The XX% lift always sounds appealing and we want to believe it (particularly when we have also designed the test—that’s why it is wise to always have somebody else to have a second look). But do we have enough data? Are the results consistent with time? Do we have traffic split balanced? Is the conversion lift driven by the change in design or just by chance? If we ignore chasing these findings, we can easily implement changes, which may not have any effect, or even worse, may decrease our performance.

The other common issue is not monitoring post-test results in the long term. Do we see the XX% lift after the winning variation has been implemented? Once we have a successful test, we tend to switch our attention to another issue and another test and we forget to monitor the previous effect.

Regarding CRO Tools

12) What are the key attributes based on which a CRO tool is chosen?

Usually, costs and benefits are the main attributes we look at. You must look at both costs and benefits from a wider perspective. Costs are not only the money you pay for the tool, but you also need to include implementation cost and maintenance cost (including the extra staff you need to hire to work on the tools).

Trying to estimate the business impact of the benefits of these tools is often challenging. Many CRO tools focus primarily on driving insights, and it is difficult to evaluate these in terms of $. Fortunately, many tools offer free trial versions, so you can get an idea on how useful they can be for you.

13) When would you say an organization should invest in developing in-house CRO tools?

When a tool is key to your business, and the cost of developing (and maintaining) the in-house tool outweighs the cost of having a third-party tool.

Regarding Coordination Between Teams

14) Does the CRO team need to coordinate closely with any other team in an organization?

It does need to cooperate with several teams. From my perspective, the following two are the key.

  • First, they need to have a close relationship with the business intelligence department to get correct data.
  • Second, they need to have a dedicated team of developers to execute the ideas that CRO team creates.
  • Then, there are many other vital cooperations. A support team has always been a great source of customer feedback; and for a CRO team, it is wise to be in touch with them. The product itself is a conversion asset so be in close touch with a product management team. How the product is marketed often defines the quality (and quantity) of leads and website visitors. Therefore, marketing is then another team I recommend being close to.

Your Thoughts

What do you think about the essential assets of a successful CRO program? Do you have any more questions for Michal? Post them in the comments section below.

Free-trial CTA

The post What Does a CRO Program Consist of? | An Interview with Avast’s Michal Parizek appeared first on VWO Blog.

Visit source: 

What Does a CRO Program Consist of? | An Interview with Avast’s Michal Parizek

On Design Systems: Sell The Output, Not The Workflow

So how do you sell a design system to the client? How do you establish a shared commitment within the company to put a pattern library on the roadmap? As designers and developers, we often know and see the benefits of an overarching system that radiates consistency throughout the different experiences of a company. But sometimes it’s seen as a very unpredictable investment, and the value isn’t necessarily visible right away.

View original: 

On Design Systems: Sell The Output, Not The Workflow

How To Land A First-Rate Graphic Design Internship

My first experience in the design world came through an internship at a small motion graphics studio called Motion Theory. I was fresh out of school and had never worked with so many talented people before. It was intense, difficult and nerve-wracking.
And I loved it.
It made me a better designer. And the lessons I learned there have served me well throughout my years as a freelancer. Because my experience was so rewarding, I’ve developed the habit of scrutinizing internship programs at every new studio I visit.

Link to article: 

How To Land A First-Rate Graphic Design Internship