Imagine that it’s a hot day. The sun is out, and the temperature is rising. Perhaps, every now and then, there’s a cool breeze. A good song is playing on the radio. At some point, you get up to get a glass of water, but the exact reason why you did that at that particular time isn’t easy to explain. It was “too hot” and you were “somewhat thirsty,” but also maybe “a little bored.” Each of these qualities isn’t either/or, but instead fall on a spectrum of values.
In contrast, our software is usually built on Boolean values. We set isHot to true and if isHot && isThirsty && isBored, then we call getWater(). If we use code like this to control our game characters, then they will appear jerky and less natural. In this article, we’ll learn how to add intelligent behavior to the non-player characters of a game using an alternative to conventional Boolean logic.
Today, we are talking about user research, a critical component of any design toolkit. Quality user research allows you to generate deep, meaningful user insights. It’s a key component of WiderFunnel’s Explore phase, where it provides a powerful source of ideas that can be used to generate great experiment hypothesis.
Unfortunately, user research isn’t always as easy as it sounds.
Do any of the following sound familiar:
During your research sessions, your participants don’t understand what they have been asked to do?
The phrasing of your questions has given away the answer or has caused bias in your results?
During your tests, it’s impossible for your participants to complete the assigned tasks in the time provided?
After conducting participants sessions, you spend more time analyzing the research design rather than the actual results.
If you’ve experienced any of these, don’t worry. You’re not alone.
Even the most seasoned researchers experience “oh-shoot” moments, where they realize there are flaws in their research approach.
Fortunately, there is a way to significantly reduce these moments. It’s called pilot testing.
Pilot testing is a rehearsal of your research study. It allows you to test your research approach with a small number of test participants before the main study. Although this may seem like an additional step, it may, in fact, be the time best spent on any research project.
Just like proper experiment design is a necessity, investing time to critique, test, and iteratively improve your research design, before the research execution phase, can ensure that your user research runs smoothly, and dramatically improves the outputs from your study.
And the best part? Pilot testing can be applied to all types of research approaches, from basic surveys to more complex diary studies.
Start with the process
At WiderFunnel, our research approach is unique for every project, but always follows a defined process:
Developing a defined research approach (Methodology, Tools, Participant Target Profile)
Pilot testing of research design
Recruiting qualified research participants
Execution of research
Analyzing the outputs
Reporting on research findings
Each part of this process can be discussed at length, but, as I said, this post will focus on pilot testing.
Your research should always start with asking the high-level question: “What are we aiming to learn through this research?”. You can use this question to guide the development of research methodology, select research tools, and determine the participant target profile. Pilot testing allows you to quickly test and improve this approach.
WiderFunnel’s pilot testing process consists of two phases: 1) an internal research design review and 2) participant pilot testing.
During the design review, members from our research and strategy teams sit down as a group and spend time critically thinking about the research approach. This involves reviewing:
Our high-level goals for what we are aiming to learn
The tools we are going to use
The tasks participants will be asked to perform
The research participant sample size, and
The participant target profile
Our team often spends a lot of time discussing the questions we plan to ask participants. It can be tempting to ask participants numerous questions over a broad range of topics. This inclination is often due to a fear of missing the discovery of an insight. Or, in some cases, is the result of working with a large group of stakeholders across different departments, each trying to push their own unique agenda.
However, applying a broad, unfocused approach to participant questions can be dangerous. It can cause a research team to lose sight of its original goals and produce research data that is difficult to interpret; thus limiting the number of actionable insights generated.
To overcome this, WiderFunnel uses the following approach when creating research questions:
Phase 1: To start, the research team creates a list of potential questions. These questions are then reviewed during the design review. The goal is to create a concise set of questions that are clearly written, do not bias the participant, and compliment each other. Often this involves removing a large number of the questions from our initial list and reworking those that remain.
Phase 2: The second phase of WiderFunnel’s research pilot testing consists of participant pilot testing.
This follows a rapid and iterative approach, where we pilot our defined research approach on an initial 1 to 2 participants. Based on how these participants respond, the research approach is evaluated, improved, and then tested on 1 to 2 new participants.
Researchers repeat this process until all of the research design “bugs” have been ironed out, much like QA-ing a new experiment. There are different criteria you can use to test the research experience, but we focus on testing three main areas: clarity of instructions, participant tasks and questions, and the research timing.
Clarity of instructions: This involves making sure that the instructions are not misleading or confusing to the participants
Testing of the tasks and questions: This involves testing the actual research workflow
Research timing: We evaluate the timing of each task and the overall experiment
Let’s look at an example.
Recently, a client approached us to do research on a new area of their website that they were developing for a new service offering. Specifically, the client wanted to conduct an eye tracking study on a new landing page and supporting content page.
With the client, we co-created a design brief that outlined the key learning goals, target participants, the client’s project budget, and a research timeline. The main learning goals for the study included developing an understanding of customer engagement (eye tracking) on both the landing and content page and exploring customer understanding of the new service.
Using the defined learning goals and research budget, we developed a research approach for the project. Due to the client’s budget and request for eye tracking we decided to use Sticky, a remote eye tracking tool to conduct the research.
We chose Sticky because it allows you to conduct unmoderated remote eye tracking experiments, and follow them up with a survey if needed.
In addition, we were also able to use Sticky’s existing participant pool, Sticky Crowd, to define our target participants. In this case, the criteria for the target participants were determined based on past research that had been conducted by the client.
Leveraging the capabilities of Sticky, we were able to define our research methodology and develop an initial workflow for our research participants. We then created an initial list of potential survey questions to supplement the eye tracking test.
At this point, our research and strategy team conducted an internal research design review. We examined both the research task and flow, the associated timing, and finalized the survey questions.
In this case, we used open-ended questions in order to not bias the participants, and limited the total number of questions to five. Questions were reworked from the proposed lists to improve the wording, ensure that questions complimented each other, and were focused on achieving the learning goals: exploring customer understanding of the new service.
To help with question clarity, we used Grammarly to test the structure of each question.
Following the internal design review, we began participant pilot testing.
Unfortunately, piloting an eye tracking test on 1 to 2 users is not an affordable option when using the Sticky platform. To overcome this we got creative and used some free tools to test the research design.
We chose to use Keynote presentation (timed transitions) and its Keynote Live feature to remotely test the research workflow, and Google Forms to test the survey questions. GoToMeeting was used to observe participants via video chat during the participant pilot testing. Using these tools we were able to conduct a quick and affordable pilot test.
The initial pilot test was conducted with two individual participants, both of which fit the criteria for the target participants. The pilot test immediately pointed out flaws in the research design, which included confusion regarding the test instructions and issues with the timing for each task.
In this case, our initial instructions did not provide our participants with enough information on the context of what they were looking for, resulting in confusion of what they were actually supposed to do. Additionally, we made an initial assumption that 5 seconds would be enough time for each participant to view and comprehend each page. However, the supporting content page was very context rich and 5 seconds did not provide participants enough time to view all the content on the page.
With these insights, we adjusted our research design to remove the flaws, and then conducted an additional pilot with two new individual participants. All of the adjustments seemed to resolve the previous “bugs”.
In this case, pilot testing not only gave us the confidence to move forward with the main study, it actually provide its own “A-ha” moment. Through our initial pilot tests, we realized that participants expected a set function for each page. For the landing page, participants expected a page that grabbed their attention and attracted them to the service, whereas, they expect the supporting content page to provide more details on the service and educate them on how it worked. Insights from these pilot tests reshaped our strategic approach to both pages.
The seemingly ‘failed’ result of the pilot test actually gave us a huge Aha moment on how users perceived these two pages, which not only changed the answers we wanted to get from the user research test, but also drastically shifted our strategic approach to the A/B variations themselves.
In some instances, pilot testing can actually provide its own unique insights. It is a nice bonus when this happens, but it is important to remember to always validate these insights through additional research and testing.
Still not convinced about the value of pilot testing? Here’s one final thought.
By conducting pilot testing you not only improve the insights generated from a single project, but also the process your team uses to conduct research. The reflective and iterative nature of pilot testing will actually accelerate the development of your skills as a researcher.
Pilot testing your research, just like proper experiment design, is essential. Yes, this will require an investment of both time and effort. But trust us, that small investment will deliver significant returns on your next research project and beyond.
Do you agree that pilot testing is an essential part of all research projects?
Have you had an “oh-shoot” research moment that could have been prevented by pilot testing? Let us know in the comments!
State Of Buyer Personas 2016 established that approximately 60% of the survey respondents took their first-ever buyer persona development initiative within the last 2 years—a result similar to the previous year survey on personas.
It has been almost two decades since the term “persona” was first coined and used by Alan Cooper in his book “The Inmates Run the Asylum.” However, organizations still struggle to develop personas effectively. As a result, the gap between what the consumer wants and what companies provide has widened.
Look at this survey graph for a quick look into the mistakes that can taint customer-business relations, when the latter does not know its ideal customer well:
In this blog post, we walk you through the process of creating effective personas, how your business can benefit from these, and why these should be a part of your conversion optimization strategy. Let’s begin:
How to Create Effective User Personas
To create personas that are effective, it is important to first understand what personas should not be:
A demographic profile
A market segment
A documentation of behavior based on a research that lacks data
Having listed the “nots” of personas, let’s deep-dive into what effective personas comprise and how to develop these. Research, qualitative and quantitative, is the foundation of personas. When based on research, personas unveil:
What your users want to accomplish?
What drives your users’ behaviors?
What do your users think?
What are their expectations?
What will make them buy?
What could be their reasons for hesitation?
What could be their hindrances?
To develop personas that can give you insights as deep as finding answers to the above questions, and a few more tough ones, we advise you to:
Use Qualitative Research.
Use A/B Testing.
Perform Competitor Analysis.
Using Qualitative Research
Qualitative research tools such as on-page surveys, in-person interviews, and so on can help you uncover the expectations and motivation of a user.
We list some use cases for on-page surveys to help you understand how these can be wisely used for gathering information that is required for developing effective personas:
Use Case 1. Understanding Purchase Decision
Understanding customer motivation for buying a product plays a significant role in replicating the buying behavior. If you knew precisely what motivated a visitor to buy from you, it is the next step to motivate other visitors in the same direction.
What you could ask?
Did you find what you were looking for?
What motivated you to complete your purchase?
What triggers to use?
Goal Completion: As soon as a user completes a signup form or makes payment for items in the cart, this survey should pop up to understand the true motivation behind the purchase.
Use Case 2. Determining Purchase Satisfaction
It is important to know the purchase satisfaction level to determine if there are reasons that can stop them from buying or make them buy from elsewhere. It can also help you categorize people who have high or low purchase satisfaction levels, if you are able to observe a pattern.
What you could ask?
On a scale of 0–10, 0 being the lowest and 10 being the highest, how satisfied are you with your last purchase?
The reason behind your rating. What do you think is good/bad about buying from us?
Analyzing the information that you have from your survey:
With regard to the question on purchase satisfaction levels, the information that your users reveal can be smartly analyzed to create user personas. Let’s say that you have an online apparel business. Running such surveys can help you:
Identify users who are not at all interested in your product (those who rate you between 0–3), users who do not have a firm opinion on your product (4-7), and users who have purchase satisfaction levels (8–10).
Understand the reasons behind high and low purchase satisfaction levels for all categories of users mentioned in the previous point.
Identify patterns, if any, in those rating your product high/low. For example, do those who rate the product on a scale of 8–10 buy the product because of “fresh styles and patterns,” do most of these people fall in the age group (20–25), and so on.
Build user personas based on this information.
What triggers to use?
Time spent on a page: Show the survey after visitors spend “X” seconds on the first webpage they visit. Target the survey using custom targeting to those who have made an online purchase earlier from you.
Asking these questions at the right time can help you fetch actionable information, uncover user motivations, as well as apprehensions.
Similarly, exit intent pop-ups and in-phone surveys also help you find out if your product/service is providing the value that your users and/or customers expect out of it.
Your qualitative research findings can then be dissected to create personas. Consider an example:
You are an eCommerce business selling antiallergic bedding. Your in-phone customer interview and on-page surveys help you determine one of your persona “Jane” with the following attributes:
Aged 32, she has very sensitive skin, which is prone to allergies.
She is willing to pay a little more if the product quality is good.
She also cares about the product being eco-friendly.
Your qualitative research would further help establish:
Jane’s motivation to buy your product: The bedding suits her needs, is priced just what she thinks is right, and can be found easily online.
Jane’s mindset while making a buying decision: She cares about her health and skin. She will not risk investing in any product that can cause allergies. She is also quality-conscious.
Jane’s bottlenecks to buying: She might return the product if she does not find it comfortable and per the quality that she expects. Style and comfort go hand in hand for her.
When you have conducted qualitative research and listed down motivations, bottlenecks, and mindset, you need to gather insights on what your user/customer is doing online. So the next logical step is to unveil Jane’s onsite behavior.
For example, using form analysis can help you identify the form fields that lead to customer hesitation or customers abandoning the form.
Using A/B Testing
Let’s say that you have listed a few findings about your personas, after conducting an in-depth research. However, you want to be as sure as possible. The following attributes can be put to test:
Comfort vs. Style
Discount vs. Buy One, Get One Free
Value of free shipping and free returns
A/B testing can help you narrow down to attributes as close as true to your real users. Whatever assumptions, observations, and opinions you have about your users, you can A/B test them to find out what your ideal users associate more with.
Performing Competitor Research
Digital intelligence tools can help you dig deeper into competitor data to analyze their traffic. Using such tools, you can find out where your competitors are putting their effort into—social media, mobile, content, email marketing, and so on.
After you have an idea of where your competitors’ major efforts go into, you can work backward to identify the audience they are targeting for creating user personas. This elaborate and well-researched post on medium will tell you how you can crack competitor research to create user personas for your business.
Benefits of Personas for Your Business
Mathilde Boyer, Customer Experience Director at the House of Kaizen, lists 5 ways in which every business can benefit by using personas.
“Personas shouldn’t only be created to trigger user empathy within an organization. They should be built with a practical application in mind so that they can be instrumental in a Conversion Optimization Strategy. Validating personas through actual user data and connecting them to target audiences increases their ability to drive business strategies.
Creating and leveraging user personas brings 5 key benefits to Marketers and Product Owners.
Connect research insights
Develop a unified view of your customers and prospects by identifying commonalities and unique attributes to provide a deep understanding of motivations, anxieties, decision making styles and moments when users find inspiration.
Strategically manage marketing budget
User personas allow you to prioritize target audiences and shift spend based on channel performance for individual audiences. Maximize your marketing investment by focusing your efforts and budget on the profitable leads.
Develop powerful brand and product storytelling
User personas can be leveraged to tailor storylines and bring your value proposition to life. They are key to understand aspirations, desires and perceptions of your customers. They are also crucial to strike the right note with unique content created to move buyers from interests to purchase.
Go beyond marketing silos
User personas allow you to ensure continuity and complementarity of messaging and creative across all user touchpoints (ads, website, emails, offline campaigns, customer service script, sales pitch, etc.).
Prioritize product roadmap
User personas should be a valuable levier to inform your product development cycles and ensure that new features are developed to solve evolving prospects’ problems and needs.”
Other than the benefits that Mathilde talks about, personas are also helpful in bringing uniformity to every department of the business regarding who their customer is. From customer service representatives to sales to marketing to the administrators, everyone is aligned to consumer goals. This helps everyone across the business keep their ideal customers happy, and thus increase overall satisfaction as well as retention.
Why Should Personas Be a Part of Your CRO Program
Protocol80 compiles some interesting facts on why personas are awesome. We list 2 of these here as evidence on why personas should be a part of your conversion optimization program.
“In the case of Intel, buyer personas surpassed campaign benchmarks by 75%. They were more cost efficient than the average campaign by 48% DemandGen Report.
In the case of Thomson Reuter, buyer personas contributed to a 175% increase in revenue attributed to marketing, 10% increase in leads sent to sales, and a 72% reduction in lead conversion time.”
Personas can help you improve conversions by:
Improving your personalization efforts.
Helping enhance product user experience.
Improving Personalization – Content
Personas help bring in more clarity on crafting tailored content that appeals to the target audience of the business. Consider an example:
You are an eCommerce business. One of your user persona is say, Mary – The Loyal, with some of the following characteristics:
Visits your website frequently
Makes a purchase every month or two
Does not purchase expensive products
Does not buy more than 2 or 3 products in a single visit
Is fashion-conscious, but does not compromise with quality
As you understand the buying behavior of this user persona, you can run campaigns with content specifically focussed at converting these users. For example, when Mary-the loyal visits your website again, you can personalize recommendations based on her last purchase, which might interest her into making a purchase.
Enhancing User Experience – Design and Development
At the design and development level, personas work as a research tool for businesses intending to enhance browsing/buying experience for their online users. These personas that are based on usage goals, browsing and exploring behavior, as well as pain points, tell the why behind the actions that users take on a website.
Such information is critical for designing any product or service. Understand, relate to, and remember the ideal user Mary-The Loyal throughout the entire product development process. The following design and development problems can be sorted by making user personas a part of the process.
When design teams do not have an understanding of which design elements on the website to prioritize. In this case, design and development teams end up wasting time on either developing or optimizing features that their ideal customer, Mary-The Loyal, does not use.
When design teams are finding it difficult to pitch their proposal to the management. This is where they can use actual data to enhance their idea and show the actual problem they are trying to solve by making the proposed changes.
Mathilde adds to how personas help enhance user experience.
“From a UX perspective, user personas are crucial to prevent self-referential design as they allow to focus the efforts on the needs of the customers and help be mindful of designing experiences as if we, marketers, were the end users.
Data-driven personas are also the foundation to map out customer journeys and ensure full alignment between user needs or perceived needs and the relevancy and length of the experience they have to go through to achieve them.
Personas become extremely powerful when they are taken beyond their naturally descriptive focus and provide a predictive view on how your product or service improves your ideal customers’ lives once they’ve used it for a certain time. The predictive side of personas is a key asset to design future-proof products and experiences.”
To Wrap It Up
When you make personas a part of your strategy, you are trying to maximize value for your ideal users. Here’s how Alan Cooper explains this concept in The Inmates Are Running The Asylum:
“The broader a target you aim for, the more certainty you have of missing the bull’s-eye. If you want to achieve a product-satisfaction level of 50%, you cannot do it by making a large population 50% happy with your products. You can only accomplish it by singling out 50% of the people and striving to make them 100% happy. It goes further than that. You can create an even bigger success by targeting 10% of your market and working to make them 100% ecstatic. It might seem counterintuitive, but designing for a single user is the most effective way to satisfy a broad population.”
Ultimately, filling the gap between the product value as perceived by your ideal user and the actual value that your product provides, will help you convince and convert your users into buyers.
When you develop a game, you need to sprinkle conditionals everywhere. If Pac-Man eats a power pill, then ghosts should run away. If the player has low health, then enemies attack more aggressively. If the space invader hits the left edge, then it should start moving right.
Usually, these bits of code are strewn around, embedded in larger functions, and the overall logic of the game is difficult to see or reuse to build up new levels.
Safari 10.1 was announced a while ago already, and this week it finally came to Macs and iOS devices around the world. The new Safari version ships CSS Grid Layouts, fetch(), IndexedDB2.0, Custom Elements, Form Validation, Media Capture, and much more.
In a world driven by the Internet, mobile apps need to share and receive information from their products’ back end (for example, from databases) as well as from third-party sources such as Facebook and Twitter.
These interactions are often made through RESTful APIs. When the number of requests increases, the way these requests are made becomes very critical to development, because the manner in which you fetch data can really affect the user experience of an app.
First of all, let’s define some vocabulary. “Internationalization” is a long word, and there are at least three widely used abbreviations: “intl,” “i18n” and “l10n.” All of them mean the same thing.
Internationalization can be generally broken down into three main challenges: Detecting the user’s locale, translating UI elements, titles as well as hints, and last but not least, serving locale-specific content such as dates, currencies and numbers. In this article, I am going to focus only on front-end part. We’ll develop a simple universal React application with full internationalization support.
We have great new technology available to enhance our websites. But while theoretical articles explain well what the technologies do, we often struggle to find real use cases or details on how things worked out in actual projects.
This week I stumbled across a couple of great posts that share exactly these precious real-life insights: stories about HTTP/2 implementation, experiences from using the Cascade of CSS in large-scale projects, and insights into employing Service Worker and BackgroundSync to build solid forms.
Every organization abides by a set of values and beliefs, which prompts the culture within the organization. This organizational culture can be seen as the way in which its members relate to each other, their work, and the outside world in comparison to other organizations.
So what type of cultural values should an enterprise invest in to be successful?
What Type of Organizational Culture to Adopt?
Company owners and decision-makers should invest in a culture of innovation and optimization during their business lifecycle.
Innovation, as they say, is the hallmark of entrepreneurship. In the early stages of development of a company, creative ideas and innovation alone bestow promising results. After that stage is crossed and you find a product-market fit comes the need for optimization. This is because when a product is reaching maturity, it needs to sustain itself and still keep earning profits.
Why to Adopt a Culture of Optimization?
Optimization, by dictionary meaning, is “an act, process, or methodology of making something (as a design, system, or decision) as fully perfect, functional, or effective as possible.”
For a mature and well-run product organization, the concept of optimization should be applied to every business process to gain optimal benefits. Whether it is your team, funnel, or website that you’re optimizing, you can be sure of making a bigger bang for the buck.
For instance, by optimizing your website, you can get more conversions for the same number of visitors. By optimizing your team, you can grow manifold the productivity and output of the same team members, and so on.
A culture of optimization is geared toward incremental, consistent, and risk-free improvements coordinated across company’s platforms to meet executive targets.
Tim Eyre, COO of a renowned digital marketing agency Big Leap, aptly puts it thus:
“Establishing a culture of optimization really starts with having an attitude of never being satisfied. For us, it means that we believe we can always improve the results we provide for our clients, whether through content pieces, paid search optimizations, or website optimization, including a heavy dose of conversion rate optimization. We strive for excellence in our work, and we have found that one of the quickest ways of approaching excellence is through testing”.
Now that we’ve established the importance of having a culture of optimization, let’s look at how to incorporate it within your organization:
How to Incorporate a Culture of Optimization
Before you begin optimizing your business processes, it is imperative to take stock of the current standing. You need to assess how you are placed currently and set benchmarks for improvements accordingly.
A good optimization program would be rooted in in-depth data and research of what we are optimizing and for which end goals. When your employees start questioning subjectivity and validate their decisions with both data and insight, it ultimately makes your organization a more efficient one.
“To facilitate a culture of testing, every aspect of an operation must have a unique data point that can be optimized against. We firmly believe that all areas of an organization, from personnel, to processes, to traffic and content, need to have a core KPI that can be measured for future improvement. Doing so creates a deeper level of accountability and a more proactive culture for experimentation and trying new approaches to do better.”
Get Top Management Buy-In
Like a waterfall, culture too flows from the top to the bottom. To inculcate a healthy culture of experimenting and optimizing, it is imperative that the top management complies.
If the top management is on the same page, it’ll realize the need for optimization and allocate sufficient budget for it. This will make it easier to execute any optimization program.
The idea of optimization can be sold to the top management by highlighting its key benefits. You can show how the returns on efforts outweigh the investment, and strengthen your pitch. Here are some key ways to influence your top management:
“If you want to incorporate a culture of Optimization, you need to allow and encourage anyone to challenge or at least question the data or ask how you could test an idea. But when someone says or asks something, be sure to ask them to back up their claim with data. They need to be able to validate any of their claims or thoughts, or at least give reasons for them.”
To make it motivating and straightforward for the team members, you should also take time to establish rules and incentives up-front.
You can also start a weekly ritual like a meeting or a report, something that will both analyze the progress of your optimization program and encourage its prosperity.
Invest in the Right Tools
To unroll the culture of optimization, you’d also be required to invest in the right tools. Having a tool in place can channelize and simplify your efforts manifold. Now the type of tool you should invest in depends on your optimization activity.