It’s 2018 already, and countless front-end developers are still leading a battle against complexity and immobility. Month after month, they’ve searched for the holy grail: a bug-free application architecture that will help them deliver quickly and with high quality. I am one of those developers, and I’ve found something interesting that might help.
We have taken a good step forward with tools such as React and Redux. However, they’re not enough on their own in large-scale applications.
Webinars are one of the most popular tools used by marketers for lead generation. Not only are they great for generating demand but they’re also a less pushy way of nurturing cold leads. The reason is that you are offering to provide information that your audience will value in your webinars. You can also demonstrate your expertise and showcase your knowledge of the industry and domain using webinars. However, webinars can be truly beneficial for your company if they are planned and implemented well. Here, we’ll take a look at some of the things you need to do to ensure…
Google Analytics (GA) is capable of generating incredibly detailed and comprehensive data. It provides the insights needed to fine-tune your site, reduce UX friction and ultimately maximize conversions. But there’s a catch. It’s only effective if you actually know how to interpret the data. Unfortunately, not all users fully understand the core metrics, and there’s uncertainty as to how to decipher them. Here, we’ll take a look at six of the most misunderstood metrics in GA to find out what the data means and how to apply it in order to optimize your site. 1. Direct Traffic At first…
(This is a sponsored post). Websites with long or infinite scrolling are becoming more and more common lately, and it’s no mere trend or coincidence. The technique of long scrolling allows users to traverse chunks of content without any interruption or additional interaction — information simply appear as the user scrolls down the page.
Infinite scrolling is a variety of long scrolling that allows users to scroll through a massive chunk of content with no finish line in sight (it’s the endless scrolling you see on Facebook, Twitter and Tumblr feeds).
Today, we are talking about user research, a critical component of any design toolkit. Quality user research allows you to generate deep, meaningful user insights. It’s a key component of WiderFunnel’s Explore phase, where it provides a powerful source of ideas that can be used to generate great experiment hypothesis.
Unfortunately, user research isn’t always as easy as it sounds.
Do any of the following sound familiar:
During your research sessions, your participants don’t understand what they have been asked to do?
The phrasing of your questions has given away the answer or has caused bias in your results?
During your tests, it’s impossible for your participants to complete the assigned tasks in the time provided?
After conducting participants sessions, you spend more time analyzing the research design rather than the actual results.
If you’ve experienced any of these, don’t worry. You’re not alone.
Even the most seasoned researchers experience “oh-shoot” moments, where they realize there are flaws in their research approach.
Fortunately, there is a way to significantly reduce these moments. It’s called pilot testing.
Pilot testing is a rehearsal of your research study. It allows you to test your research approach with a small number of test participants before the main study. Although this may seem like an additional step, it may, in fact, be the time best spent on any research project.
Just like proper experiment design is a necessity, investing time to critique, test, and iteratively improve your research design, before the research execution phase, can ensure that your user research runs smoothly, and dramatically improves the outputs from your study.
And the best part? Pilot testing can be applied to all types of research approaches, from basic surveys to more complex diary studies.
Start with the process
At WiderFunnel, our research approach is unique for every project, but always follows a defined process:
Developing a defined research approach (Methodology, Tools, Participant Target Profile)
Pilot testing of research design
Recruiting qualified research participants
Execution of research
Analyzing the outputs
Reporting on research findings
Each part of this process can be discussed at length, but, as I said, this post will focus on pilot testing.
Your research should always start with asking the high-level question: “What are we aiming to learn through this research?”. You can use this question to guide the development of research methodology, select research tools, and determine the participant target profile. Pilot testing allows you to quickly test and improve this approach.
WiderFunnel’s pilot testing process consists of two phases: 1) an internal research design review and 2) participant pilot testing.
During the design review, members from our research and strategy teams sit down as a group and spend time critically thinking about the research approach. This involves reviewing:
Our high-level goals for what we are aiming to learn
The tools we are going to use
The tasks participants will be asked to perform
The research participant sample size, and
The participant target profile
Our team often spends a lot of time discussing the questions we plan to ask participants. It can be tempting to ask participants numerous questions over a broad range of topics. This inclination is often due to a fear of missing the discovery of an insight. Or, in some cases, is the result of working with a large group of stakeholders across different departments, each trying to push their own unique agenda.
However, applying a broad, unfocused approach to participant questions can be dangerous. It can cause a research team to lose sight of its original goals and produce research data that is difficult to interpret; thus limiting the number of actionable insights generated.
To overcome this, WiderFunnel uses the following approach when creating research questions:
Phase 1: To start, the research team creates a list of potential questions. These questions are then reviewed during the design review. The goal is to create a concise set of questions that are clearly written, do not bias the participant, and compliment each other. Often this involves removing a large number of the questions from our initial list and reworking those that remain.
Phase 2: The second phase of WiderFunnel’s research pilot testing consists of participant pilot testing.
This follows a rapid and iterative approach, where we pilot our defined research approach on an initial 1 to 2 participants. Based on how these participants respond, the research approach is evaluated, improved, and then tested on 1 to 2 new participants.
Researchers repeat this process until all of the research design “bugs” have been ironed out, much like QA-ing a new experiment. There are different criteria you can use to test the research experience, but we focus on testing three main areas: clarity of instructions, participant tasks and questions, and the research timing.
Clarity of instructions: This involves making sure that the instructions are not misleading or confusing to the participants
Testing of the tasks and questions: This involves testing the actual research workflow
Research timing: We evaluate the timing of each task and the overall experiment
Let’s look at an example.
Recently, a client approached us to do research on a new area of their website that they were developing for a new service offering. Specifically, the client wanted to conduct an eye tracking study on a new landing page and supporting content page.
With the client, we co-created a design brief that outlined the key learning goals, target participants, the client’s project budget, and a research timeline. The main learning goals for the study included developing an understanding of customer engagement (eye tracking) on both the landing and content page and exploring customer understanding of the new service.
Using the defined learning goals and research budget, we developed a research approach for the project. Due to the client’s budget and request for eye tracking we decided to use Sticky, a remote eye tracking tool to conduct the research.
We chose Sticky because it allows you to conduct unmoderated remote eye tracking experiments, and follow them up with a survey if needed.
In addition, we were also able to use Sticky’s existing participant pool, Sticky Crowd, to define our target participants. In this case, the criteria for the target participants were determined based on past research that had been conducted by the client.
Leveraging the capabilities of Sticky, we were able to define our research methodology and develop an initial workflow for our research participants. We then created an initial list of potential survey questions to supplement the eye tracking test.
At this point, our research and strategy team conducted an internal research design review. We examined both the research task and flow, the associated timing, and finalized the survey questions.
In this case, we used open-ended questions in order to not bias the participants, and limited the total number of questions to five. Questions were reworked from the proposed lists to improve the wording, ensure that questions complimented each other, and were focused on achieving the learning goals: exploring customer understanding of the new service.
To help with question clarity, we used Grammarly to test the structure of each question.
Following the internal design review, we began participant pilot testing.
Unfortunately, piloting an eye tracking test on 1 to 2 users is not an affordable option when using the Sticky platform. To overcome this we got creative and used some free tools to test the research design.
We chose to use Keynote presentation (timed transitions) and its Keynote Live feature to remotely test the research workflow, and Google Forms to test the survey questions. GoToMeeting was used to observe participants via video chat during the participant pilot testing. Using these tools we were able to conduct a quick and affordable pilot test.
The initial pilot test was conducted with two individual participants, both of which fit the criteria for the target participants. The pilot test immediately pointed out flaws in the research design, which included confusion regarding the test instructions and issues with the timing for each task.
In this case, our initial instructions did not provide our participants with enough information on the context of what they were looking for, resulting in confusion of what they were actually supposed to do. Additionally, we made an initial assumption that 5 seconds would be enough time for each participant to view and comprehend each page. However, the supporting content page was very context rich and 5 seconds did not provide participants enough time to view all the content on the page.
With these insights, we adjusted our research design to remove the flaws, and then conducted an additional pilot with two new individual participants. All of the adjustments seemed to resolve the previous “bugs”.
In this case, pilot testing not only gave us the confidence to move forward with the main study, it actually provide its own “A-ha” moment. Through our initial pilot tests, we realized that participants expected a set function for each page. For the landing page, participants expected a page that grabbed their attention and attracted them to the service, whereas, they expect the supporting content page to provide more details on the service and educate them on how it worked. Insights from these pilot tests reshaped our strategic approach to both pages.
The seemingly ‘failed’ result of the pilot test actually gave us a huge Aha moment on how users perceived these two pages, which not only changed the answers we wanted to get from the user research test, but also drastically shifted our strategic approach to the A/B variations themselves.
In some instances, pilot testing can actually provide its own unique insights. It is a nice bonus when this happens, but it is important to remember to always validate these insights through additional research and testing.
Still not convinced about the value of pilot testing? Here’s one final thought.
By conducting pilot testing you not only improve the insights generated from a single project, but also the process your team uses to conduct research. The reflective and iterative nature of pilot testing will actually accelerate the development of your skills as a researcher.
Pilot testing your research, just like proper experiment design, is essential. Yes, this will require an investment of both time and effort. But trust us, that small investment will deliver significant returns on your next research project and beyond.
Do you agree that pilot testing is an essential part of all research projects?
Have you had an “oh-shoot” research moment that could have been prevented by pilot testing? Let us know in the comments!
AdWords can be an incredibly powerful platform for growing your business, but the truth is that most businesses are wasting thousands of dollars every month due to poor account management. If you feel like you could be generating more revenue from AdWords, chances are that you’re making some of these ten common mistakes that are inhibiting your performance. In this post I will be showing you ten common AdWords mistakes that I’ve seen literally hundreds of advertisers make. And, more importantly, I’m going to show you exactly how to fix them, so you can take next steps towards creating an…
The virtual realm is uncharted territory for many designers. In the last few years, we’ve witnessed an explosion in virtual reality (VR) hardware and applications. VR experiences range from the mundane to the wondrous, their complexity and utility varying greatly.
Taking your first steps into VR as a UX or UI designer can be daunting. We know because we’ve been there. But fear not! In this article, we’ll share a process for designing VR apps that we hope you’ll use to start designing for VR yourself.
There is a lot to learn this week. It starts with non-technical things like going for a walk to refresh your mind and finishes with how to prevent reverse XSS attacks in forms. But it doesn’t matter whether you learn how to build self-contained web components using the new specification or to maximize the efficiency of your Angular 2 app or just how you can write less code. What matters is that you keep asking questions and that you try to get better and smarter at your craft.
One of the biggest risks of building a product is to build the wrong thing. You’ll pour months (even years) into building it, only to realize that you just can’t make it a success. At Hanno, we see this happening time and time again. That’s why we’ve put together a Lean Validation Playbook.
“Lean” in this case means that you’re moving swiftly to figure out what you’re going to build and how you’re going to build it with as few resources as possible. These resources might include time, money and effort. The lean startup methodology is advocated by Eric Reis, who has massively influenced the way we work through his book The Lean Startup.
Digital Marketing Agency RevUnit rocked the house for their client by turning a deceptively simple idea into a 400% lift in PPC conversions.
When I first met Seth Waite over a Google Hangout a few weeks ago, he mentioned that his agency, RevUnit, had done some “pretty fun things with Unbounce” for clients.
It took a little while for me to understand what Seth really meant by “fun;” he meant innovative, experimental digital marketing that actually moves the needle on results. I’ll admit, fun isn’t the first word I’d use to describe Seth’s story.
It’s also deceptively simple.
Based out of Las Vegas, Seth is the CMO at RevUnit, a full-scale digital agency that takes pride in their ability to “Build Small. Learn Fast. Iterate Often.”
This is the story of how Seth’s team at RevUnit used Unbounce to iterate a PPC — and it all started with a simple audit.
A little bit of background
RevUnit’s newest client, School of Rock, had a little bit of an Adwords addiction. Their PPC spending was on overdrive. But the ROI? Well, there was room for improvement.
School of Rock is a music school with more than 160 franchise locations worldwide. They came to RevUnit after experiencing poor-performing Adwords campaigns with a specialized PPC agency. Lead acquisition via PPC for new enrolments was slow and lagging.
School of Rock’s main goal was to drive new student enrolment to individual franchises. In other words, they needed to get more students signed up for music classes at one of the more than 160 locations worldwide.
The question was, how could they increase enrolments and lower the cost of acquisition at the same time?
It all started with a simple audit
Before digging in and building new campaigns from scratch, RevUnit performed a full audit of School of Rock’s Adwords account concentrating on keywords, ads and landing pages.
The AdWords account consisted of 160+ campaigns, 800,000+ keywords and 160+ landing pages. It’s important to note that each campaign represents a franchise location (for instance, “School of Rock Scottsdale” is a single campaign) and each of those franchises locations had their own dedicated landing page.
During the audit Seth’s team found some pretty common mistakes, particularly with the landing pages associated with each campaign. Here’s what they were working with in the beginning:
Problems with the “before” landing pages:
Pages were very slow to load. Search engines like Google see this as a poor experience for users, and as a result, penalize pages with a lower quality score.
The lead forms embedded into each landing page were pretty long. Too many form fields can cause visitors friction, meaning they’re less likely to complete the form (and more likely to bounce).
There were some general design and copy issues, the biggest being that content was not designed for easy reading. While there was a lot of information on the pages, it not tell a compelling story.
The pages did not mirror their upstream ads. Without a strong message match, visitors are more likely to bounce, again resulting in a lower quality score from Google.
Campaigns weren’t enabled with click-to-call tracking so it was impossible to measure how many phone calls were generated from Adwords activities.
Seth’s team hypothesized that if they tackled each of the problems above, School of Rock would yield better results from their AdWords campaigns.
But (and this was a pretty big ‘but’), they couldn’t really afford to tackle 160 different landing pages without knowing for sure.
Here’s the good part
Instead of jumping in willy nilly, Seth’s team decided to use Unbounce to create a template for just one of the franchise locations. Basically, he created a single landing page to test out his hypothesis. The idea was that if the template actually increased enrollment for one of the franchise locations it could be replicated for others.
Sidnee Schaefer, RevUnit’s Senior Marketing Strategist, then went to the whiteboard with Seth and other members of the team to design the new strategic landing pages. After creating a mockup of the new page’s layout, Sidnee jumped into the Unbounce builder to implement the design.
The newly designed landing page template aimed to follow a story that is easy-to-digest and comprehend while presenting a clean and well-structured format. The page was built to create the shortest path to conversion without sacrificing need-to-know information.
According to Seth,
Every brand has a very different story and we knew how important it was to tell the story of how School of Rock is different than the average music school. We designed the page to reflect this brand positioning.
For the new School of Rock landing pages, content was strategically placed into sections covering who, what, where and why (including reviews). “We kept the copy clear and strong to avoid burdening people with too much information,” says Seth.
RevUnit also used Zapier to bridge a connection between Unbounce and School of Rock’s CRM system, so new leads go directly to franchises once submitted.
The result of RevUnit’s pilot was pretty convincing: a 75% increase in average weekly conversions and a 50% decrease in cost per conversion.And, all these new leads were acquired using half the budget.
But that’s not all.
Seth didn’t stop with “good enough” – that’s just not his kind of fun.
Here’s the even *better* good part
The cherry on top of this masterminded plan is how RevUnit implemented Dynamic Text Replacement (DTR) to really match Google search queries with the landing page’s headline.
DTR is an Unbounce feature that lets you tailor the text on your landing page to match keyboard parameters, pay-per-click (PPC) campaigns, and other sources, using external variables you can attach to the URL.
DTR automatically updates specified content on your page (like a word in your headline) based on a visitor’s search query. RevUnit used DTR on their client’s landing page to ensure each visitor was served up the most relevant copy possible.
We used dynamic content on the landing page which allowed us to show personalized content to different site visitors based on keywords and locations from the ads. This helped us match the perfect ad with the perfect landing page.
In other words, when a searcher types in “drum lessons, Scottsdale, AZ” dynamic text replacement (DTR) is used to match the landing page headline with the Google search query. As a result, when the visitor clicks through to the School of Rock landing page, the headline would look something like this, “Scottsdale Drum Lessons.”
A strong message match between the traffic source (PPC ad, social media, dedicated email or otherwise) and the landing page headline helps visitors understand that they are in the right place (and prompts thoughts like “yes, this is exactly what I was looking for!”).
According to Seth, here’s why DTR was a game changer for this campaign, “because our PPC keyword strategy was very focused on instrument lessons (guitar, piano, etc), we’d need five landing pages (a different landing page for each instrument type) for each franchise location.”
This would have normally been a painful and timely undertaking but, as Seth put it, “Unbounce had a solution.”
Here’s how they used DTR:
We strategically designed the pages with DTR in mind, so that instrument keywords could be placed throughout the page. Instead of having to create 750+ landing pages, we only had to create one for each franchise location.
After the pilot’s stellar performance, Seth knew with confidence that it was time to roll it out to the rest of the 160+ School of Rock franchise locations.
Again, the results were incredible:
The number of monthly conversions improved 5x, by 250%, and the cost per conversion decreased by 82%. School of Rock has seen a huge improvement to their ROI on AdWords and their lead volume is stabilized.
What did the mean for School of Rock? Well, according to Seth, the “average value of improvements made based on customer lifetime value is potentially a 400% increase in yearly revenue based on new leads.”
The numbers are impressive but the best part of this story is that it’s easy for data-driven marketers to replicate. Start with a guess – a hunch, a hypothesis, an idea – and test it out. In other words, “Build Small. Learn Fast. Iterate Often.”