Modernizing Auto Navigator to connect car buyers with financing
Background: Auto Navigator is a car buying platform from Capital One. On Auto Navigator, a prospective car-buyer can both pre-qualify for financing as well as browse cars and calculate their real rates and payments for vehicles. Auto Navigator had been operating in a pilot mode for several years, with the core audience being existing Capital One customers. A majority of traffic to the site was generated by direct marketing campaigns to existing customers. However, the number of these customers was just a drop in the bucket in comparison to the total market size.
Timeline: October 2018 - January 2019
The Problem: The Auto Navigator website heavily focused on “financing first, car second”, but research revealed that the majority of car buyers shopped by first selecting a vehicle, then determining financing. As Auto Navigator worked to attract more visitors to the site, it found that people were often bouncing, and rarely pre-qualified or looked at vehicles. Car buyers were confused about the purpose of the site, which also meant Capital One was missing out on users converting to Auto Loans.
Design and Project Objectives:
Align the Auto Navigator website with user expectations and behavior in order to increase conversion
Improve the return-user experience in order to move users further down the business funnel
Increase the rate of pre-qualification for financing
Improve SEO
My Role and the Team:
I was the Lead interaction designer, and was responsible for setting UX design and testing strategy, and facilitating workshops.
I split the research responsibilities and initiatives with a UX Researcher
I worked with a Lead visual designer to create dev-ready designs. The visual designer was responsible for aligning the direction with the brand and marketing team, as well as creating and finalizing components for our pattern library.
1 Product Manager, 1 Engineering Lead, 1 Marketing Manager
Design Outcomes:
Reduced bounce rate of original web traffic from 60% to 39%
Doubled the engagement with vehicle search
12% increase in rate of pre-qualification
6% increase in application completion
7% increase in users buying a car using Capital One financing
The Approach
Stakeholder Interviews
Desk Research
Usability Analysis
User Interviews
Wireframing
User Testing
Visual Design
Multivariate Testing
The Original Designs
The anonymous landing page (left), and the authenticated landing page (right).
Anonymous/non-authenticated landing page
logged-in dashboard
Stakeholder Interviews
Interview Objectives:
Understand the motivation behind starting this project
Understand the stakeholders’ perspective on success for the project
Gain context on what has been done previously, what has worked well, what hasn’t worked well
Develop rapport and understand communication preferences
Learnings:
Product leaders were hesitant of redesigns as they felt the UX team pushed for “too much” without a strong business case for why changes should be made. User research and testing was infrequently conducted in past projects.
The impetus for the project was to convert to Angular 8, which was required to meet Capital One’s security and Accessibility standards. However, this would require building pages entirely from scratch, and leadership was hesitant about adding too much additional tech effort.
The business was hoping to move Auto Navigator toward a national launch with expanded marketing campaigns. As a first step, they wanted to improve SEO as the bounce rate was below average.
My Takeaways:
Stakeholders and product leaders were very business conscious. I would need to communicate the impact that designs had not only on the user, but also on the business value.
Design changes needed to be data driven and supported with research in order to increase stakeholders’ confidence.
If there was a feature that could provide high value for the user, but the tech effort was considered too high, I would need to find low-effort ways to test the feature to demonstrate business impact and increase buy-in.
Desk Research
Timeframe: <1 week
Before starting anything else, I conducted desk research. Not only would this help get a holistic understanding of the problem, but also aid in identifying knowns and unknowns in order to guide our research plan.
Step 1: Focusing on the user
Step 2: Craft questions to guide desk research
Step 3: Identify themes, knowns and unknowns
Step 1: Focusing on the user
Auto Navigator had several defined segments of users which fell into 2 groups: Financing-First car buyers, and Vehicle-First car buyers. I kept these users’ wants and needs in mind when conducting desk research.
Financing-First Car Buyers
Thinks in terms of “how much car can I afford?” Determines how much they can get qualified for and searches for vehicles based on that number.
Financially knowledgeable, and can use this to their advantage in dealership negotiations
Knows their high credit score can provide them with many financing options, and is likely to switch to a lender that gets them the lowest rate
Vehicle First
Searches for cars based on their wants and needs. May know the exact make and model, or searches based on features
May be less familiar with or concerned finances, and looks to the dealership for guidance.
Focuses on monthly payment rather than rate to get the car they want. More likely to go with whatever lender will get them into their desired car, regardless of rate
Where in the car buying journey do users typically encounter Auto Navigator? Is Auto Navigator matching their mental model when users arrive?
Do users find our product to be valuable?
What do users find valuable about the existing experience?
Is there something missing from our experience that our users need?
Where is there the most friction in our experience?
Do our users understand how to use our product?
Sources utilized:
Web Page Analytics
Competitive analysis of vehicle search aggregators
Clicktale
Customer Support Teams’ reports for common product complaints, support requests
Internal user research and marketing research
Baymard eCommerce
Step 2: Craft questions to guide and execute desk research
I crafted the following questions to guide my desk research:
Step 3: Identify themes, knowns and unknowns.
From the desk research I created an Excel sheet with preliminary findings, remaining open questions, and ideas to explore in design and research. Below are some examples of the findings and related design ideas.
Users appeared to be confused about what a pre-qualification is, and how it relates to buying a car.
Customer support received many calls from users asking why they couldn’t just look at vehicles without submitting extensive information (the pre-qualification)
High engagement with the pre-qualify button, followed by a high drop off on the first page of the application
Low engagement with the website after becoming pre-qualified.
Non pre-qualified users who interacted with the Vehicle Search section were more than twice as likely to apply for financing
Despite the fact that the vehicle search was well below the fold, users who did engage with the search experience were more likely to pre-qualify
Users appeared confused about how to use the website and needed additional support.
Our website’s layout and navigation was different in comparison to other vehicle search sites, so if a user were to encounter our site in their car search, they would have a learning curve which may deter them from engaging with the site.
Users who interacted with the “how it works” video were more likely to later pre-qualify
Clicktale revealed many users quickly navigating back and forth between pages as if they were looking for something, only for them to quickly leave.
Preliminary design ideas:
Make it easier to search and find cars before becoming pre-qualified.
Current designs force pre-qualification early on, consider moving options to pre-qualify down the funnel, such as on the vehicle details page
Test different content options to better communicate the value of the pre-qualification
Preliminary design ideas:
Move search capability above the fold
Consider multiple ways of conducting a vehicle “search”, such as a carousel of popular vehicles, searching by body style, and more.
Preliminary design ideas:
Expose navigation, removing the hamburger menu.
Align the anonymous and authenticated experience to use similar designs to reduce learning curve
Creating the design and research plan
The scope of the redesign covered several major pages: Landing Page, Search Results Page, and Vehicle Details Page. Keeping in mind that the stakeholders didn’t want a project that felt too “sprawling”, I divided the plan according to the pages of the redesign.
We would first conduct research on the end-to-end experience, and then focus on the Landing Page, followed by the Search Results and Vehicle Page. With this approach, we would save time as the team would be able to start each subsequent project with insights already gathered, and any required research sessions would be focused on user feedback and testing.
Research Plan:
PURE Analysis —Analysis to identify and quantify major usability challenges that could be contributing to reduced engagement and high call volumes
Interviews and testing of existing site with prospective car buyers to understand their wants and needs
Kano Survey of common vehicle shopping features to evaluate and quantify desirability to users
Landing Page Plan:
Wireframes
Visual Design
User Testing to eliminate any unseen risks that could impact conversion
Multivariate Testing to compare and measure impact of designs and features on business metrics
PURE Analysis
Timeframe: 2 days
The desk research revealed many signs of user confusion and drops in engagement across the site. Since the entire website would be redesigned, we didn’t want to conduct an expensive usability test to identify these problems in the existing site. However, we did want to avoid repeating the same mistakes, as well as have a way to quantify any improvements.
The PURE analysis was our solution to this. In a PURE analysis, designers, researchers, or other usability experts conduct tasks similar to that of the users, and rate the level of friction for each task. Then, the results are compared and averaged to get a total usability score. Similar to golf, the lower the score, the better.
Myself, a researcher, and a fellow designer sat down to evaluate Auto Navigator, and documented our results.
A sample of the guide we used to evaluate Auto Navigator
Outcomes:
Identification and measurement of which user flows have the highest amount of friction or dead ends
A prioritized list of which user flows or features need to be addressed or reimagined in the subsequent design efforts, such as: improving search filter design and behavior, bugs to be addressed by development, content to be rewritten for clarity
Baseline score to measure usability improvements off of
Identified questions to follow up on in user interviews
User Interviews and Feedback
Timeframe: 3 weeks for recruitment through synthesis
Our researcher led the recruitment and planning for our interviews, while I partnered to conduct the interviews and synthesis workshop.
Research Objectives:
Understand the users’ goals and motivations when searching for and buying a vehicle, and how the user selects and knows the car they want to buy.
Understand how and when a car buyer considers financing, and what information is important to them.
Understand what resources and information they leverage in their car search, how they use them, and what is valuable about those resources.
Identify what from the existing experience is helpful to the user, what is challenging, and where their needs are not being met.
Research Format
Pre-interview to ask them about where they were in the car buying process
Invited participants to use our product while we recorded their screen and observed from the other room.
Post-interview, where we replayed back their experience and asked them what they were thinking and feeling.
(a snippet of) What we learned:
Participants valued search, and expect to easily search for vehicles before becoming pre-qualified
Participants wanted to search for vehicles before getting pre-qualified, especially when they saw how lengthy the pre-qualification form was. “I don’t want to waste my time filling that out if I can’t even find a car I like”. Participants were clicking around trying to find a way to search, which was hidden in the hamburger menu.
The search options, which were a series of dropdowns, were cumbersome and annoying to users. If they wanted to restart their search from scratch, they had to navigate to a different page.
Pre-qualification created confusion…
After pre-qualifying, participants didn’t understand why some cars were in or out of policy, or what could be done to resolve it. “I feel like this is something the dealership would do for me — help me figure out how to get the car I want.”
Capital One offers dynamic APRs depending on your credit score and the vehicle, with the lowest rates on brand new cars. “Financing-first” participants wanted to see the APRs that Capital One offers on the landing page before applying. When they pre-qualified, they were confused why they didn’t receive an APR right away, and why the APR was different from vehicle.
… But pre-qualification was also valuable
For participants who have had the experience of being declined in a dealership before, the pre-qualification increased confidence.
Participants liked that they could play around with the monthly payment calculator on the vehicle details page and see a more accurate monthly payment than with estimates. But they didn’t know that this was an option before accidentally discovering it.
Kano analysis
Timeframe: 2 weeks for creation, launch, and synthesis
From the very beginning of this project, there were many internal debates about what users valued, as well as many ideas for additional features to add. In order for the team to have clarity on what our users expected for the Landing Page, as well as to satisfy our stakeholders’ appetite for quantitative data, I conducted a Kano analysis.
Shortly after conducting desk research, I met with the team to refine our list of ideas for the Landing Page, and launched the survey so that the results would arrive closely to the conclusion of the user interviews.
I included a control option, “Log in”, to act as a baseline. There was also debate over whether to include “pre-qualification”, but since this was a core aspect of Auto Navigator’s business model and was going to be included no matter what, I opted to instead include options that were relevant to pre-qualification.
Screenshot of the results of the survey.
Wireframes
Timeline: 2 weeks from initial design concepts to feedback and refinement
Utilizing the previous research for direction, I created wireframes for both the anonymous, authenticated, and returning user experience.
In addition, since we did not have a dedicated content strategist at the time, I partnered with a member of the marketing team to draft content in Capital One’s voice, which I incorporated into the wireframes.
Mobile Wireframes for Anonymous Users
Wireframes for returning and authenticated users
Overcoming the fear of Search through A/B/C testing
Despite user interviews, competitive analysis, and the Kano analysis revealing that users wanted to search for cars first, having a “search first” experience was a monumental shift from how the team, especially senior stakeholders, viewed Auto Navigator.
Improving the search experience would be a large technical investment. In addition, the business was concerned that focusing on search would impact the number of pre-qualifications, and thus impact the number of loans funded with Capital One.
I recommended conducting an A/B test using the original landing page so that we could reduce technical effort, as well as have a quick response time for results.
The testing options
Control: Search functionality is far below the fold
Results — a win for a search-first experience!
Both Option A and B performed better than the control, with Option B being the winner! Exposing the feature more than doubled the engagement with search, and also resulted in 6% more completed pre-qualifications — not bad for a simple change! The number of funded loans with Capital One did not increase by a statistically significant amount, but it also was not harmed.
This gave the leadership team more confidence in investing in the search experience, the details of which I address in a separate case study (coming soon).
Testing Design:
Test would run for 2 weeks
In a region with no concurrent tests, the population of non-prequalified visitors would receive either the control, option A, or option B.
Test would monitor:
Interactions with search features
Start rate of pre-qualification
Completion rate of pre-qualification
Number of loans funded with Capital One (technically this was monitored outside of the test)
