Customer Survey
& Archetypes
A quantitative research initiative leveraging in-depth statistical analyses of large-scale survey data for the development of distinguished customer archetypes.
The results are in.
"Seriously so impressed with your ability to push this through despite the NUMEROUS setbacks, resistances, pivots etc. Rachael kept holding the vision and adapting and moving forward. LEGEND."
— Laura, Senior UX Researcher
"And on our original timeline to boot! EPIC!"
— Karin, Vice President of Design
Project overview
Opportunity
After years of developing the WM product with only one type of user in mind, the organization lacked understanding of who our customers actually are, how they shop, and what they need from a shopping experience.
Contribution
Responsible for designing, administering, and analyzing a large-scale customer survey, deriving archetypes from the dataset, documenting all findings, and reporting key insights to the executive team and beyond.
Outcome
Statistically discernible customer segments translated to non-personified user archetypes, which were cross-functionally adopted and integrated into the collective product development process.
-
UX Research, Public Relations, Brand Marketing, Product Marketing, Product Management, & UX Design
-
- Survey to gain quantitative insight from our current customer base
- Descriptive analysis to break down the dataset into percentiles and determine prominent characteristics across the sample
- Regression analysis to identify patterns and relationships and between variables
- Cluster analysis to segment respondents into meaningful segments
-
Qualtrics, Figma, Google Workspace
Background
What’s this project about?
The Weedmaps product had initially been designed with a single type of user in mind, based on early assumptions and ideals. Since then, our customer base has grown exponentially, and this narrow focus restricted our ability to appeal to a broader audience and limited our potential market reach. We administered a large-scale survey aimed at understanding the diverse segments that comprise our user base so we can better serve their needs.
Why a survey?
Broad reach & inclusivity
Surveys have the capability to reach a wide and diverse audience, ensuring that we capture inputs from various customer segments and avoid biases.
Quantitative insights
Surveys allow for the collection of quantitative data, providing statistically significant insights into customer demographics, attitudes, and behaviors.
Direct customer feedback
By asking targeted questions directly to our customers, we could gather firsthand information about their shopping habits, preferences, and needs.
Baseline for future studies
The survey results provide a baseline of customer insights that can be referenced in future research projects to track changes over time and measure strategic impact.
What would the results inform?
The survey would produce a robust dataset that would inform the creation of detailed customer archetypes representative of the current customer base. These archetypes are now instrumental in guiding our marketing strategies and product development, ensuring that our efforts are aligned with the true needs and desires of our users.
Project planning
To kick off, I scheduled a series of meetings with various teams to capture the considerations, requirements, and constraints of the project.
Product Management, Marketing, and UX to align on objectives
Marketing to coordinate survey email content strategy & deployment
Finance to discuss incentive budget & reward delivery procedure
Legal to determine screening criteria & review/approval process
Survey design & execution
Question development
After conducting stakeholder interviews to understand each team’s objectives, I drafted a multi-dimensional survey with questions from five key segmentation categories:
Demographic
Attributes & characteristics, such as age, gender, and geographic location
Behavioral
Purchasing & usage habits, such as purchase method, spend, and frequency
Psychographic
Attitudes & interests, such as purchase considerations and product priorities
Competitive
Preferences & comparison, such as experiences with and opinions of competitors
Inactive customer
Lapsed & abandoned, shown only to those who have not purchased in 3+ months
Question types included multiple choice, semantic differential, rank-order, and some open-ended response, with skip logic and conditional branching built in to ensure wording and flow would be appropriate for each participant based on their individual responses.
Targeting strategy
It was crucial that we hear from our current or former customers so that the data accurately represented our audience. With the benefits outweighing the risks, we decided to administer the survey via email.
Email survey benefits:
Wide reach with our own customers (email file exceeds 3 million individual users)
Owned channel: no additional cost to distribute
Allows the possibility to hear from lapsed or churned customers (branch to inactive customer survey questions)
Little risk of interrupting a current shopping session
Email survey potential risks:
Sample is limited to marketing email opted-in customers, who may have different characteristics vs. the opted-out segment
Low response rate relative to more targeted methods
Certain segments may be more or less effectively incentivized
Incentive strategy
With a finance-approved budget of $5k, I partnered with legal and marketing to decide between two incentives:
Chance to win $X gift card, Y winners chosen
$X gift card to first Y responses
We landed on rewarding the first 250 respondents with a $20 Uber Eats gift card to create urgency, optimize perceived chances of winning, balance reward value with quantity, and avoid legal hurdles concerning sweepstakes.
Data collection
With strategy in place, I socialized the survey draft for feedback & sign-off, then built the questions & logic in Qualtrics and applied the screener. Once the preview was generated, I recruited the UX Design team to stress-test for optimal comprehension and functionality.
Then, the survey email was scheduled and deployed.
Within 4 hours we received more than
12,000 responses,
exceeding our goal with five days to spare!
Given the remarkable response rate, I decided the planned reminder email would not be necessary. The survey was kept live for the initially determined to give customers a fair chance to participate, including those who may not check email frequently.
In total, we collected more than 22k submissions, 6k of which were eliminated during QA, leaving us with 16.3k responses in our dataset.
Analysis & archetypes
After cleaning the dataset to exclude unreliable participants, I got to work conducting a thorough quantitative analysis using three key statistical methods.
Descriptive analysis
To summarize the data in terms of percentages & averages and compare to historical data
Regression analysis
To identify trends & patterns and measure the strength of relationships between variables
Cluster analysis
To segment respondents into groups based on association of set variables, informing archetypes
Key survey insights
Through extensive and collaborative analysis with two other UX Researchers, we uncovered hundreds of actionable insights about our customer. Findings ranged from trends in basic demographics to complex and compelling statistical relationships between variables. Below is a sample insight and three corresponding How Might We statements.
Sample insight
Our competitor outperforms us in overall customer sentiment in a key market region where new product discovery is reported as a higher priority than in other geographies.
How might we facilitate product discovery for users who desire newness?
How might we alert customers about new product launches?
How might we compel users to switch from our competitor to our product?
Customer archetypes
I conducted a cluster analysis to separate the 16k survey participants into meaningful groups based on their responses to certain questions by:
Determining how many & which variables to include in segmentation
Identifying the optimal number of clusters
Running the cluster analysis in Qualtrics
Analyzing resulting clusters for further development
Articulating the unique characteristics of each cluster
I repeated the analysis, refining parameters to maximize the silhouette score (i.e. a measure of the strength of the relationship between datapoints within a cluster), resulting in distinguishable clusters.
*Note: Archetypes have been modified due to employer confidentiality
Results & next steps
Outcome
The WM customer archetypes have been adopted company-wide as true representatives of our customers, inspiring empathy for the various users of our product and guiding decision-making during the product development process.
Next steps
There is high demand for a repeat study focused on the B2B client, with the objective of developing archetypes representing WM SaaS product users segmented by attributes such as job function, product usage, and company size.