AI Can Do User Research — But Should You Trust It?
AI can read thousands of reviews, tickets and survey answers in minutes. Here's where that genuinely helps — and where you still need to talk to people.
Most teams are sitting on more user feedback than they will ever read. App store reviews, support tickets, survey comments, sales call notes, interview transcripts. It piles up because reading it properly takes time nobody has.
AI changes that. It can read all of it, group it and summarise it faster than any team could. That's genuinely useful. But speed isn't the same as understanding, and it's worth being clear about what AI is actually doing when it "does research".
What AI is good at
AI is excellent at the parts of research that are about volume.
Sorting and grouping
Give an AI model five hundred support tickets and it can cluster them into themes: login problems, billing confusion, missing features, slow performance. Work that would take a person a few days takes minutes.
Summarising long material
Interview transcripts are long. AI can produce a first-pass summary of each one, pull out the moments where a participant was frustrated, and highlight quotes worth revisiting.
Spotting patterns across sources
Reviews, surveys and tickets usually live in different tools. AI can look across all of them and point out when the same complaint shows up in several places — a strong signal that something is worth investigating.
Tagging at scale
Consistent tagging is tedious. AI can apply a tagging scheme across thousands of entries, which makes the data searchable and easier to share.
Where it falls short
The problems start when AI output is treated as the conclusion rather than the starting point.
It only sees what was written down
Feedback data is a record of what people chose to say, in the moment they said it. It misses what they didn't mention, what they didn't notice, and what they worked around without complaining. Watching someone use a product often reveals problems they would never report.
It can't ask a follow-up question
The most valuable moment in an interview is usually the follow-up: "You paused there — what were you expecting to happen?" AI analysis of existing data can't do that. It can only work with the answers it already has.
It flattens context
A complaint about "slow checkout" from a first-time buyer on a weak mobile connection means something different from the same complaint from a power user on a desktop. Summaries tend to strip out exactly that kind of detail.
It sounds confident either way
AI summaries read well whether the underlying evidence is strong or thin. A theme based on three comments can look just as convincing as one based on three hundred. Someone needs to check.
AI can tell you what people said. It takes a researcher to work out what they meant.
A practical way to use it
We treat AI as a research assistant, not a researcher. In practice that looks like this:
- Use AI to process the backlog. Cluster existing feedback, summarise transcripts, surface recurring themes.
- Check the evidence. For every theme, look at the raw entries behind it. How many are there? Are they really saying the same thing?
- Turn themes into questions. A theme like "users are confused by pricing" becomes a question: where exactly does the confusion happen, and why?
- Answer those questions with real people. Interviews, usability tests, observation. This is where the understanding comes from.
- Use AI again to speed up analysis — then make the final calls yourself.
So, should you trust it?
Trust it to save you time. Trust it to find things you might have missed in a pile of feedback. Don't trust it to replace the moment where you sit with a real person, watch them use your product, and ask why.
The teams getting the most from AI research tools aren't the ones doing less research. They're the ones spending less time on sorting and more time on understanding.