AI in product management
How to use AI to analyse customer interviews without losing the evidence
Updated
Uploading a stack of interview transcripts and asking AI for “the key themes” is tempting, and the result reads well. The trouble is what you cannot see: quotes that were never said, two customers blended into one, and the context of who had which problem quietly gone. This article shows where AI genuinely helps with interview analysis and a workflow that keeps every insight traceable to what a real person said.
The problem with “summarise these interviews”
Teresa Torres tested ChatGPT on real customer interviews and checked its output against the transcripts. Around 30% of the direct quotes it produced were inaccurate or did not appear in the source at all. It also invented details, such as a participant’s job title, and merged two separate statements into one with a new meaning.
Invented quotes are the dangerous kind of error, because a quotation mark signals “a customer said this”. Once a fabricated quote is in a slide deck, it carries the full weight of real evidence. Generic prompts make it worse: “themes” and “common pain points” flatten each customer’s story and lose who had which problem, which is exactly what you need to act on it.
What AI is good at here
- Transcription and notes. Accurate, fast and cheap. Let AI do it.
- Finding candidate quotes. Pulling out passages worth keeping from a long transcript saves real time, as long as each one is checked against the source.
- Tagging and search. Adding persona, product area and topic tags makes old interviews findable when a new question comes up.
- A second opinion. After you have synthesised an interview yourself, asking AI what you missed or how else a passage could be read surfaces things you would have overlooked.
A workflow that keeps the evidence intact
- Record and transcribe. Get consent, and remove names and contact details from the transcript. Use a role and a participant reference such as P07 instead.
- Synthesise each interview yourself first. Spend 15 to 20 minutes on each one while it is fresh. Torres recommends this order: do your own synthesis, then bring AI in as a collaborator.
- Ask AI for verbatim quotes with locations. Ask it to return exact quotes only, each with a timestamp or line number, and nothing paraphrased. This turns verification from rereading into a quick lookup.
- Verify every quote before you keep it. Check each one against the transcript. Fix or discard anything that does not match word for word. No exceptions for quotes that “sound right”.
- Keep one insight per observation. One quote, its source, the persona and the product area. Small insights can be linked precisely to the hypotheses they support.
- Look across interviews in a fresh conversation. Analyse each interview in its own conversation, then compare across them in a new one. Long conversations drift and mix up details.
- Compare AI’s patterns with yours. Where AI found something you did not, go back to the transcripts and confirm it before it becomes an insight.
Prompts to reuse
Here is one customer interview transcript with line numbers. Return up to 8 passages worth keeping as evidence. Rules: - Quote the customer's exact words only. Never paraphrase or combine lines. - Give the line numbers for each quote. - Say in one sentence why each quote matters. - If you are unsure a passage is relevant, leave it out.
Here is my synthesis of this interview, and the transcript. What did I miss or misread? For each point: - quote the transcript lines that support it - say whether it is a need, a pain point or a solution idea Do not repeat points I already made.
Signs your AI analysis has gone wrong
- Quotes that are suspiciously tidy, or that sound like the same person across different interviews
- Themes so general they would fit any product: “users want it to be easier”
- No way to tell which customer said what
- Solutions presented as needs: “customers need a dashboard”
- Nobody on the team could name the interview a finding came from
Analysed insights are the start of the chain, not the end. Turn the strongest patterns into testable hypotheses, and keep them linked to the insights behind them.
Sources
Keep your evidence and decisions linked
Groundwork takes your team from customer insights to evidence-based requirements. Free to start, with your whole team.
Get started