How to Run AI-Moderated User Interviews: A Practical Guide

Three bottlenecks vanish. Scheduling: respondents join whenever they want, from a link, with no calendar Tetris. Moderator time: fielding 100 interviews costs you roughly the same effort as fielding 10. Consistency: every respondent hears the same core questions in the same order, which makes cross-interview comparison honest.

One thing does not change. A vague guide still produces vague transcripts. If anything, AI moderation raises the stakes on design, because you are not in the room to rescue a bad question 40 times.

AI moderation shines when you need many voices on an open question. Why users churn, where onboarding loses people, how customers describe your pricing in their own words, what a confusing feature actually feels like. These are questions where 60 fifteen-minute conversations beat 6 hour-long ones.

It is the wrong tool for some jobs, and we cover those honestly below. If your question needs deep rapport, live facilitation of a prototype, or the judgment to sit with a difficult silence, book human sessions instead.

A good discussion guide for AI moderation has six to ten questions: a warm-up, a core block, and a short wrap. Every question should be open, neutral, and answerable by anyone in your sample.

Then decide, per question, how much freedom the AI gets. In unitQ Research, your questions are asked as you wrote them, and you set the follow-up behavior for each one: a factual question might get no follow-ups at all, while a core question gets live AI probing that asks “what did you try next?” or “what did you expect to happen?” based on the actual answer. Scoping follow-ups per question is the single biggest lever for transcript quality.

Two drafting rules earn their keep. First, one idea per question; compound questions produce half-answers. Second, ban your own vocabulary; if respondents call it “the widget thing,” a question about “the configuration module” will stall.

Voice, chat, and video are not interchangeable, and the right answer depends on your audience and topic.

Voice produces the richest answers per minute, because talking is faster than typing and people ramble usefully. It suits busy consumers on phones. Chat lowers the barrier for sensitive topics; people will type things they hesitate to say aloud, and they can respond while multitasking. Video adds facial reaction and lets respondents show you things, at the cost of higher drop-off from camera shyness. unitQ Research supports all three, so you can match mode to study rather than forcing one.

When in doubt for consumer research, start with voice and offer chat as the fallback.

Panels are fine for general audiences, but the highest-yield respondents are usually already in your data: users who left a one-star review last week, ticket submitters with an unresolved billing complaint, NPS detractors, or accounts that went quiet. Recruiting from your feedback data means the interview picks up exactly where the complaint left off, and unitQ customers often route interview invitations from the segments their monitoring already surfaces. (A dedicated guide on recruiting interview participants from feedback is coming soon.)

Distribution is just a link, so meet people where they are: a follow-up email, an in-product prompt, a support-ticket closer. Offer a real incentive for 15 minutes of time, and be explicit that an AI will conduct the conversation. Respondents react better to transparency than to discovering it mid-interview. For lapsed users specifically, see the dedicated guide on interviewing churned users, because timing and tone change when someone has already left.

Run the study on a small batch and read every transcript end to end, not the summaries. You are looking for four failure smells: a question people misread, follow-ups that wander off topic, interviews running long past the promised time, and answers so short the probing never got traction.

Fix the guide and pilot again. This loop is cheap; a revised AI study re-fields in hours. A flawed guide that reaches 200 people is expensive in a way no analysis can repair.

Once the pilot reads clean, open recruitment fully. Check the first batch of completions the same day: completion rate, median duration, and whether any question is turning into a drop-off cliff.

Resist fixing a sample size in advance. Field until new transcripts stop teaching you things, which for a focused consumer question often happens somewhere between 30 and 100 interviews. Saturation, not a round number, is the stopping rule.

Every platform in this category generates automatic summaries and themes. Treat them as a first pass, not a finding. Verify each proposed theme against real verbatims, count how many respondents actually voiced it, and pull the two or three quotes that make it undeniable in a readout.

Then connect the qualitative result to your quantitative signal. If interviews say the paywall feels deceptive, your review and ticket data should show the same theme moving; when interview findings and monitoring data agree, roadmap arguments get much shorter. This pairing is the reason unitQ Research sits inside a quality intelligence platform rather than standing alone.

Honesty matters more in a how-to than anywhere else, so here is where this method loses.

Deep ethnography and contextual inquiry need a human in the participant’s environment. High-stakes B2B stakeholder interviews, where the relationship itself is part of the outcome, deserve a person. Emotionally heavy or legally sensitive topics call for human judgment about when to press and when to stop. Live moderated usability testing of a prototype, where the facilitator improvises tasks, is a different discipline; platforms like Maze approach that problem from the usability side. And at very small N, say five expert users, you will simply learn more moderating the sessions yourself.

One more honest note: some respondents give shorter answers to an AI than to a warm human. Good live follow-ups narrow that gap considerably, but they do not always erase it, which is why the pilot read matters.