Generative AI
November 19, 2026

Building AI Support That Learns From Customer Conversations: 6 Lessons from Quizlet’s Matt Kosko

Jessica Jess
Content Strategist, Voice of Customer

Key takeaways from Matt Kosko of Quizlet and Enterpret CEO Varun Sharma on customer context, chatbot measurement, and self-improving feedback loops.

For years, support at scale meant keeping conversations to a minimum.

Contact options sat deep in account settings. Chatbots walked people through rigid decision trees. Anyone who still wanted a person could wait 12 to 24 hours for a reply, sometimes longer.

AI flips that. Companies can now invite as many conversations as they want.

But that only pays off if teams can learn from every one of them.

That idea framed a conversation between Matt Kosko, who leads user operations at Quizlet, and Enterpret CEO Varun Sharma. Matt has spent about 15 years in customer service and a decade in tech operations, and his team sits inside Quizlet’s product organization.

Watch the full webinar on demand →

Here are six lessons for teams building AI support that keeps getting better.

1. Invite conversations instead of deflecting them

Before AI, every support conversation cost human time. So companies at scale put filters in front of it: a help center to search, a rules-based chatbot that pointed to the next step, a contact form a few clicks deep. Sometimes that friction was intentional. Sometimes it wasn’t.

Varun described the shift as moving from defense to offense. With AI, companies can invite as many conversations as possible, across the product and the website.

Matt tested this at his previous company, which offered live chat. Customers got a choice when they arrived. An AI assistant grounded in a strong knowledge base could try to answer right away, or they could chat with a person. If the AI couldn’t help, they would go straight to a human.

Many people picked the AI. They figured there was no downside to trying.

The lesson Matt took from it: let AI try first, keep the human path visible, and invest in quality. As the automated experience improves, more traffic shifts to it on its own.

The takeaway: Make AI the fastest route to a good answer and keep a person one step away. Let quality, not friction, move volume to automation.

2. A complete view makes customer feedback more useful

Inviting more conversations also means more places to listen. Customers talk about products in chat, support tickets, app reviews, Reddit, surveys and recorded calls.

When a team analyzes only escalations, it gets a narrow view. In a subscription business, escalations tend to skew toward billing questions and negative experiences. That picture is hard to share with the rest of the company because it doesn’t reflect what most customers are saying.

They also miss feedback that never turns into a ticket. Matt pointed out that product feedback often shows up in the first message of a chat, from someone who would never fill out a feedback form.

Bringing every channel into one view helps teams weigh themes accurately. It also separates a loud signal from a representative one. Matt described Reddit as a megaphone for customer problems. A theme gaining traction there can be checked against what customers say in support conversations and surveys.

Both Matt and Varun call themselves silent churners. When a product lets them down, they don’t contact support. They leave.

That’s why Varun pushes back when someone asks whether 100 complaints is statistically significant. The people who write in are a fraction of the people affected. Matt wants to go further by connecting feedback to product analytics, to estimate how many silent users share an issue that only a few people report.

As Varun put it, teams need the quantitative and the qualitative together. Behavior data shows what is happening. Customers’ own words show why.

The takeaway: Count every conversation before you weigh any of them. A theme’s true size only shows up when every channel sits in one place.

3. Put customer insight where teams can act on it

At Quizlet, customer experience sits within the product team. Varun sees that reporting line as a signal of intent: customer feedback exists to fuel the product, not to be managed as a cost center.

Matt believes CX teams hold more insight than they get credit for. But a seat at the table only goes so far if one person has to fill it. So he makes customer insight self-serve. Teams can explore it on their own early in the development cycle, and his team brings relevant context into Slack discussions as they happen.

Next year, Matt plans to go further. He wants to look at what each product team is building next quarter and send it customer insight specific to that work, including feedback on what it has already launched.

The reason is fatigue. In many subscription businesses, a few topics such as monetization questions dominate the feedback. Teams see the same themes again and again and start tuning them out. Fresh, targeted evidence gives them a reason to dig in.

Varun framed it across the development lifecycle: prioritizing and scoping, writing the PRD, choosing beta users, and closing the loop with customers after launch. It also keeps everyone working from one set of facts. Without that, ten people asking an AI tool the same question can get ten different answers.

The takeaway: Make customer context self-serve, then push targeted evidence to each team at the moment it’s making decisions.

4. Build the feedback layer, not the plumbing

Varun has watched product and CX operations teams change roles. They used to produce the artifact: a monthly top-10 list or slide deck of customer pain points. Now they increasingly build the systems that let everyone else find answers on their own.

As AI makes analysis cheaper, teams may be tempted to build more of it themselves. Matt draws a line between building useful workflows on top of customer data and building the infrastructure that makes that data usable.

AI still works within context windows. With thousands or millions of feedback records, you can’t point a general-purpose model at the full stream and expect reliable answers. A structured taxonomy layer organizes the feedback first. That’s what makes an analysis like a top-10 list of customer issues fast and cheap, when it used to take someone weeks of digging.

Varun added that every channel has its own nuance. A short comment on a launch video, a 45-minute sales call that opens with ten minutes of small talk, and a Reddit thread where the context sits three replies up all need different handling. Feed raw data straight into a model and it samples differently every time, burning tokens without producing consistent answers.

Matt explained why this matters on the front line. A support rep spots a new problem and escalates it. Engineering asks how many people reported it. The answer is two, and the rep suspects it’s bigger. Until there’s data behind it, nobody listens.

The takeaway: Build the agents and feedback loops that act on customer context. Don’t rebuild the infrastructure beneath them.

5. Measure chatbot results with definitions you trust

When Matt evaluates chatbot vendors, he looks for two things.

The first is a natural-language script with guardrails rather than a deterministic decision tree. Customers can describe a problem in their own words, so responses are more tailored. And when an insight reveals a gap, he can add a rule to the script instead of tracing the right branch through a sprawling workflow map.

The second is a straightforward way to export chat transcripts. Teams need the underlying conversations, not only the vendor’s reporting, so they can see chat alongside every other channel.

Matt’s theory on why vendors resist exports is that they’ve invested heavily in their own analytics. But building a chatbot is no longer hard. He thinks vendors should compete on conversation quality and let customers decide what to do with their data.

Resolution rate is a different kind of metric. Varun pointed out that measures like time to first response are objective, but resolution is open to interpretation. The customer should own that definition, not the vendor. And as Matt noted, when resolution rates climb into the 80s, every point matters.

An honest number matters even when you don’t pay per resolution. An inflated rate hides the gaps a team needs to fix, and customers keep running into them.

The takeaway: Choose AI support for conversation quality and data portability. Then measure resolution by your definition, not your vendor’s.

6. Create loops that improve with every conversation

Matt’s test for a good loop is how much of the quality you control within your part of it. Keep a human involved as long as needed, and remove the human whenever possible.

The bug loop. Incoming reports are checked against known limitations and pain points. Known issues get added as another example, with no new noise for anyone. New bugs go to the right team’s Jira board, where engineering’s own loop can pick them up. Matt noted that working out which team owns a bug is easy at a startup and much harder at a large company.

The long-term goal is for some bugs to go from customer report to fix to customer notification without a human touching them. Even a small share, Matt said, would be meaningful.

He doesn’t see this as AI taking jobs. It removes low-level work and gets customers’ problems fixed faster.

The knowledge loop. Matt wants customers to know they’re talking to AI and to trust they’ll get a high-quality answer in 30 seconds or less. That depends on closing knowledge gaps fast. This loop finds unanswered questions and flags the help content that needs to change. The next step is drafting new articles from existing internal product documentation the same day, instead of waiting two or three weeks.

As AI takes on more routine work, Matt expects human handle time to go up, not down. People should get the conversations where they make the biggest difference: empathy, edge cases and complex issues that need more time. If a conversation takes twice as long and the customer leaves happy, that’s a win.

Varun noted that a chatbot’s return is easy to show in labor saved, while a customer intelligence layer is harder to put a single number on. Matt ties its value to how fast these loops run, which shows up in customer satisfaction, refunds and retention. For the harder-to-measure value to product teams, he suggested something as simple as an employee survey.

The takeaway: Build loops you control end to end, measure them by how fast they improve the experience, and save your people for the conversations that need them.

Make customer context part of the system

Better AI support depends on more than the model.

Teams also need a complete view of customer feedback, a way to connect that context to product work, and honest measures of whether their systems are helping.

That changes the job of operations teams, from compiling monthly reports to building the loops that let the whole company act on customer context.

When those pieces work together, every customer conversation can help the next team make a better decision.

Want to hear more? Watch the full conversation with Matt Kosko and Varun Sharma, including how Matt tested AI against live chat, why he wants chatbot vendors to make data export easy, and how he thinks about routing bugs at scale.

Watch the webinar on demand →

‍

Related Blogs
See all blogs
Nov 18, 2026
Did I Build the Right Thing? 6 Lessons from Ravi Mehta on Product Management in the Age of AI
Announcements
Nov 17, 2026
Product reviews from Walmart, Target, Best Buy, and more, now in Enterpret
Announcements
Nov 10, 2026
Close the Loop: Automations built around your changelog
Voice of Customer
Nov 9, 2026
Insights Are Cheap. Decisions Are the Bottleneck: 5 Lessons from Zack Hamilton on the Future of CX

AI That Learns Your Business

Generic AI gives generic insights. Enterpret is trained on your data to speak your language.

Book a demo

Start transforming feedback into customer love.

Leading companies like Perplexity, Notion and Strava power customer intelligence with Enterpret

BOOK A DEMO