Fine-Tune a Small Model or Prompt Claude to Classify Customer Feedback? 6 Factors That Decide It

November 12, 2026

Prompt Claude when your label set is still changing, you have only a handful of labeled examples per class, or you need more than a label back, such as a reason and a supporting quote. Fine-tune a small model, such as a BERT-style encoder or a small open-weight LLM, when the label set is stable, you can label a few hundred examples per class, and volume or latency makes per-call cost matter. Research favors fine-tuning on accuracy for fixed, narrow tasks: in a 2024 study, fine-tuned smaller models beat zero-shot GPT-4 and Claude Opus on every task tested. But for customer feedback the model is rarely the expensive part. The label set is, because it changes every time the product does.

What the research says

Two 2024 studies are the most cited on this question. Bucher and Martini compared fine-tuned smaller models with zero-shot GPT-4 and Claude Opus on sentiment, stance, emotion and political position tasks, and found that fine-tuning with task-specific data won in every case. The gap was smallest on a common task, sentiment (0.92 accuracy for fine-tuned RoBERTa-Large and DeBERTa-V3, against 0.87 for GPT-4), and largest on stance (0.94 for DeBERTa-V3 against 0.61 for Claude Opus).

About 200 labels The point where a fine-tuned RoBERTa-Large classifier's performance began to level off, with a "sweet spot" of 200 to 500 labeled examples. Source: Bucher and Martini, "Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification" (arXiv, 2024)

Edwards and Camacho-Collados ran a larger comparison across 16 classification datasets and reached a similar conclusion: fine-tuning smaller, more efficient models can still beat few-shot prompting of larger ones.

Read both with care. They used 2024 model versions, a single prompt design and no customer feedback data, and frontier models have improved since. Treat them as evidence that fine-tuning remains a strong option for fixed tasks, not as a verdict on today's Claude models with your taxonomy.

The 6 factors that decide it

FactorLeans fine-tuneLeans prompt
Label churnLabels stable for quartersLabels change with releases
Examples per class200 or more for most classesLong tail with few examples
Volume and latencyMillions of records, real-time routingThousands per day, batch is fine
OutputOne label or a fewLabels plus reason, quote, entities
UpkeepML engineers own retrainingProduct or ops owns definitions
ConsistencySame input, same outputVariation must be measured

1. How often the label set changes

Customer feedback labels move with the product. A new feature brings new complaints, a redesign retires old ones, and two themes that used to be distinct merge. A fine-tuned model needs relabeled data and a retraining run for each change. A prompted model needs an edited definition. If your taxonomy changes monthly, that difference outweighs most accuracy gains. Watch for the signals of taxonomy drift to see how fast yours moves.

2. How many labeled examples you have per class

The 200 to 500 example range is per class. A ten-label sentiment-style task gets there quickly. A feedback taxonomy with 150 themes, most of them rare, may never get there for the long tail. A common split is fine-tuning for the high-volume classes and prompting for the rest.

3. Volume and latency

A small fine-tuned encoder classifies a record in milliseconds on modest hardware, which matters for real-time routing of millions of tickets. Prompted classification costs per call, though batch processing and prompt caching reduce it a lot. At thousands of records a day, cost rarely decides the question.

4. What you need back

A fine-tuned classifier returns labels and scores. Prompting can return labels, a one-line reason, the quote that justifies the label and extracted fields such as the feature mentioned. If people will read and trust the output, the reason and quote are often worth more than a few points of accuracy.

5. Who maintains it

Fine-tuning brings a training pipeline, a labeled dataset that has to grow with the taxonomy, model versioning and evaluation on every retrain. Prompting still needs an eval set and regression tests, but changes are text edits. Be honest about who will own retraining in a year; the hidden costs of building feedback analytics in-house tend to show up there.

6. Consistency between runs

A fine-tuned classifier returns the same output for the same input. A prompted model can vary, and current Claude models such as Sonnet 5.5 and Haiku 5.5 do not accept temperature settings, so variation is something you measure rather than switch off. The risk is largest when the model also decides what the labels are. In Enterpret's study of run-to-run instability across eight frontier models, raw models asked for the top themes in the same feedback changed more than half of their theme set between runs, and their counts on matched themes moved 7.5 to 9.8%. Fixing the label set in the prompt removes most of that, but you should still test agreement across two runs.

The middle path: prompt first, distill later

Many teams do both. Start by prompting Claude with clear label definitions, have people review a sample, and build a golden dataset for feedback classification evals. Once a branch of the taxonomy is stable and has enough reviewed examples, fine-tune a small model for it and keep the LLM for the long tail and for new themes. Route by confidence: the small model handles records it is sure about, and the rest go to the LLM.

Where Enterpret fits

Both options leave the hardest job with you: designing a taxonomy that matches your product and keeping it current as the product changes. Enterpret's Adaptive Taxonomy is built from your product, documentation and feedback, classifies each record at ingestion and proposes changes as new patterns appear, so neither retraining nor prompt rewrites fall on your team. For a direct comparison, see Enterpret vs building a feedback classifier on Claude.

Example: the classifier that aged out

Picture a support team that fine-tuned an encoder on 14 ticket categories and got strong results on its test set. Two quarters later the product shipped a new workspace model and a usage-based plan. Tickets about both landed in "Other" or the closest old category, and the weekly report quietly stopped matching what the support team heard. Retraining meant labeling a fresh set for three new categories. What worked: speed and cost on the stable categories. What did not: treating the label set as finished.

What changes next

As small models get better and cheaper to fine-tune, the accuracy question will matter less than the label question. The teams that do well will be the ones that treat their taxonomy as a maintained product, whichever model sits underneath it.

FAQ

Is fine-tuning more accurate than prompting for text classification?

Often, for fixed and narrow tasks. Bucher and Martini found fine-tuned smaller models beat zero-shot GPT-4 and Claude Opus on all four tasks they tested, and Edwards and Camacho-Collados found fine-tuning can still beat few-shot prompting across 16 datasets. Both used 2024 models, so test current models on your own data.

How many labeled examples do I need to fine-tune a feedback classifier?

In Bucher and Martini's tests, performance began to level off at about 200 labels, with a sweet spot of 200 to 500. For a feedback taxonomy, that applies per class, which is why rare themes are hard to fine-tune.

Which small models do teams fine-tune for classification?

BERT-style encoders such as RoBERTa and DeBERTa are common, and were among the strongest in Bucher and Martini's study. Small open-weight LLMs are another option when you want generated output as well as labels.

Does few-shot prompting close the gap?

Not reliably in the published studies. Edwards and Camacho-Collados found fine-tuned smaller models could still beat few-shot prompting, and Bucher and Martini saw no gain from informal few-shot tests. Clear label definitions and an eval set usually help more than extra examples.

If you would rather not own a classifier at all, see how Enterpret works with your own feedback.

Research cited

  1. Bucher, M. J. J., & Martini, M. (2024). Fine-Tuned "Small" LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification. arXiv:2406.08660.
  2. Edwards, A., & Camacho-Collados, J. (2024). Language Models for Text Classification: Is In-Context Learning Enough? arXiv:2403.17661.
  3. Bagal, V., Ganta, R., & Chellingi, P. (2026). Same Feedback, Different Answer: Measuring Run-to-Run Instability in Frontier-Model Customer Feedback Analysis. arXiv:2610.08036.

Heading

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

This is some text inside of a div block.
Related Guides
See all guides

AI That Learns Your Business

Generic AI gives generic insights. Enterpret is trained on your data to speak your language.

Book a demo

Start transforming feedback into customer love.

Leading companies like Perplexity, Notion and Strava power customer intelligence with Enterpret.

Book a demo