Why does one AI model cost a tenth of another from the same company, yet still answer most of your customer questions just as well? The short answer is a training technique called model distillation, and in 2026 it has become one of the most argued-about ideas in the AI industry.
This guide explains what distillation is in plain language, why OpenAI, Anthropic and the US government are suddenly talking about it, and what it means for a Hong Kong business owner choosing which AI tool to pay for.
What is AI model distillation?
AI model distillation is a training method in which a large, expensive AI model (the "teacher") is used to train a smaller, cheaper model (the "student"). The student studies the teacher's answers until it can imitate them closely. The result is a model that keeps much of the teacher's skill while running faster and costing far less.
Think of a master dim sum chef training an apprentice. The apprentice never reads the chef's 30 years of experience. Instead, they watch the chef fold thousands of har gow and copy the result. After a few months, the apprentice cannot invent new dishes, but they can produce the restaurant's best sellers at speed.
That is distillation. The apprentice is smaller and cheaper to keep in the kitchen, and for the everyday menu, most customers cannot tell the difference.
How does AI model distillation work?
Distillation works in three steps: developers send a large teacher model many example questions, collect its answers, and then train a smaller student model to reproduce those answers. Modern versions also copy the teacher's written reasoning steps, which gives the student far richer material to learn from than final answers alone.
Step 1: Ask the teacher
Engineers feed the teacher model a large set of prompts. These might be customer service questions, pieces of code or maths problems.
Step 2: Record the answers
The teacher's outputs are saved as training material. Some teachers can also show their working, the step-by-step reasoning behind an answer.
Step 3: Train the student
The smaller model is trained until its answers match the teacher's as closely as possible on the same prompts.
ETH Zurich professor Florian Tramèr compared it to maths homework, as reported by eWeek: learning from a book that shows only final answers is much harder than learning from one that walks through every step. That is why AI companies now guard their models' reasoning far more tightly than their plain answers.
A well-known early example is DistilBERT, released by Hugging Face in 2019. It cut the size of Google's BERT model by nearly half, ran about 60% faster and kept roughly 97% of its performance on standard tests.
Why are AI companies fighting about distillation in 2026?
AI companies are fighting because distillation can be done without permission. A rival can open thousands of accounts, ask a top model millions of questions and use the answers to train its own model. In 2026, Anthropic, OpenAI and a US government agency all published accusations that this was happening at industrial scale.
The technique itself is legal and widely used. The dispute is about consent and scale.
--- February 2026: Anthropic said Chinese AI firms DeepSeek, Moonshot AI and MiniMax used around 24,000 fraudulent accounts to generate more than 16 million exchanges with Claude, targeting coding and reasoning skills.
--- 8 September 2026: the US Cybersecurity and Infrastructure Security Agency issued advisory AA26-251A, naming six China-based AI companies. The same advisory states that distillation is a legitimate and useful technique in AI research.
--- 1 October 2026: OpenAI disclosed that it had disrupted a campaign that peaked on 24 and 25 July with 16,000 attempted extraction requests from more than 4,000 users, according to The Hacker News. OpenAI attributed the core activity to people associated with Moonshot AI, which has rejected similar claims.
There is an awkward side to the story too. Both OpenAI and Anthropic trained their own models partly on other people's copyrighted work, and both face lawsuits over it. So the question being argued in courts and governments is not whether one model may learn from another, but who has to give permission first.
For a Hong Kong buyer, the dispute matters for one practical reason: price. eWeek reported in July 2026 that China-built models had captured about 60% of US business usage on OpenRouter, a popular AI marketplace, largely because they are cheap. Cheap is not wrong, but it makes "where did this model come from?" a fair question to ask any supplier.
Where do you already use distilled AI models?
If you have used a "mini", "flash", "lite" or "haiku" version of a popular AI tool, you have used the kind of small, fast model that distillation was invented to create. These tiers handle short, routine tasks such as summarising, sorting and drafting replies at a fraction of the price of the flagship models.
AI vendors rarely publish the exact recipe behind each small model, so you should not assume every cheap tier is distilled. But the business logic is the same: a big model does the hard research once, and a smaller one serves millions of everyday requests.
The price gap is real. Anthropic's Claude Haiku 5.5, released on 7 October 2026, is listed in third-party coverage at US$0.10 per million input word pieces (called tokens) and US$0.50 per million output tokens for prompts under 100,000 tokens. That is one tenth of the previous Haiku 4.5 rates of US$1 and US$5.
What that means in Hong Kong dollars
Imagine a florist that wants AI to sort 10,000 WhatsApp and email messages a month into "order", "complaint", "delivery question" and "spam". At roughly 500 tokens in and 100 tokens out per message, that is 5 million input tokens and 1 million output tokens.
--- At Haiku 4.5 rates, the job costs about US$10 a month (around HK$78).
--- At Haiku 5.5 rates, it costs about US$1 a month (under HK$8), or roughly US$1.30 if the newer counting method produces about 30% more tokens, as some reviewers report.
For this kind of repetitive sorting, the cheap model is the sensible choice. We explain which everyday jobs suit it in our Claude Haiku 5.5 task guide.
What do business owners get wrong about distilled models?
The three most common mistakes are assuming small models are bad at everything, assuming every cheap model is stolen, and assuming you can save money by distilling ChatGPT yourself. Each one leads to a poor buying decision, either paying too much for simple work or taking on legal and security risk you did not need.
Misconception 1: "Smaller means worse"
A distilled model is usually worse at difficult, open-ended thinking, such as a complex contract review. For narrow, repeated tasks like tagging enquiries or drafting standard replies, the difference is often invisible to customers. Paying flagship prices for that work is like hiring a head chef to wash vegetables.
Misconception 2: "Distilled means stolen"
Most distillation is done by AI companies on their own models, with full permission. The 2026 disputes concern a specific practice: using someone else's model, through fake accounts, against its terms of service.
Misconception 3: "I can distil ChatGPT and stop paying"
The major AI providers' terms of service prohibit using their outputs to build competing models. Beyond the legal risk, training and hosting your own model needs engineers, servers and ongoing testing. For almost every SME, buying a cheap tier from a reputable vendor is far less expensive than building one.
How should a small business choose between a big model and a small one?
Match the model to the task, not to the brand. Send routine, high-volume work to a small, cheap model, keep complex or high-stakes work on a flagship model, and test both on your own real examples before deciding. Also check where any unfamiliar budget model comes from and where it stores your data.
A simple five-step routine
--- List your AI jobs. Write down every task you use AI for, such as replying to enquiries, writing product descriptions or summarising meetings.
--- Sort by difficulty. Mark each one "routine" (same pattern every time) or "judgement" (needs careful reasoning or carries legal or money risk).
--- Run a 20-example test. Give the same 20 real cases to a small model and a large one. If you cannot tell the answers apart on a routine task, use the small one.
--- Check the origin. For an unknown low-cost model, ask who built it, where it is hosted and whether its training data is documented. eWeek notes that training-data provenance is now tied to export policy debates in Washington.
--- Protect customer data. Hong Kong's Personal Data (Privacy) Ordinance still applies whichever model you choose, so confirm where conversations are stored before you send customer names or phone numbers.
A typical result for a ten-person trading company: enquiry sorting and first-draft replies move to a small model, while quotation checks and supplier disputes stay with a flagship model and a human reviewer. Monthly AI spend falls, and nothing important is left to the cheapest tool.
Frequently asked questions about AI model distillation
These are the questions Hong Kong business owners ask most often about distilled models: whether they are legal, whether they are safe for customer data, whether prices will keep falling, and whether a small business ever needs to train its own model. The short answers are below, each one self-contained.
Is AI model distillation legal?
Yes. Distillation is a standard research technique, and AI companies use it on their own models every day. What providers object to is using their models without permission, through fake accounts, to train a rival product.
Is a distilled model safe to use with customer data?
Safety depends on the vendor, not on distillation itself. A small model from an established provider with clear business data terms carries the same data protections as that provider's flagship model.
Will small AI models keep getting cheaper?
The trend points that way. Claude Haiku 5.5 launched at one tenth of its predecessor's price in October 2026, and competition between providers keeps pushing the cost of routine AI work down.
Does my business need to train its own model?
Almost certainly not. A ready-made small model, given clear instructions and a few examples of your own replies, handles most SME tasks without any custom training.
The bottom line on AI model distillation
Distillation is the reason powerful AI has become cheap enough for a ten-person shop. A big model does the expensive learning, and a smaller one serves your everyday requests at a fraction of the price. The 2026 headlines are about permission, not about whether the small models work.
For a business owner, the practical lesson is simple: stop paying flagship prices for routine work, test cheap tiers on your own examples, and buy from vendors whose origins and data terms you can check.
Choosing the right model is less about technology and more about knowing your own work. We understand AI. UD stands with you.
Reviewed by the UD AI team.
Not sure which AI model or tool fits each job in your business? Browse the UD AI Directory and we will walk you through it step by step, from sorting your routine and high-stakes tasks to matching each one with a tool you can trust.