Small language models: size, uses, and tradeoffs
MarkdownHere's everything you need to know about small language models, including how they differ from LLMs, what they're best used for, and how much they cost
What are small language models?
Even though the flashiest LLMs keep getting bigger and more powerful, there are still smaller large language models—or small language models—that are more than capable of working effectively in the real world. And that's before you even factor in the cost of running large language models. In most cases, you should be employing the smallest language model that will reliably do the job you need it to, not the largest one you can.
So let's take a look at small language models, how they differ from large language models, and what they're good for.
SLM vs. LLM: What is a small language model?
A small language model (SLM) is a language model that is smaller than current large language models. The label describes relative size rather than a fixed boundary.
SLMs and resource use
Adding more parameters also comes at a cost. And once again, all else being equal, the more parameters you add, the higher the costs are going to be.
Smaller models can require less computing capacity to run, although actual requirements depend on the model and hardware.
Advantages of small language models
Small language models are faster and cheaper to run
Smaller language models can cost less to run and may generate output faster than larger ones. Actual cost and speed depend on the model, task, and hardware.
Small language models are faster and cheaper to train
Smaller models may require less computing capacity to train or fine-tune when compared on similar hardware.
Some small language models can run on local devices
Some small language models are designed to run locally on smartphones, laptops, or other low-powered devices.
Local processing can avoid a network round trip, though total response time depends on the model and device.
Local processing can keep inputs on the device when the full workflow runs locally; confirm how the application handles data.
A locally available model may integrate with on-device workflows, depending on the application.
Should you use a small language model?
Both large language models and small language models have their uses.
Model selection depends on the quality a task requires and the resources available.
Put this guide to work with Ailoot
Connect AI models to your applications, agents and automations through one Ailoot API. Choose the model that fits your task and follow the Ailoot documentation to get started.