Glossary
Inference
Definition
Inference is running a trained model to produce an output, such as a prediction, an answer or generated text.
Why it matters when you’re buying
Training happens now and then, but inference happens every time someone uses the model, so it is where most ongoing AI cost and latency sit. Pricing per token, per request or per GPU hour can change the economics of a project.
How it relates to other terms
Large models usually run on GPU clusters or accelerated cloud services, and MLOps covers how models are deployed and monitored for inference.
Where to go next
- AI & ML Platforms: news, companies and community questions
- How to evaluate AI & ML Platforms vendors: criteria, questions to ask and red flags
- MLOps and inference: the latest stories and threads on this topic
Written by Techarda editors · Updated . Think a definition is wrong or missing something? Email engage@techarda.com. All terms.