← AI Terminology
Ollama
Ollama is a developer-friendly tool to download, run, and serve open LLMs locally with simple CLI/API UX, built on llama.cpp-class inference.
It became a default on-ramp for local model experimentation.
It became a default on-ramp for local model experimentation.
Why It Matters in AI
Raw engines scare newcomers; Ollama wraps model pull, GGUF management, and an API into minutes-to-first-token local AI. Teams prototype offline agents and private RAG with it.
Key Points
| Aspect | Description |
|---|---|
| UX | ollama run llama3 simplicity |
| API | Local HTTP compatible with common clients |
| Limits | Not always max throughput vs tuned vLLM GPU servers |
| Models | Library of popular open weights |
| Related | llama.cpp, GGUF, on-device AI |
| Modelfile | Customise system prompts and params |
Simple Analogy
Docker-like convenience for local brains — pull a model name and chat, without hand-building engines.
Common Usage Examples
ollama pull mistralollama runinteractive chat- Point LangChain at local Ollama base URL
- Modelfile for custom system prompt
Summary
In short: Ollama makes running open LLMs locally as easy as pulling a model name — the friendly front door to llama.cpp-class inference.