← AI Terminology
GPT - Generative Pre-trained Transformer
GPT stands for Generative Pre-trained Transformer: a family of large language models from OpenAI, trained by predicting the next token in a sequence — the architecture that sparked the modern LLM era and produced ChatGPT.
Each GPT generation (GPT-1 through GPT-4o) dramatically scaled capabilities.
Each GPT generation (GPT-1 through GPT-4o) dramatically scaled capabilities.
Why It Matters in AI
GPT demonstrated that a single objective — predict the next word — applied at sufficient scale produces a model with broad language understanding, reasoning, and generation capability. GPT-3 (2020, 175B parameters) showed in-context learning; GPT-4 (2023) became the first AI to pass bar exams, write production code, and reason multimodally. ChatGPT (GPT-3.5) was the fastest-growing consumer application in history, bringing LLMs to 100M users in 2 months.
Key Points
| Aspect | Description |
|---|---|
| GPT-3 | 175B parameters (2020) — demonstrated few-shot and zero-shot capability without fine-tuning |
| GPT-4 | Multimodal (text + image input), ~1.8T parameters (estimated MoE), state-of-the-art performance |
| GPT-4o | Omni model (2024): native multimodal, real-time voice, faster and cheaper than GPT-4 |
| Competitors | Claude (Anthropic), Gemini (Google), Llama (Meta), Mistral, Grok (xAI) |
| Architecture | Transformer decoder-only — causal (left-to-right) self-attention, no encoder |
| Pre-training | Next-token prediction on internet-scale text — billions of tokens from Common Crawl, Books, etc. |
Simple Analogy
GPT learned to write by reading the entire internet and practising "what word comes next?" billions of times. By the time it can reliably predict the next word in a legal contract, a scientific paper, or a Python program, it has implicitly learned the structure of all these domains — and can generate, translate, summarise, and answer questions about them.
Common Usage Examples
from openai import OpenAI; client.chat.completions.create(model="gpt-4o", messages=[...])- ChatGPT: GPT-3.5/4 with RLHF alignment — 100M users in first 2 months (2023)
- GitHub Copilot: GPT-4-class model fine-tuned on code — powers AI code completion in IDEs
- GPT-4V: vision input —
messages=[{"role":"user","content":[{"type":"image_url","image_url":...}]}] - Function calling:
tools=[{"type":"function","function":{...}}]— GPT-4 as AI agent backbone
Summary
In short: GPT is OpenAI's line of decoder-only Transformers trained by next-token prediction — the architecture that produced ChatGPT and defined the modern LLM era.