AI Terminology — invest-in.ai

A2A Protocol - Agent-to-Agent A2A (Agent-to-Agent) protocols define standard ways for autonomous agents to discover, authenticate, message,…

Accuracy Accuracy is the fraction of predictions a model gets correct out of all predictions made. It is computed as …

Activation Function An activation function is a mathematical function applied to a neuron's output that introduces non-linearity,…

Activation Steering Activation steering edits model behaviour at inference by adding direction vectors into activations (or simil…

Active Learning Active learning lets a model select the most informative unlabeled examples for human annotation to maximise …

Actor-Critic Actor-Critic is a reinforcement learning architecture that trains two components simultaneously: an actor tha…

Adam Optimizer Adam (Adaptive Moment Estimation) is an optimisation algorithm that adapts the learning rate for each paramet…

AdamW AdamW is the Adam optimiser with decoupled weight decay — L2 regularisation applied directly to weights rathe…

Adapter Layers Adapter layers are small trainable modules inserted into a frozen pretrained network (often bottleneck MLPs a…

Advantage Function The advantage function A(s,a) = Q(s,a) − V(s) measures how much better an action is than the average action i…

Adversarial Example An adversarial example is an input crafted with small, deliberate perturbations — often imperceptible to huma…

Adversarial Robustness Adversarial robustness is a model's ability to maintain correct predictions when its inputs are intentionally…

Agent An agent is an AI system that perceives its environment, makes decisions, and takes actions — often in a loop…

Agentic AI Agentic AI refers to AI systems that operate autonomously over extended tasks — perceiving context, planning …

Agentic RAG Agentic RAG lets an agent decide iteratively what to retrieve, whether to re-query, which tools to use, and w…

Agent Memory Agent memory is the store of information an agent retains across steps or sessions — short-term working conte…

AGI - Artificial General Intelligence AGI stands for Artificial General Intelligence: a hypothetical AI system that can perform any intellectual ta…

AI Alignment AI alignment is the technical and philosophical challenge of ensuring AI systems pursue goals that are actual…

AI Governance AI governance is the set of laws, regulations, standards, frameworks, and voluntary norms used to guide the d…

AI Safety AI safety is the research field focused on ensuring that AI systems behave in ways that are safe, beneficial,…

Algorithm An algorithm is a finite, ordered set of instructions that transforms input into output — a precise recipe fo…

Alignment Alignment in AI refers to the degree to which an AI system's behaviour matches the intentions, values, and go…

AMD Instinct AMD Instinct is AMD's line of data-centre GPU accelerators designed for AI training and HPC workloads, compet…

ANN - Approximate Nearest Neighbor Approximate nearest neighbour (ANN) algorithms find vectors that are close to a query without scanning every …

Anomaly Detection Anomaly detection is the task of identifying data points, events, or observations that deviate significantly …

Apple Silicon (MLX) Apple Silicon refers to Apple's ARM-based SoC family (M1, M2, M3, M4) with unified memory shared between CPU,…

Artificial Intelligence - AI Artificial Intelligence (AI) refers to computer systems designed to perform tasks that would normally require…

ASI - Artificial Superintelligence ASI stands for Artificial Superintelligence: a hypothetical AI system that surpasses human cognitive ability …

Attention Mechanism Attention mechanism is a technique that lets a neural network dynamically weight how much each part of its in…

Attention Sink Attention sink refers to the finding that early tokens (especially the first) attract disproportionate attent…

AUC - Area Under the Curve AUC stands for Area Under the (ROC) Curve: a single scalar measuring a classifier's ability to distinguish be…

Augmentation (CV) Augmentation in computer vision (CV) is the practice of applying label-preserving transformations to training…

Autoencoder An autoencoder is a neural network trained to compress input data into a low-dimensional latent representatio…

AutoML - Automated Machine Learning AutoML stands for Automated Machine Learning: the use of algorithms to automate the process of selecting, bui…

Autoregressive Generation Autoregressive generation produces a sequence step by step, where each new token is sampled conditioned on al…

AWQ / GPTQ AWQ (Activation-aware Weight Quantization) and GPTQ (Generative Pre-trained Transformer Quantization) are two…

Backdoor Attack A backdoor attack plants a hidden trigger during training or supply chain compromise so the model behaves nor…

Backpropagation Backpropagation is the algorithm that computes gradients of the loss function with respect to every weight in…

Bagging - Bootstrap Aggregating Bagging (Bootstrap Aggregating) is an ensemble technique that trains multiple independent models on different…

Batch A batch is a subset of the training dataset processed together in one forward and backward pass before updati…

Batch Normalization Batch Normalization is a technique that normalises a layer's inputs to zero mean and unit variance across the…

Bayesian Optimization Bayesian optimization is a sequential strategy for optimising expensive-to-evaluate black-box functions by bu…

Beam Search Beam search is a heuristic search algorithm that generates sequences (text, translations, speech) by maintain…

Bellman Equation The Bellman equation expresses the value of a state (or state-action pair) in reinforcement learning as the i…

Benchmark A benchmark is a standardised test or dataset used to measure and compare the performance of AI models on a d…

BERT - Bidirectional Encoder Representations from Transformers BERT stands for Bidirectional Encoder Representations from Transformers: a Transformer encoder pre-trained by…

Best-of-N Sampling Best-of-N sampling generates N candidate completions and picks the one with the highest score under a reward …

BF16 - Brain Floating Point BF16 (bfloat16) is a 16-bit float with the same exponent range as FP32 but fewer mantissa bits — popularised …

Bias & Fairness Bias in ML refers to systematic errors that cause a model to produce unfair or inaccurate outputs for specifi…

Bias (in Evaluation) Evaluation bias refers to systematic errors in how a model's performance is measured — including data leakage…

Bias-Variance Tradeoff The bias-variance tradeoff is the fundamental tension in supervised learning between a model that is too simp…

BIG-bench BIG-bench (Beyond the Imitation Game Benchmark) is a collaborative suite of hundreds of diverse tasks probing…

BitNet / 1-bit LLM BitNet refers to extreme low-bit neural net lines (including 1-bit / 1.58-bit ternary weight research) aiming…

BLEU Score - Bilingual Evaluation Understudy BLEU stands for Bilingual Evaluation Understudy: an automatic metric for evaluating machine translation quali…

BM25 BM25 is a ranking function for lexical retrieval scoring documents by term frequency, inverse document freque…

Boosting Boosting is an ensemble method that sequentially trains weak learners (usually shallow decision trees), each …

BPE - Byte Pair Encoding BPE stands for Byte Pair Encoding: a subword tokenization algorithm that starts with individual characters an…

Brier Score The Brier score is a metric that measures the accuracy of probabilistic predictions by computing the mean squ…

Calibration Calibration is the degree to which a model's predicted probabilities match actual outcome frequencies — a per…

Capsule Network A Capsule Network (CapsNet) is a neural network architecture proposed by Geoffrey Hinton (2017) that replaces…

Catastrophic Forgetting Catastrophic forgetting is when a neural network trained on a new task rapidly loses performance on previousl…

Causal Masking Causal masking (look-ahead masking) prevents attention from seeing future tokens: position i may attend only …

Chain-of-Thought - CoT Chain-of-Thought (CoT) is a prompting technique where a model is instructed or demonstrated to produce interm…

Chat Template A chat template is the model-specific string format that serialises roles (system/user/assistant/tool) and sp…

Chunking Strategy Chunking strategy is how source documents are split into segments for embedding and retrieval — sizes, overla…

Classifier-Free Guidance Classifier-free guidance (CFG) steers diffusion sampling by extrapolating between conditional and uncondition…

Class Imbalance Class imbalance occurs when some labels are much rarer than others, so naive accuracy is misleading and learn…

CLIP - Contrastive Language-Image Pretraining CLIP stands for Contrastive Language-Image Pretraining: a model from OpenAI (2021) trained to align images an…

CLIP Score CLIP Score measures alignment between an image and a text caption using CLIP embeddings’ similarity — higher …

Clustering (K-Means, DBSCAN) Clustering is an unsupervised learning task that groups data points into clusters based on similarity, withou…

CNN - Convolutional Neural Network A CNN (Convolutional Neural Network) is a neural network architecture that uses convolutional layers to autom…

Code Interpreter A code interpreter is a sandboxed runtime (usually Python) that an LLM can write code for, execute, and read …

ColBERT ColBERT is a late-interaction retrieval model that encodes queries and documents into multi-vector token-leve…

Common Crawl Common Crawl is a nonprofit’s massive ongoing scrape of the public web, published as open WARC datasets widel…

Compound AI System A compound AI system combines multiple models, retrievers, tools, and control logic into one product — not a …

Computer Use Computer use is an agent capability where a model operates a computer GUI or browser — seeing screenshots or …

Concept Drift Concept drift is when the relationship between inputs and the target changes over time (P(y|x) shifts), even …

Confusion Matrix A confusion matrix is a table that shows the counts of true positives, true negatives, false positives, and f…

Constitutional AI Constitutional AI (CAI) is an alignment technique developed by Anthropic where a model is given a set of writ…

Constrained Decoding Constrained decoding restricts the tokens a model may emit at each step so outputs obey a grammar, regex, or …

Context Engineering Context engineering is the discipline of selecting, structuring, and budgeting everything placed in an LLM’s …

Contextual Bandit A contextual bandit chooses actions based on observed context features each round, receives a reward for the …

Context Window A context window is the maximum amount of text (measured in tokens) that a language model can process in a si…

Continual Learning Continual learning (lifelong learning) trains models on a stream of tasks or data distributions over time whi…

Continued Pretraining Continued pretraining (domain-adaptive pretraining) further trains a foundation model with the original self-…

Continuous Batching Continuous batching (also called in-flight batching) is an LLM serving technique where new requests are dynam…

Contrastive Learning Contrastive learning is a self-supervised learning approach where a model is trained to bring representations…

ControlNet ControlNet adds spatial conditioning paths to diffusion models so generation follows edges, depth, pose, segm…

Convergence Convergence in machine learning refers to the state where a model's training loss (and validation loss) stabi…

Convolutional LSTM A Convolutional LSTM (ConvLSTM) is a recurrent neural network architecture that replaces the matrix multiplic…

Copilot Pattern The copilot pattern pairs AI suggestions with a human operator who stays in control — AI drafts or proposes, …

Cosine Decay Cosine decay anneals the learning rate following a cosine curve from a peak down toward a minimum over traini…

Cosine Similarity Cosine similarity measures the cosine of the angle between two vectors — 1 for same direction, 0 for orthogon…

Cost Function A cost function (also called a loss function or objective function) is a mathematical function that quantifie…

CoWoS - Chip on Wafer on Substrate CoWoS (Chip on Wafer on Substrate) is TSMC's 2.5D advanced packaging technology that bonds multiple chiplets …

Cross-Attention Cross-attention is attention where queries come from one sequence and keys/values from another (e.g. decoder …

Cross-Entropy Cross-entropy is a loss function that measures how different a predicted probability distribution is from the…

Cross-Validation Cross-validation is a model evaluation technique that splits data into multiple folds, trains the model on di…

CUDA - Compute Unified Device Architecture CUDA stands for Compute Unified Device Architecture. It is a parallel computing platform and programming mod…

cuDNN - CUDA Deep Neural Network Library cuDNN stands for CUDA Deep Neural Network Library: NVIDIA's closed-source GPU-accelerated library of primitiv…

Curriculum Learning Curriculum learning is a training strategy where examples are presented to a model in a meaningful order — ty…

DALL-E DALL-E is OpenAI's text-to-image generation model series — named after Salvador Dalí and Pixar's WALL-E — cap…

Data Augmentation Data augmentation is the practice of creating additional training examples by applying label-preserving trans…

Data-Centric AI Data-centric AI emphasises systematically improving datasets (labels, cleaning, coverage) as the primary leve…

Data Deduplication Data deduplication removes exact or near-duplicate documents/examples from training or evaluation sets to imp…

Data Drift Data drift is a change in the input distribution seen by a deployed model compared to training or earlier pro…

Data Mixture Data mixture is the proportional recipe of sources/domains in a training run (web, code, math, multilingual, …

Data Parallelism Data parallelism is a distributed training strategy where the same model is replicated across multiple device…

Data Poisoning Data poisoning is an adversarial attack where an attacker manipulates the training data of a machine learning…

Dataset A dataset is a structured collection of examples — inputs paired with labels (supervised) or inputs alone (un…

Deceptive Alignment Deceptive alignment is a hypothetical or observed regime where a model appears aligned during training/oversi…

Decision Tree A decision tree is a supervised ML model that makes predictions by repeatedly splitting the data on feature t…

Deepfake A deepfake is synthetic media (often faces/voices) generated or manipulated by deep learning to appear real, …

Deep Learning Deep learning is a subfield of machine learning that uses neural networks with many layers (hence "deep") to …

DeepSpeed ZeRO ZeRO (Zero Redundancy Optimizer) in DeepSpeed partitions optimiser states, gradients, and parameters across d…

DenseNet - Dense Convolutional Network DenseNet (Dense Convolutional Network) is a CNN architecture where every layer is connected to every subseque…

Dense Retrieval Dense retrieval encodes queries and documents into continuous embedding vectors and retrieves by nearest-neig…

Depth Estimation Depth estimation is the computer vision task of predicting the distance from a camera to each pixel in an ima…

DETR - Detection Transformer DETR stands for Detection Transformer: an object detection model from Facebook AI (2020) that frames detectio…

Differential Privacy Differential privacy (DP) is a mathematical framework for privacy-preserving data analysis that provides a ri…

Diffusion Model A diffusion model is a generative model that learns to reverse a gradual noise-addition process — it is train…

Dimensionality Reduction (PCA, t-SNE, UMAP) Dimensionality reduction is the process of transforming high-dimensional data into a lower-dimensional repres…

Distributed Training Distributed training is the practice of splitting the work of training a neural network across multiple compu…

Domain Adaptation Domain adaptation adapts a model trained on a source distribution to perform well on a different but related …

DoRA - Weight-Decomposed Low-Rank Adaptation DoRA decomposes weights into magnitude and direction and applies LoRA-style updates primarily to direction, i…

DPO - Direct Preference Optimization DPO stands for Direct Preference Optimization: an alignment technique that trains a language model to prefer …

DQN - Deep Q-Network DQN stands for Deep Q-Network: the first algorithm to combine Q-learning with a deep neural network function …

Dropout Dropout is a regularisation technique where, during each training step, a random fraction of neurons is tempo…

Dynamic Batching Dynamic batching groups inference requests that arrive close in time into a batch to raise GPU utilisation, w…

Early Exit Early exit attaches intermediate predictors so easy inputs can leave a deep network at shallower layers, savi…

Early Stopping Early stopping is a regularisation technique that halts training when the model's performance on a validation…

Edge Detection Edge detection is a computer vision technique that identifies sharp boundaries between regions in an image — …

EfficientNet EfficientNet is a family of CNN architectures from Google (2019) that scales width, depth, and input resoluti…

Elo Rating (for Models) Elo rating is a competitive rating system — originally developed for chess — adapted for AI model evaluation …

Embedding An embedding is a dense, low-dimensional vector representation of a discrete object (word, sentence, image, u…

Embedding Model An embedding model maps text, images, or other inputs into fixed-size vectors that place similar items close …

Emergent Abilities Emergent abilities in LLMs are capabilities that appear suddenly and unpredictably at certain model scales — …

Emergent Behavior Emergent behavior in AI refers to system-level properties or actions that arise from the interaction of simpl…

Encoder-Decoder An encoder-decoder architecture is a neural network design where an encoder compresses input into a latent re…

Energy Efficiency (Green AI) Green AI is the movement to measure, minimise, and report the energy consumption and carbon footprint of AI t…

Ensemble Learning Ensemble learning is the practice of combining predictions from multiple models to produce a single, more acc…

Epoch An epoch is one complete pass through the entire training dataset — every training example has been seen exac…

Evolutionary Algorithms Evolutionary algorithms (EAs) are optimisation methods inspired by biological evolution — maintaining a popul…

Exact Match Exact match (EM) scores 1 only when a predicted answer string equals the reference (after simple normalisatio…

Experience Replay Experience replay is a technique in deep reinforcement learning where past interactions (state, action, rewar…

Experiment Tracking Experiment tracking logs parameters, metrics, code versions, and artifacts for each training or eval run so r…

Explainable AI - XAI Explainable AI (XAI) is the field of methods and practices that make AI model predictions understandable to h…

Exploding Gradient Exploding gradients occur during backpropagation when gradient magnitudes grow exponentially as they propagat…

Exploration vs Exploitation Exploration vs exploitation is the fundamental dilemma in reinforcement learning and decision-making: should …

F1 Score The F1 score is the harmonic mean of Precision and Recall — a single metric that balances the tradeoff betwee…

Fairness Metric A fairness metric is a quantitative measure that captures whether an ML model's predictions or errors are equ…

FAISS - Facebook AI Similarity Search FAISS is a library from Meta for efficient similarity search and clustering of dense vectors, including exact…

Feature Engineering Feature engineering is the craft of transforming raw data into input variables that make learning easier — ra…

Features Features are the individual measurable properties or input variables fed into a machine learning model to mak…

Federated Learning Federated learning is a distributed machine learning approach where a model is trained across many decentrali…

Feedforward Network A feedforward network (also called a multilayer perceptron or fully connected network) is a neural network wh…

Few-Shot Learning Few-shot learning is the ability of a model to generalise to new tasks or classes from very few labelled exam…

FID - Fréchet Inception Distance FID (Fréchet Inception Distance) compares distributions of real and generated images in Inception network fea…

Fine-Tuning Fine-tuning is the process of taking a pre-trained model and continuing to train it on a smaller, task-specif…

FineWeb / The Pile The Pile is an influential curated 800GB open language-modelling dataset; FineWeb is a modern large high-qual…

Flash Attention Flash Attention is an IO-aware, memory-efficient implementation of the attention mechanism that computes exac…

Flow Matching / Rectified Flow Flow matching trains continuous normalising flows by regressing vector fields that transport noise to data; r…

Flow Model A flow model (normalising flow) is a generative model that learns an invertible mapping between a simple dist…

Fork A fork is an independent copy of a codebase or model repository, created so that a developer or team can expe…

Foundation Model A foundation model is a large AI model trained on broad data at scale that can be adapted (fine-tuned, prompt…

FP8 - 8-bit Floating Point FP8 is an 8-bit floating-point format (variants E4M3/E5M2) used for low-precision training and inference on m…

FPN - Feature Pyramid Network FPN stands for Feature Pyramid Network: a multi-scale feature extraction architecture for computer vision tha…

Frontier Model A frontier model is a leading general-purpose model at the cutting edge of capability (and often cost), typic…

FSDP - Fully Sharded Data Parallel FSDP (Fully Sharded Data Parallel) shards model parameters, gradients, and optimiser states across data-paral…

Full Fine-Tuning Full fine-tuning updates all (or nearly all) model parameters on downstream data, unlike PEFT methods that fr…

Function Calling Function calling is a model capability and API pattern where the LLM emits structured calls to named tools (f…

GAE - Generalized Advantage Estimation GAE (Generalized Advantage Estimation) computes a exponentially weighted mix of multi-step TD advantages to t…

GAN - Generative Adversarial Network GAN stands for Generative Adversarial Network: a generative model architecture introduced by Ian Goodfellow (…

Gaussian Process A Gaussian Process (GP) is a probabilistic model that defines a distribution over functions — instead of pred…

Gaussian Splatting 3D Gaussian splatting represents scenes as clouds of anisotropic Gaussians rasterised quickly for real-time n…

GDPR / AI Act GDPR (General Data Protection Regulation) is the EU's data privacy law (2018) governing how personal data is …

GELU - Gaussian Error Linear Unit GELU (Gaussian Error Linear Unit) is a smooth activation: x · Φ(x), where Φ is the standard normal CDF — roug…

Generalization Generalization is the ability of a trained model to perform well on new, unseen data that it was not exposed …

GGUF GGUF is a binary file format for quantized ML models (successor spirit to GGML formats) widely used by llama.…

GLUE / SuperGLUE GLUE (General Language Understanding Evaluation) and SuperGLUE are benchmark suites for evaluating natural la…

GNN - Graph Neural Network A GNN (Graph Neural Network) is a neural network designed to operate on graph-structured data — learning node…

Goal Misgeneralization Goal misgeneralization occurs when an agent learns a proxy objective that performs well in training environme…

Golden Dataset A golden dataset is a curated, trusted set of inputs with reference answers or labels used as the fixed bench…

GPQA - Graduate-Level Google-Proof Q&A GPQA is a benchmark of extremely hard multiple-choice science questions written so that even experts struggle…

GPT - Generative Pre-trained Transformer GPT stands for Generative Pre-trained Transformer: a family of large language models from OpenAI, trained by …

GPU - Graphics Processing Unit A GPU (Graphics Processing Unit) is a massively parallel processor originally designed for rendering graphics…

GQA - Grouped-Query Attention GQA (Grouped-Query Attention) shares key/value heads across groups of query heads — a middle ground between m…

Gradient Accumulation Gradient accumulation sums gradients over multiple micro-batches before applying an optimiser step, simulatin…

Gradient Boosting (XGBoost, LightGBM) Gradient boosting is an ensemble method that builds a strong predictor by sequentially adding weak learners (…

Gradient Checkpointing Gradient checkpointing (activation checkpointing) saves memory by storing only a subset of activations and re…

Gradient Clipping Gradient clipping is a training technique that prevents exploding gradients by rescaling the gradient vector …

Gradient Descent Gradient descent is the optimisation algorithm that trains neural networks by iteratively moving model parame…

GraphRAG GraphRAG builds a knowledge graph (entities, relations, communities) from a corpus and retrieves via graph st…

GRPO - Group Relative Policy Optimization GRPO (Group Relative Policy Optimization) is a reinforcement learning algorithm that updates a policy using r…

GRU - Gated Recurrent Unit GRU stands for Gated Recurrent Unit: a simplified recurrent neural network architecture (Cho et al., 2014) th…

GSM8K / MATH GSM8K and MATH are grade-school and competition-math word-problem benchmarks used to measure multi-step quant…

Guardrails Guardrails are constraints, filters, and safety mechanisms applied to AI systems to prevent them from generat…

Hallucination Hallucination in AI refers to when a language model generates text that is factually incorrect, fabricated, o…

Hallucination Mitigation Hallucination mitigation is the set of techniques used to reduce the frequency and severity of AI-generated f…

HBM - High Bandwidth Memory HBM (High Bandwidth Memory) is a 3D-stacked DRAM standard that bonds multiple DRAM dies vertically with throu…

He Initialization He initialization (also called Kaiming initialization) is a weight initialisation scheme for neural networks …

HELM - Holistic Evaluation of Language Models HELM is a living evaluation framework from Stanford CRFM that measures LLMs across many scenarios and metrics…

HNSW - Hierarchical Navigable Small World HNSW is a graph-based ANN index that links vectors in a hierarchical small-world graph for fast approximate n…

Hugging Face Hugging Face is the leading open-source AI platform and community — providing the HuggingFace Hub (host for 5…

HumanEval HumanEval is a benchmark from OpenAI (Chen et al., 2021) consisting of 164 programming problems — each with a…

Human Evaluation Human evaluation is the practice of using human judgement — rather than automated metrics — to assess AI mode…

Human-in-the-Loop Human-in-the-loop (HITL) designs keep humans approving, correcting, or labeling critical steps in an AI workf…

Hybrid Architecture (Transformer–SSM) Hybrid architectures interleave or combine transformer attention blocks with SSM/linear modules to balance qu…

Hybrid Search Hybrid search combines lexical (e.g. BM25) and semantic (vector) retrieval, usually fusing scores or ranks to…

Hyperparameter A hyperparameter is a configuration value that controls the training process itself — as opposed to a paramet…

IFEval - Instruction-Following Eval IFEval evaluates precise instruction following using verifiable constraints (e.g. “include keyword X”, “exact…

Image Classification Image classification is the computer vision task of assigning a single label (or probability distribution ove…

Imitation Learning Imitation learning trains agents from expert demonstrations (state-action data) rather than from sparse rewar…

Inception Network The Inception network (GoogLeNet) is a CNN architecture from Google (2014) that introduced the "Inception mod…

In-Context Learning In-context learning (ICL) is the ability of large language models to perform new tasks from a handful of exam…

Inference Inference is the process of running a trained AI model on new input data to produce predictions, classificati…

Inference Engine An inference engine is a software system optimised for running trained AI models at production scale — handli…

Inference-Time Scaling Inference-time scaling is the empirical and engineering practice of improving model performance by allocating…

Inpainting Inpainting fills in missing or masked regions of an image (or other signal) with plausible content, optionall…

Instance Segmentation Instance segmentation is a computer vision task that simultaneously detects every object in an image and prod…

Instruction Following Instruction following is a model’s ability to comply with user/system directives — format, constraints, role,…

Instruction Tuning Instruction tuning (also called supervised fine-tuning for instruction following) is the process of fine-tuni…

Interpretability Interpretability is the degree to which a human can understand why an AI model produced a specific output — e…

Inter-Token Latency Inter-token latency (ITL) is the time between consecutive output tokens during decode (sometimes reported as …

IoU - Intersection over Union IoU (Intersection over Union) is a metric that measures the overlap between two regions — typically a predict…

Iteration Iteration in machine learning refers to one pass of the optimiser over a single mini-batch of data — updating…

Jailbreak A jailbreak is a prompt or technique that bypasses an AI model's safety guidelines, causing it to produce con…

JAX JAX is a Google open-source numerical computing library that combines NumPy-compatible array operations with …

Jevons Paradox Jevons Paradox (in AI) is the observation that as AI inference becomes cheaper and more efficient, total ener…

Kalman Filter The Kalman filter is a recursive Bayesian algorithm that estimates the true state of a dynamic system from no…

Keras Keras is a high-level deep learning API that provides a clean, modular interface for building and training ne…

K-Means K-means clusters data into K groups by iteratively assigning points to the nearest centroid and updating cent…

Knowledge Distillation Knowledge distillation is a model compression technique where a smaller "student" model is trained to mimic t…

Knowledge Graph A knowledge graph represents facts as a network of entities (nodes) and relationships (edges), often with typ…

KV Cache KV cache (Key-Value cache) is an inference optimisation for transformer models that stores the key and value …

KV-Cache Quantization KV-cache quantization stores attention key/value caches in lower precision (INT8/FP8/INT4) during generation …

L1/L2 Regularization L1 and L2 regularization are techniques that add a penalty term to the loss function proportional to the magn…

Label A label is the ground-truth output value associated with a training example — the correct answer that a super…

Label Smoothing Label smoothing replaces hard one-hot targets with softened distributions (e.g. 0.9 on the correct class, sma…

LangChain LangChain is an open-source Python/JavaScript framework for building applications with large language models …

Latent Diffusion Latent diffusion runs the diffusion process in a compressed latent space from a pretrained autoencoder instea…

Layer Normalization Layer normalization is a technique that normalises activations across the feature dimension of each individua…

Learning Rate The learning rate is a hyperparameter that controls how large each gradient descent step is — determining how…

Learning Rate Scheduler A learning rate scheduler is a component that automatically adjusts the learning rate during training accordi…

Linear Attention Linear attention reformulates attention so cost scales roughly O(n) in sequence length, typically via kernel …

LiveCodeBench LiveCodeBench continuously harvests new coding problems from contests to evaluate LLMs with reduced contamina…

llama.cpp llama.cpp is a popular C/C++ inference engine for running LLaMA-family and many other LLMs efficiently on CPU…

LlamaIndex LlamaIndex (formerly GPT Index) is an open-source Python framework focused on connecting LLMs to external dat…

LLM-as-Judge LLM-as-judge uses a language model to score or compare other model outputs (quality, safety, preference) when…

LLM - Large Language Model A large language model is a neural network trained on massive text corpora to predict the next token — at suf…

LLMOps LLMOps is the operational discipline for large language model applications: prompt/version control, eval harn…

LMSYS Arena LMSYS Chatbot Arena is a crowdsourced LLM benchmark platform developed by UC Berkeley's LMSYS team where huma…

Logistic Regression Logistic regression is a linear classification model that predicts class probabilities via a logistic/sigmoid…

Log-Loss Log-loss (logarithmic loss, also called binary cross-entropy for two-class problems) is a loss function that …

LoRA - Low-Rank Adaptation LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that freezes a pre-trained model's wei…

Loss Function A loss function (also called cost function or objective function) is a mathematical function that quantifies …

Lost in the Middle Lost in the middle is the failure mode where models use information at the start and end of a long context mo…

LSTM - Long Short-Term Memory LSTM (Long Short-Term Memory) is a type of recurrent neural network (RNN) architecture that uses gated mechan…

Machine Translation Machine translation (MT) is the automated conversion of text from one natural language to another — a task th…

Machine Unlearning Machine unlearning aims to remove the influence of specific training data from a deployed model (for privacy,…

MAE - Mean Absolute Error Mean Absolute Error (MAE) is a regression evaluation metric that measures the average absolute difference bet…

Majority Voting Majority voting aggregates multiple model outputs by choosing the most frequent discrete answer (or label) am…

Mamba Mamba is a selective state space model architecture that makes SSM parameters input-dependent, enabling stron…

mAP - mean Average Precision mAP (mean Average Precision) averages precision–recall performance across classes and detection thresholds — …

Mask R-CNN Mask R-CNN is a two-stage instance segmentation model that extends Faster R-CNN with a parallel mask predicti…

MCP Server An MCP server (Model Context Protocol server) is a lightweight service that implements Anthropic's open Model…

MDP - Markov Decision Process A Markov Decision Process is a mathematical framework for sequential decision-making under uncertainty — defi…

Mechanistic Interpretability Mechanistic interpretability reverse-engineers neural networks into human-understandable algorithms — circuit…

Membership Inference Attack A membership inference attack is a privacy attack on a trained ML model where an adversary determines whether…

Mid-Training Mid-training is a staged training phase between initial pretraining and late alignment/SFT where models see c…

MinHash / LSH MinHash estimates Jaccard similarity between sets via hash signatures; LSH (locality-sensitive hashing) bucke…

Mini-Batch A mini-batch is a small, random subset of the training dataset used for one gradient update — a compromise be…

MIOpen MIOpen is AMD's open-source deep learning primitives library — the AMD equivalent of NVIDIA's cuDNN — providi…

Mixed Precision Training Mixed precision training is a technique that uses lower-precision floating-point formats (FP16 or BF16) for m…

Mixture of Depths Mixture of Depths (MoD) routes tokens to use more or fewer layers/compute dynamically, so easy tokens take ch…

Mixup Mixup is a data-augmentation regulariser that trains on convex combinations of pairs of examples and their la…

MLA - Multi-Head Latent Attention MLA (Multi-Head Latent Attention) is DeepSeek’s attention variant that compresses keys and values into a low-…

ML - Machine Learning Machine learning is a subfield of AI in which systems learn from data — automatically improving their perform…

MLOps MLOps is the set of practices and tools that take ML models from experiment to reliable production — data/ver…

MLP - Multilayer Perceptron A multilayer perceptron (MLP) is a fully connected feedforward neural network with at least one hidden layer …

MMLU - Massive Multitask Language Understanding MMLU (Massive Multitask Language Understanding) is a benchmark that tests language model knowledge across 57 …

MobileNet MobileNet is a family of lightweight CNN architectures from Google designed for mobile and edge deployment — …

Model A model in machine learning is a mathematical function with learned parameters that maps inputs to outputs — …

Model-Based RL Model-based reinforcement learning learns or uses a model of environment dynamics to plan or train policies w…

Model Collapse Model collapse is a failure mode in which AI models trained on AI-generated data progressively lose diversity…

Model Merging Model merging combines multiple trained checkpoints into one set of weights (via averaging, task arithmetic, …

Model Parallelism Model parallelism is a distributed training strategy that partitions a model's layers or parameters across mu…

Model Registry A model registry is a central catalogue of trained model versions with metadata, lineage, stage labels (stagi…

Model Serving Model serving is the production practice of hosting models behind APIs with batching, scaling, routing, obser…

Model Soup Model soup averages the weights of multiple fine-tuned models (often from different hyperparameters) that sha…

Model Stealing Model stealing (extraction) attacks query a victim model to train a substitute that imitates its predictions …

MoE - Mixture of Experts Mixture of Experts is a neural network architecture where a model contains multiple specialised sub-networks …

Momentum Momentum is an optimisation technique that accelerates gradient descent by accumulating a velocity vector in …

MQA - Multi-Query Attention MQA (Multi-Query Attention) uses many query heads but only a single shared key head and value head. Proposed…

MSE - Mean Squared Error Mean Squared Error (MSE) is a regression loss function and evaluation metric that measures the average square…

Multi-Agent System A multi-agent system is an architecture where multiple AI agents — each with its own context, tools, and goal…

Multi-Armed Bandit A multi-armed bandit is a sequential decision problem: choose among actions (“arms”) with unknown reward dist…

Multi-Head Attention Multi-head attention is the attention mechanism used in transformers that runs multiple parallel attention op…

Multi-Hop Reasoning Multi-hop reasoning requires combining multiple pieces of evidence or intermediate inferences to reach an ans…

Multimodal Multimodal refers to AI systems that process and reason over multiple types of data — such as text, images, a…

Multi-Turn Conversation Multi-turn conversation is dialogue spanning multiple user–assistant exchanges where each turn conditions on …

NAS - Neural Architecture Search Neural Architecture Search is an automated ML technique that searches the space of possible neural network ar…

NeRF - Neural Radiance Fields NeRF represents a 3D scene as a neural network mapping 3D coordinates + view direction to colour and density,…

NER - Named Entity Recognition Named Entity Recognition is an NLP task that identifies and classifies proper nouns in text into predefined c…

Neural Network A neural network is a computational model loosely inspired by biological neurons — organised as layers of int…

Neurosymbolic AI Neurosymbolic AI is a research approach that combines neural networks (learning from data, pattern recognitio…

Next-Token Prediction Next-token prediction is the self-supervised objective of predicting the probability distribution over the vo…

NMS - Non-Maximum Suppression Non-Maximum Suppression is a post-processing algorithm used in object detection to eliminate duplicate boundi…

N:M Sparsity N:M sparsity is a structured pattern where every group of M weights keeps at most N non-zeros (e.g. 2:4), mat…

NPU - Neural Processing Unit An NPU (Neural Processing Unit) is a dedicated hardware accelerator designed specifically for neural network …

Nucleus Sampling Nucleus sampling (top-p sampling) is a text generation strategy that at each step samples the next token from…

NVIDIA NVIDIA is an American semiconductor company that designs GPUs and AI computing platforms — the dominant suppl…

Object Detection Object detection is a computer vision task that simultaneously identifies what objects are present in an imag…

Observability (LLM) Observability for AI systems means rich traces, metrics, and logs that explain what happened inside multi-ste…

OCR / Document AI OCR (optical character recognition) extracts text from images of documents; Document AI extends this to layou…

Offline Reinforcement Learning Offline RL (batch RL) learns policies from a fixed dataset of logged experience without further environment i…

Ollama Ollama is a developer-friendly tool to download, run, and serve open LLMs locally with simple CLI/API UX, bui…

On-Device AI On-device AI runs models locally on phones, laptops, or embedded hardware rather than sending data to cloud G…

One-Shot Learning One-shot learning is a machine learning approach where a model learns to recognise a new class or perform a n…

ONNX - Open Neural Network Exchange ONNX (Open Neural Network Exchange) is an open-source format for representing machine learning models as a co…

OOD Detection - Out-of-Distribution Detection Out-of-distribution (OOD) detection is the capability of a model to identify when an input belongs to a diffe…

OpenCL - Open Computing Language OpenCL stands for Open Computing Language. It is an open, vendor-neutral framework for writing programs that…

Open Weights vs Open Source Open weights means model parameters are downloadable for use; open source further implies usable source, trai…

Optical Flow Optical flow is a computer vision technique that estimates the apparent motion of pixels or regions between c…

Optimizer An optimizer is the algorithm that updates model weights during training — computing how to adjust each param…

ORPO - Odds Ratio Preference Optimization ORPO (Odds Ratio Preference Optimization) is a fine-tuning method that combines supervised instruction tuning…

Outcome Reward Model An outcome reward model (ORM) scores only the final result of a trajectory (answer correct/incorrect, prefere…

Overfitting Overfitting is when a model learns the training data too well — including its noise and irrelevant patterns —…

Paged Attention PagedAttention is a memory management algorithm for LLM inference, developed for vLLM, that manages the KV ca…

Parameter A parameter in machine learning is a learnable numerical value within a model — primarily weights and biases …

Pass@k Pass@k is an evaluation metric for code (and similar) generation: the probability that at least one of k samp…

PCA - Principal Component Analysis PCA is a linear dimensionality-reduction method that finds orthogonal directions (principal components) of ma…

PEFT - Parameter-Efficient Fine-Tuning PEFT (Parameter-Efficient Fine-Tuning) refers to a family of techniques that adapt a large pre-trained model …

Perceptron The perceptron is the simplest neural network unit — a single artificial neuron that computes a weighted sum …

Perplexity Perplexity is an intrinsic language model evaluation metric that measures how well a model predicts a held-ou…

Pipeline Parallelism Pipeline parallelism is a distributed training strategy that partitions a model's layers across multiple GPUs…

Planning (LLM) Planning in LLM agents is the process of decomposing a goal into ordered steps, subgoals, or tool sequences b…

Policy Gradient Policy gradient is a class of reinforcement learning algorithms that directly optimise the policy — the funct…

POMDP - Partially Observable Markov Decision Process A POMDP is a decision process where the agent cannot see the full true state — only observations — and must a…

Pose Estimation Pose estimation is a computer vision task that detects and localises the key points (joints) of a human body …

Positional Encoding Positional encoding is a technique that injects information about token position into transformer input embed…

Post-Training Quantization - PTQ Post-training quantization (PTQ) reduces numerical precision of a trained model (e.g. FP16→INT8/INT4) without…

PPO - Proximal Policy Optimization PPO (Proximal Policy Optimization) is a reinforcement learning algorithm that optimises a policy by taking gr…

Precision Precision is a classification evaluation metric that measures the fraction of positive predictions that are a…

Predictors Predictors are the input variables used by a model to predict an outcome — synonymous with features or indepe…

Preference Data Preference data records which among candidate responses is better (pairwise or ranked), used to train reward …

Prefill vs Decode Prefill processes the full input prompt in parallel to build KV cache; decode generates new tokens one (or fe…

Prefix Caching Prefix caching stores the KV cache (or equivalent state) for a token prefix so later requests sharing that pr…

Prefix Tuning Prefix tuning learns continuous task-specific vectors prepended to keys/values (or embeddings) while freezing…

Pre-Norm vs Post-Norm Pre-norm applies normalisation before attention/MLP; post-norm applies it after the residual add — two placem…

PRM - Process Reward Model A process reward model (PRM) scores intermediate reasoning steps, not only the final answer — providing dense…

Process Supervision Process supervision labels or rewards intermediate reasoning steps as correct or incorrect, rather than only …

Prompt Caching Prompt caching reuses computation (and often offers cheaper pricing) for long shared prefixes across requests…

Prompt Engineering Prompt engineering is the practice of designing and refining the text inputs (prompts) given to large languag…

Prompt Injection Prompt injection is an attack on LLM-based applications where malicious text in user input or external data o…

Prompt Tuning Prompt tuning learns a small set of continuous embeddings (soft prompts) prepended to inputs, keeping the ent…

Pruning Pruning is a model compression technique that removes parameters (weights, neurons, attention heads, or entir…

PyTorch PyTorch is an open-source deep learning framework developed by Meta AI that provides a dynamic computation gr…

Q-Learning Q-learning is a model-free reinforcement learning algorithm that learns the value of taking an action in a gi…

QLoRA - Quantized Low-Rank Adaptation QLoRA is a fine-tuning method that combines 4-bit quantisation of the base model with LoRA adapters — enablin…

Quantization Quantization is a model compression technique that represents model weights and/or activations in lower-preci…

Quantization-Aware Training Quantization-aware training (QAT) is a training technique that simulates low-precision quantization during th…

Quantum Machine Learning Quantum machine learning (QML) is an interdisciplinary field exploring the intersection of quantum computing …

Question Answering Question answering (QA) is an NLP task where a model produces a direct answer to a natural language question …

RAG - Retrieval-Augmented Generation RAG (Retrieval-Augmented Generation) is an architecture that enhances LLM responses by first retrieving relev…

Random Forest A random forest is an ensemble learning algorithm that trains many decision trees on random subsets of the da…

ReAct ReAct (Reasoning + Acting) is a prompting and agent framework that interleaves explicit reasoning traces ("Th…

Reasoning Model A reasoning model is an LLM specialised or trained to allocate extended internal chain-of-thought (and often …

Recall Recall (also called sensitivity or true positive rate) is a classification evaluation metric that measures th…

Red Teaming Red teaming in AI is the systematic adversarial testing of AI models by a dedicated team (or automated system…

Reflection / Self-Reflection Reflection (self-reflection) is a pattern where an agent critiques its own draft, plan, or tool outcome and r…

Regularization Regularization is any technique that reduces overfitting by constraining model complexity — adding a penalty …

Rejection Sampling (Alignment) Rejection sampling in alignment generates multiple candidates and keeps only those that pass a reward thresho…

ReLU - Rectified Linear Unit ReLU (Rectified Linear Unit) is an activation function defined as f(x) = max(0, x) — outputting the input if …

Reranker / Cross-Encoder A reranker (often a cross-encoder) re-scores a shortlist of retrieved candidates by jointly encoding query an…

Residual Connection A residual connection (skip connection) is a shortcut path in a neural network that adds the input of a layer…

ResNet - Residual Network ResNet (Residual Network) is a CNN architecture introduced by He et al. (Microsoft Research, 2015) that uses …

Responsible AI Responsible AI is the practice of designing, deploying, and governing AI systems to be safe, fair, privacy-pr…

Reward Hacking Reward hacking occurs when an agent exploits flaws in a reward function to score highly without achieving the…

Reward Model A reward model is a neural network trained on human preference data to predict a scalar quality score for any…

RLAIF - Reinforcement Learning from AI Feedback RLAIF trains or aligns models using preference labels or rewards generated by AI systems rather than (only) h…

RLHF - Reinforcement Learning from Human Feedback RLHF (Reinforcement Learning from Human Feedback) is a training technique that uses human preference judgemen…

RL - Reinforcement Learning Reinforcement learning is a machine learning paradigm where an agent learns to make decisions by interacting …

RMSNorm - Root Mean Square Layer Normalization RMSNorm normalises activations by their root-mean-square, without mean-centering: y = x / RMS(x) · g, with a …

RMSprop RMSprop (Root Mean Square Propagation) is an adaptive learning rate optimiser that divides each parameter's g…

RNN - Recurrent Neural Network A recurrent neural network is a neural network architecture designed for sequential data — processing inputs …

Robustness Robustness in AI refers to a model's ability to maintain reliable performance under distribution shifts, adve…

ROC Curve An ROC curve (Receiver Operating Characteristic curve) is a graph that plots the true positive rate (recall) …

ROCm - Radeon Open Compute ROCm (Radeon Open Compute) is AMD's open-source GPU compute platform — a software stack providing compilers, …

RoPE - Rotary Position Embedding RoPE (Rotary Position Embedding) encodes positions by rotating query and key vectors in 2D planes as a functi…

ROUGE Score ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a set of automatic metrics for evaluating text s…

RWKV RWKV is a family of RNN-style language models that achieve transformer-like performance with linear attention…

S4 - Structured State Space Sequence Model S4 is a structured state space sequence layer that made deep SSMs practical by parameterising stable long-ran…

SAC - Soft Actor-Critic SAC (Soft Actor-Critic) is an off-policy actor-critic RL algorithm that maximises expected return plus policy…

Saddle Point A saddle point is a critical point (zero gradient) in the loss landscape of a neural network where the loss i…

Scalable Oversight Scalable oversight is an AI safety research agenda aimed at developing methods to supervise AI systems that a…

Scaling Laws Scaling laws are empirical relationships showing that the performance of language models improves predictably…

Scikit-learn Scikit-learn is the standard Python library for traditional machine learning — providing a unified, consisten…

Search at Inference Search at inference uses deliberate exploration algorithms (beam search, MCTS, tree-of-thoughts expansion, to…

Self-Attention Self-attention is a mechanism where each token in a sequence attends to all other tokens in the same sequence…

Self-Consistency Self-consistency samples multiple independent reasoning paths for the same question and selects the answer th…

Self-Supervised Learning Self-supervised learning is a training paradigm where labels are automatically generated from the input data …

Semantic Search Semantic search retrieves documents by meaning using embeddings and similarity, rather than only exact keywor…

Semantic Segmentation Semantic segmentation is a computer vision task that assigns a class label to every pixel in an image — paint…

Semi-Supervised Learning Semi-supervised learning is a machine learning paradigm that leverages a small labelled dataset together with…

Sentiment Analysis Sentiment analysis is an NLP task that automatically identifies and extracts subjective information from text…

SFT - Supervised Fine-Tuning SFT (Supervised Fine-Tuning) is the training phase that converts a raw pre-trained language model into an ins…

SGD - Stochastic Gradient Descent Stochastic Gradient Descent (SGD) is the foundational optimisation algorithm for neural networks — updating m…

SHAP / LIME SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) are post-hoc …

Sigmoid Sigmoid is an activation function σ(x) = 1/(1+e^(−x)) that maps any real number to the open interval (0, 1). …

Sim-to-Real Sim-to-real is the transfer of policies or models trained in simulation to physical robots or real environmen…

Singularity The technological singularity (in AI) is the hypothesised future point at which artificial intelligence becom…

Sliding Window Attention Sliding window attention restricts each token to attend only to a fixed local neighbourhood of tokens, often …

SLM - Small Language Model A small language model (SLM) is a compact LLM (often 1–10B parameters, sometimes smaller) designed for effici…

Softmax Softmax is a function that converts a vector of raw scores (logits) into a probability distribution — each ou…

Sparse Attention Sparse attention computes only a subset of token-pair interactions (local, strided, random, or learned patter…

Sparse Autoencoder (Interpretability) Sparse autoencoders (SAEs) for interpretability train overcomplete sparse reconstructions of activations so i…

Sparse MoE - Sparse Mixture of Experts Sparse Mixture of Experts is the standard MoE architecture used in LLMs where only a small number k of N expe…

Specification Gaming Specification gaming is behaviour that satisfies the literal specification of an objective while violating th…

Speculative Decoding Speculative decoding is an LLM inference optimisation that uses a small, fast "draft" model to speculatively …

Speech-to-Text Speech-to-text (STT), also called automatic speech recognition (ASR), is the technology that converts spoken …

SSD - Single Shot Detector SSD (Single Shot MultiBox Detector) is a one-stage object detection architecture that predicts bounding boxes…

Stable Diffusion Stable Diffusion is an open-source latent diffusion model (LDM) developed by Stability AI (2022) that generat…

Stacking Stacking (stacked generalisation) is an ensemble method that trains a meta-learner (blender) on the predictio…

State Space Model (SSM) State space models (SSMs) map input sequences through latent states with structured linear dynamics, enabling…

Streaming Response Streaming response delivers tokens (or events) to the client as they are generated instead of waiting for the…

Structured Outputs / JSON Mode Structured outputs (including JSON mode) constrain model completions to a schema — valid JSON, enums, or gram…

Structured Sparsity Structured sparsity zeroes parameters in hardware-friendly patterns (blocks, channels, N:M) so accelerators c…

Summarization Text summarization is an NLP task that condenses a longer document into a shorter version preserving the key …

Superposition (Neural) Superposition is the hypothesis that neural networks represent more features than they have neurons by packin…

Super-Resolution Super-resolution (SR) reconstructs a higher-resolution image from a lower-resolution input, using classical p…

Supervised Learning Supervised learning is the dominant machine learning paradigm where a model is trained on labelled examples —…

SVM - Support Vector Machine A Support Vector Machine is a supervised learning algorithm that finds the hyperplane in a high-dimensional f…

Swarm Intelligence Swarm intelligence is a field of AI inspired by the collective behaviour of social insects (ants, bees, birds…

SWE-bench SWE-bench evaluates models on real GitHub issues: given a codebase and bug report, produce a patch that passe…

SwiGLU - Swish-Gated Linear Unit SwiGLU is a gated feed-forward activation: SwiGLU(x) = Swish(xW) ⊗ (xV), combining a SiLU/Swish gate with a l…

Sycophancy Sycophancy in AI systems is the tendency to agree with a user’s stated beliefs or preferences even when they …

Synthetic Data Synthetic data is artificially generated data that mimics the statistical properties of real data — produced …

System Prompt A system prompt is the highest-priority instruction block that sets an AI assistant’s role, policies, tools, …

Temperature Temperature (in LLM sampling) is a scalar parameter that controls the randomness of token selection during te…

Temporal Difference Learning Temporal Difference (TD) learning is a class of reinforcement learning methods that update value estimates ba…

Tensor Core Tensor Cores are specialised processing units in NVIDIA GPUs (Volta and later) designed exclusively for matri…

TensorFlow TensorFlow is Google's open-source machine learning framework — initially built around static computation gra…

Tensor Parallelism Tensor parallelism is a distributed training and inference strategy that splits individual weight matrices (t…

TensorRT TensorRT is NVIDIA's SDK for high-performance deep learning inference — it compiles trained models into optim…

TensorRT-LLM TensorRT-LLM is NVIDIA’s high-performance library for optimised LLM inference on NVIDIA GPUs, with kernels fo…

Test Set A test set is a held-out portion of the data that is never shown to the model during training or hyperparamet…

Test-Time Compute Test-time compute is extra inference-time work (longer chains of thought, search, sampling many candidates, v…

Text-to-Image Text-to-image is an AI capability that generates photorealistic or artistic images from natural language text…

Text-to-Video Text-to-video is an AI capability that generates coherent video clips from natural language text descriptions…

TF-IDF - Term Frequency–Inverse Document Frequency TF-IDF weights terms by how often they appear in a document (TF) discounted by how common they are across the…

TGI - Text Generation Inference Text Generation Inference (TGI) is Hugging Face’s production-oriented server for serving large language model…

Time Series Forecasting Time series forecasting is the ML task of predicting future values of a sequentially ordered, time-indexed da…

Token A token is the basic unit of text that a language model processes — a fragment of text produced by a tokenize…

Tokenization Tokenization is the process of splitting raw text into a sequence of tokens — the discrete units that a langu…

Tokens Per Second Tokens per second (TPS / tok/s) measures generation throughput — how many tokens the system emits per unit ti…

Tool Use / Function Calling Tool use (also called function calling) is an LLM capability where the model generates structured requests to…

Top-p / Top-k Sampling Top-k sampling limits token selection to the k most probable candidates at each step; top-p (nucleus) samplin…

TPU - Tensor Processing Unit A TPU (Tensor Processing Unit) is Google's custom ASIC designed exclusively for machine learning matrix opera…

Training Training is the process of optimising a model's parameters by repeatedly presenting examples from the trainin…

Training a Model Training a model is the process of adjusting a model's internal parameters so that its predictions match the …

Training Set A training set is the portion of labelled data on which a model learns — the examples the model sees during t…

Transfer Learning Transfer learning is the practice of taking a model pre-trained on a large dataset for a general task and ada…

Transformer Transformer is a neural network architecture that processes sequences by learning which parts of the input to…

Tree of Thoughts Tree of Thoughts (ToT) is a prompting and inference framework that extends chain-of-thought by allowing LLMs …

TruthfulQA TruthfulQA measures whether models avoid common human falsehoods and misconceptions when answering questions …

t-SNE t-SNE is a nonlinear dimensionality-reduction technique that embeds high-dimensional points into 2D/3D by pre…

TTFT - Time to First Token TTFT (Time to First Token) is the latency from request arrival until the first output token is produced — dom…

UMAP - Uniform Manifold Approximation and Projection UMAP is a nonlinear dimensionality-reduction algorithm that builds a fuzzy graph of local structure and optim…

Underfitting Underfitting occurs when a model is too simple to capture the underlying patterns in the training data — prod…

U-Net U-Net is a convolutional neural network architecture with a symmetric encoder-decoder structure connected by …

Unsupervised Learning Unsupervised learning is a machine learning paradigm where models learn patterns, structure, or representatio…

VAE - Variational Autoencoder A VAE (Variational Autoencoder) is a generative model that encodes inputs into a probability distribution ove…

Validation Set A validation set is a held-out subset of labelled data — separate from the training set — used to evaluate mo…

Value Function A value function in reinforcement learning estimates the expected cumulative discounted reward an agent will …

Vanishing Gradient The vanishing gradient problem occurs during backpropagation when gradients shrink exponentially as they flow…

Variables Variables are named containers for data values in a model or programme — in statistics and ML, they refer spe…

Vector Database A vector database stores embedding vectors and supports efficient similarity search (ANN) so applications can…

Verifier Model A verifier model is a model (or program) trained or designed to check whether a candidate answer or reasoning…

ViT - Vision Transformer ViT (Vision Transformer) is an image classification architecture that splits an image into fixed-size patches…

vLLM vLLM is an open-source LLM inference engine optimised for high-throughput and memory-efficient serving — achi…

VLM - Vision-Language Model A vision-language model (VLM) jointly processes images (and sometimes video) with text — enabling captioning,…

Vocabulary A vocabulary (or tokenizer vocabulary) is the fixed set of tokens a language model can process — the complete…

Voice Cloning Voice cloning is a text-to-speech technique that synthesises speech in a specific person's voice from a short…

VQA - Visual Question Answering Visual question answering (VQA) is the task of answering natural-language questions about an image’s content.…

Wafer A wafer is a thin, circular disc of ultra-pure silicon (typically 300 mm in diameter) on which hundreds of id…

Warmup Learning-rate warmup starts training with a small learning rate and ramps it up (usually linearly) over the f…

Watermarking AI watermarking is the embedding of imperceptible signals into AI-generated content — text, images, audio, or…

Weak Supervision Weak supervision builds training labels from noisy programmatic sources (heuristics, distant supervision, oth…

Weight A weight in a neural network is a learnable scalar parameter that multiplies an input signal — the numerical …

Weight Decay Weight decay is a regularisation technique that adds a penalty proportional to the squared magnitude of weigh…

Weight Tying Weight tying shares the input token embedding matrix with the output language-model head (or a transpose), re…

WER - Word Error Rate Word error rate (WER) is the edit-distance error rate between recognised speech transcripts and reference tex…

Word2Vec Word2Vec learns dense vector representations of words from local context (CBOW or skip-gram) so algebraic rel…

Xavier Initialization Xavier initialisation (also called Glorot initialisation) sets neural network weights by sampling from a dist…

XGBoost XGBoost is a highly optimised gradient-boosting library for decision tree ensembles, famous for dominating ta…

xLSTM xLSTM modernises LSTM architectures with exponential gating and revised memory structures to compete with tra…

YaRN - Yet another RoPE extensioN YaRN is a RoPE scaling method that extends a model’s context window beyond its original training length by co…

YOLO - You Only Look Once YOLO (You Only Look Once) is a real-time object detection architecture that frames detection as a single regr…

Zero-Shot Learning Zero-shot learning is the ability of a model to perform a task or recognise a class it has never been explici…