← AI Terminology

DQN - Deep Q-Network

DQN stands for Deep Q-Network: the first algorithm to combine Q-learning with a deep neural network function approximator, enabling reinforcement learning agents to play Atari games at superhuman level from raw pixel input.

Introduced by DeepMind in 2013–2015, it launched the deep reinforcement learning era.
Why It Matters in AI
DQN proved that a single algorithm could learn to play 49 different Atari games from raw pixels using the same architecture and hyperparameters — a generalisation that classical RL with hand-crafted features could not match. It introduced two key stabilisation techniques (experience replay and target networks) that made deep RL tractable. DQN's success triggered the wave of RL research that produced AlphaGo, OpenAI Five, and AlphaStar.
Key Points
Aspect Description
Input 4 stacked greyscale frames (84×84) → CNN → Q-values for each discrete action
ε-greedy Explore with probability ε (random action), exploit with 1-ε (best Q-value action)
Key variants Double DQN, Dueling DQN, Prioritised Replay, Rainbow (combines all improvements)
Target network A separate, periodically-updated copy of the Q-network used to compute TD targets — stabilises training
Experience replay Stores (s, a, r, s') tuples in a replay buffer; samples random mini-batches — breaks correlation
Q-function approx Neural network approximates Q(s,a) — predicts expected future reward for each action in state s
Simple Analogy
DQN learns to play video games the way a human child might: by watching the screen (raw pixels), taking actions, receiving rewards (game score), and remembering past experiences. Experience replay is like keeping a diary of past games to study during quiet moments — rather than only learning from what just happened.
Common Usage Examples
  • stable_baselines3.DQN — standard DQN implementation with all improvements
  • OpenAI Gym Atari: env = gym.make('Breakout-v4') — classic DQN training environment
  • Double DQN: uses online network to select action, target network to evaluate — reduces overestimation
  • Prioritised Experience Replay: samples transitions with high TD error more frequently
  • Rainbow DQN: combines 6 improvements — state-of-the-art on Atari across 57 games
Summary
In short: DQN fused deep learning with Q-learning — the algorithm that first showed a single neural network could master dozens of different games from raw pixels, launching the deep RL era.