← AI Terminology
POMDP - Partially Observable Markov Decision Process
A POMDP is a decision process where the agent cannot see the full true state — only observations — and must act under uncertainty, often maintaining a belief state.
It formalises realistic sensing limits beyond fully observed MDPs.
It formalises realistic sensing limits beyond fully observed MDPs.
Why It Matters in AI
Robots and users rarely see full state. POMDPs frame filtering, memory, and active sensing. LLM agents with partial tool feedback are informal POMDPs too.
Key Points
| Aspect | Description |
|---|---|
| Use | Dialogue, robotics perception, inventory with uncertainty |
| Belief | Distribution over states updated from observations |
| Vs MDP | Hidden state; observation function |
| Related | MDP, filtering, Kalman filters |
| Hardness | Solving exact POMDPs is very difficult |
| Practice | RNNs/transformers as implicit memory policies |
Simple Analogy
Playing a card game without seeing opponents’ hands — you track possibilities and act under fog, not perfect information.
Common Usage Examples
- Belief tracking in dialogue state
- RNN policies for partial observability
- Sensor fusion before control
- Classic POMDP solvers on tiny discrete problems
Summary
In short: POMDPs model decisions when state is only partially observed — the formal home of memory, filtering, and acting under fog.