An overview of reinforcement learning, its principles, and the difference between supervised and unsupervised learning.
Introduction to Reinforcement Learning (RL)
Key concepts, terminology, and the basic learning loop
| Element | Symbol | Description |
|---|---|---|
| Agent | – | The decision‑making entity (e.g., robot, software bot). |
| Environment | – | Everything the agent interacts with (world, simulator, game). |
| State | (s_t) | A representation of the environment at time t. |
| Action | (a_t) | Decision taken by the agent at time t. |
| Reward | (r_{t+1}) | Scalar feedback received after action (a_t). |
| Policy | (\pi) | Mapping from states to actions (deterministic or stochastic). |
| Value Function | (V^\pi(s)) / (Q^\pi(s,a)) | Expected return starting from a state (or state‑action pair) following policy (\pi). |
| Model (optional) | – | Agent’s internal prediction of next state and reward (used in model‑based RL). |
Return (cumulative reward)
[
G_t = \sum_{k=0}^{\infty} \gamma^k r_{t+k+1}
]
Goal: Find an optimal policy (\pi^* = \arg\max_\pi \mathbb{E}[G_t \mid \pi]).
| Category | Main Idea | Typical Algorithms |
|---|---|---|
| Model‑free | Learn directly from interaction, no explicit model of the environment. | - Value‑based: Q‑learning, SARSA - Policy‑gradient: REINFORCE, Actor‑Critic |
| Model‑based | Build a model (\hat{P}(s' | s,a), \hat{R}(s,a)) and plan with it. |
| Off‑policy vs. On‑policy | Off‑policy learns about a policy different from the one used to generate data (e.g., Q‑learning). On‑policy learns about the behavior policy (e.g., SARSA). | - Off‑ |