Q-Learning

Q-learning is a model-free, off-policy reinforcement-learning algorithm that learns the action-value function (Q(s,a)): expected discounted return for taking action (a) in state (s).

[Q(s,a)\leftarrow Q(s,a)+\alpha[r+\gamma\max_{a'}Q(s',a')-Q(s,a)]]

Here (\alpha) is learning rate, (\gamma) is discount factor, (r) is immediate reward, and (s') is the next state. An ε-greedy policy balances exploration and exploitation. With sufficient exploration and suitable learning rates, tabular Q-learning converges under standard finite Markov decision process assumptions.