WSU Lexicon: Quantitative Methods
Reinforcement Learning
Reinforcement learning is a branch of machine learning in which an agent learns a decision-making policy through trial and error. It takes actions in an environment, receives rewards or penalties, and aims to maximize cumulative reward over time rather than to fit labeled data.
It sits alongside supervised and unsupervised learning as one of the three core machine learning approaches.
The entry
The agent, environment and reward loop
Reinforcement learning is built around a repeated cycle. An agent observes the current state of its environment, chooses an action, and the environment responds with a new state and a numerical reward. The rule the agent uses to map states to actions is called its policy, and learning consists of improving that policy so that the total reward collected across many steps is as large as possible. Future rewards are usually discounted, so the objective is the sum of each future reward multiplied by a discount factor raised to the number of periods ahead.
Two features make the problem distinctive. Rewards may arrive long after the action that earned them, so the agent must assign credit across time. The agent also faces a tradeoff between exploitation, repeating actions known to pay off, and exploration, trying new actions that might pay more.
The entry
How it differs from supervised and unsupervised learning
Supervised learning trains on inputs paired with known correct outputs, such as past loans labeled as defaulted or repaid, and learns to predict the label for new cases. Unsupervised learning receives inputs with no labels and searches for structure, such as grouping stocks into clusters by return behavior. Reinforcement learning differs from both: there is no fixed dataset of answers, and the data the agent sees depends on the actions it chooses.
Because learning comes from interaction, reinforcement learning typically needs an enormous volume of experience. That experience is often produced in a simulator rather than gathered from the real world. When the policy or value estimates are represented by neural networks, the approach is called deep reinforcement learning, which underlies most prominent recent successes.
The entry
Applications in finance and generative AI
In investment management, reinforcement learning has been applied to problems framed as sequential decisions: splitting a large order over time to reduce market impact, adjusting hedges as conditions change, and rebalancing portfolios. Training usually takes place in simulated markets, sometimes built with Monte Carlo methods, before any live deployment.
The technique also shapes generative AI. Large language models are commonly refined with reinforcement learning from human feedback, in which people rank candidate responses, a reward model is trained on those rankings, and the language model is tuned to produce higher-scoring answers. In markets, important limitations remain. Financial data are noisy and nonstationary, a policy tuned to a simulator may fail when real conditions differ, and a poorly specified reward can lead the agent to optimize the wrong objective.
The entry
What a Level I candidate needs to be able to do
At Level I, the emphasis is classification and conceptual understanding, not implementation. A candidate should be able to read a short description of a modeling task and decide whether it is supervised, unsupervised or reinforcement learning. The giveaway for reinforcement learning is an agent taking actions, observing consequences and being rewarded, with no labeled target variable.
Candidates should also know the vocabulary of agent, environment, state, action, reward and policy, and the exploration versus exploitation tradeoff. They should recognize the practical cautions: heavy data and computing requirements, dependence on realistic simulation, and the risk of overfitting in markets where relationships shift. Finally, they should understand its role in fine-tuning large language models through human feedback.
Common trap
“Reinforcement learning is just supervised learning, because the rewards act as the labels the algorithm trains on.”
Supervised learning is told the correct output for each input. A reinforcement learner is never told the right action. It receives only a reward signal, often delayed, and must explore to discover which sequences of actions earn the most over time.
In practice
DeepMind's AlphaGo and AlphaGo Zero learn the game of Go
In March 2016, DeepMind's AlphaGo beat world champion Lee Sedol four games to one. Its successor, AlphaGo Zero (2017), learned purely by playing millions of games against itself, using wins and losses as the reward, and surpassed the original without human game data.
Candidates ask
Questions on this term
- How does reinforcement learning differ from supervised learning?Supervised learning fits a model to labeled examples with known correct outputs. Reinforcement learning has no labels; an agent acts, receives rewards, and learns through trial and error which actions maximize cumulative reward.
- Is reinforcement learning actually used in investment management?Yes, mainly for sequential decision problems such as optimal trade execution, dynamic hedging and portfolio rebalancing, usually trained in simulated market environments. Adoption is more cautious than in games because market data are noisy and regimes change. A policy that performs well in simulation can disappoint live, so firms pair it with risk limits and human oversight.
- What does reinforcement learning have to do with large language models?Many large language models are refined with reinforcement learning from human feedback. Human raters rank alternative responses, those rankings train a reward model, and the language model is then tuned to generate responses the reward model scores highly. This step helps align outputs with what users find helpful and appropriate, but it can also introduce the raters' own preferences and blind spots.
See also
Related entries
- Monte Carlo SimulationMonte Carlo simulation is a computational technique that repeatedly draws random values from assumed probability distributions for a model's uncertain inputs, generating thousands of trial outcomes.
- Robo-AdviserA robo-adviser is an automated digital platform that builds and manages a client portfolio using algorithms driven by an online questionnaire covering goals, time horizon and risk tolerance.
- Simple Linear RegressionSimple linear regression estimates a straight-line relationship between one independent variable and one dependent variable.
One entry from the WSU Lexicon every day. Browse the full archive.