Difference between revisions of "Allen's REINFORCE notes"
(→Motivation) |
|||
| Line 10: | Line 10: | ||
Recall that the objective of Reinforcement Learning is to find an optimal policy <math> \pi^* </math> which we encode in a neural network with parameters <math>\theta^*</math>. These optimal parameters are defined as | Recall that the objective of Reinforcement Learning is to find an optimal policy <math> \pi^* </math> which we encode in a neural network with parameters <math>\theta^*</math>. These optimal parameters are defined as | ||
| − | <math>\theta^* = \text{argmax}_\theta E_{\tau \sim p_\theta(\tau)} \left[ \sum_t r(s_t, a_t) \right] </math> | + | <math>\theta^* = \text{argmax}_\theta E_{\tau \sim p_\theta(\tau)} \left[ \sum_t r(s_t, a_t) \right] </math>. Let's unpack what this means. To phrase it in english, this is basically saying that the optimal policy is one such that the expected value of the total reward over following a trajectory determined by the policy is the highest over all policies. |
=== Learning === | === Learning === | ||
Revision as of 21:46, 24 May 2024
Allen's REINFORCE notes
Links
Motivation
Recall that the objective of Reinforcement Learning is to find an optimal policy which we encode in a neural network with parameters . These optimal parameters are defined as . Let's unpack what this means. To phrase it in english, this is basically saying that the optimal policy is one such that the expected value of the total reward over following a trajectory determined by the policy is the highest over all policies.
![{\displaystyle \theta ^{*}={\text{argmax}}_{\theta }E_{\tau \sim p_{\theta }(\tau )}\left[\sum _{t}r(s_{t},a_{t})\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/74bb8656d55a9f9e2fc28ad99b15be9a11e8b3bb)
Learning
Learning involves the agent taking actions and the environment returning a new state and reward.
- Input: : States at each time step
- Output: : Actions at each time step
- Data:
- Learn to maximize
