53
edits
Changes
Created page with "Intuition: Want to avoid too large of a policy update #Smaller policy updates more likely to converge to optimal #Falling "off the cliff" might mean it's impossible to recover..."
Intuition: Want to avoid too large of a policy update
#Smaller policy updates more likely to converge to optimal
#Falling "off the cliff" might mean it's impossible to recover
How we solve this: Measure how much policy changes w.r.t. previous, clip ratio to <math>[1-\varepislon, 1 + \varepsilon] removing incentive to go too far.
#Smaller policy updates more likely to converge to optimal
#Falling "off the cliff" might mean it's impossible to recover
How we solve this: Measure how much policy changes w.r.t. previous, clip ratio to <math>[1-\varepislon, 1 + \varepsilon] removing incentive to go too far.