def discount_bootstrap_gae(rewards_buffer, dones_buffer, values_buffer, final_values, discount_factor, gae_lambda,<EOL>number_of_steps):
advantage_buffer = torch.zeros_like(rewards_buffer)<EOL>sum_accumulator = <NUM_LIT:0><EOL>for i in reversed(range(number_of_steps)):<EOL><INDENT>if i == number_of_steps - <NUM_LIT:1>:<EOL><INDENT>next_value = final_values<EOL><DEDENT>else:<EOL><INDENT>next_value = values_buffer[i + <NUM_LIT:1>]<EOL><DEDENT>bellman_delta = (<EOL>rewards_buffer[i] + discount_factor * next_value * (<NUM_LIT:1.0> - dones_buffer[i]) - values_buffer[i]<EOL>)<EOL>advantage_buffer[i] = sum_accumulator = (<EOL>bellman_delta + discount_factor * gae_lambda * sum_accumulator * (<NUM_LIT:1.0> - dones_buffer[i])<EOL>)<EOL><DEDENT>return advantage_buffer<EOL>
Calculate state values bootstrapping off the following state values - Generalized Advantage Estimation
https://arxiv.org/abs/1506.02438