Question

How can you best describe the Bellman Equations for a Markov Reward Process (MRP)?

A) The value of a state is the reward from that state plus the sum over the product of transition probabilities for the next n states.

B) The value of a state is the sum over all actions, a, given the state, s of the policy, times the sum over the product of transition probabilities from the state to the next state, s’ and the reward from the state plus the discounted value of the next state.

C) The value of a state is the discounted sum over next the product of transition probabilities for next states.

D) The value of a state is the sum over all transition probabilities from the state, s, to the next state, s’, times the sum over the product of the policy, and the reward from the state plus the discounted value of the next state.

I am tending to A) but I am not sure. Can you help? Thx!

Answer 1

The bell man equation for value of function of a states of with policy π can be donored by VT(S) T(S) = EH {"1+ry " (std)) sta s} Emmediate des cound value rewond fonction of heat state

by removing expectation, the bellman equation turns into Tīrni VT(S) = I 11(sa) pa (RSS! +7V (8) Sum over of policy pannition all actions it probabilities to next states' Re keto rewond from the state rul (s') => discound rewand of next state so option.b is correer