Forward projections under a fixed plan

Forward projections can now be defined under the constraint that a particular plan is executed. The specific expression of actions is replaced by $\pi$. Each time an action is needed from a state $x \in X$, it is obtained as $\pi (x)$. In this formulation, a different $U(x)$ may be used for each $x \in X$, assuming that $\pi$ is correctly defined to use whatever actions are actually available in $U(x)$ for each $x \in X$.

First we will consider the nondeterministic case. Suppose that the initial state $x_1$ and a plan $\pi$ are known. This means that $u_1 = \pi (x_1)$, which can be substituted into (10.10) to compute the one-stage forward projection. To compute the two-stage forward projection, $u_2$ is determined from $\pi (x_2)$ for use in (10.11). A recursive formulation of the nondeterministic forward projection under a fixed plan is

$$\begin{split}X_{k+1}(x_1,\pi ) = \{ x_{k+1} \in X \;\vert\; &... ...mbox{ and } x_{k+1} = f(x_k,\pi (x_k),\theta_k)\} . \end{split}$$ (10.28)

The probabilistic forward projection in (10.10) can be adapted to use $\pi$, which results in

$$P(x_{k+2}\vert x_k,\pi ) = \sum_{x_{k+1} \in X} P(x_{k+2}\vert x_{k+1},\pi (x_{k+1})) P(x_{k+1}\vert x_k,\pi (x_k)) .$$ (10.29)

The basic idea can be applied $k-1$ times to compute $P(x_k \vert x_1, \pi )$.

A state transition matrix can be used once again to express the probabilistic forward projection. In (10.15), all columns correspond to the application of the action $u$. Let $M_\pi$, be the forward projection due to a fixed plan $\pi$. Each column of $M_\pi$ may represent a different action because each column represents a different state $x_k$. Each entry of $M_\pi$ is

$$m_{i,j} = P(x_{k+1} = i \; \vert \; x_k = j, \;\pi (x_k)) .$$ (10.30)

The resulting $M_\pi$ defines a Markov process that is induced under the application of the plan $\pi$.