Action conditioned prediction

Predicting future observations or states while taking proposed actions into account.

In a world model, an action can change the predicted future. Evaluation should examine whether the prediction responds appropriately to that action and preserves relevant physical and temporal relationships. Visually plausible output alone does not establish an accurate account of what the action would cause.

In practice

The predicted object motion should differ when a gripper releases it versus when the gripper keeps holding it.

Further reading

• World Models research