Open Problems

It is more important in practice to reduce the sim-to-real performance gap than to reduce the reality gap. This motivates the question of whether accurate physics modeling is needed to perform reliable model-based control. An open question remains as to how a robot can most efficiently use a simulator with incorrect parametrization to learn either a policy or stochastic world model for use in generating robust, real-world performance.

Differentiable simulators offer the computation of gradients of quantities involved in the simulation which allows them to be integrated in gradient-based optimization workflows. An open question remains as to how to best leverage differentiable simulation and augment it with learning-based dynamics model.

Video models are primarily designed to process, generate, or predict sequences of visual frames. World models pursue a broader goal: learning internal representations of an environment that enable simulation and prediction of future states. In future iterations, the paradigms of world modeling and simulation may not be so distinct, with simulators naturally providing data to bootstrap world models for deployment in reality.

Simulation-based inference is a statistical technique to approximate the posterior distribution of simulation parameters. An interesting opportunity for future research is to use other modalities, not directly state observations, to estimate simulation parameters. For example, we could use videos from both real and simulation as the inputs to the neural posterior estimator significantly simplifying the experimental setup.