Abstract: Widely adopted motion forecasting datasets sub-stitute the observed sensory inputs with higher-level abstractions such as 3D boxes and polylines. These sparse shapes are inferred through ...