AI TrackerFrontier desk · AEST
Back to news
normalaitech

Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary Rigs

arXiv cs.AI · 21 Oct 2026, 03:59 AEST

A new model co-generates video and 3D motion for arbitrary rigs, allowing for realistic animations in 3D scenes. This approach separates 3D motion from video generation, enabling more flexible and accurate animations.

Key facts

Read aloud

A new model co-generates video and 3D motion for arbitrary rigs, allowing for realistic animations in 3D scenes. This approach separates 3D motion from video generation, enabling more flexible and accurate animations. Video generation models can now depict remarkably realistic motion. For instance, a video model can show a monkey jumping over a crossbar, but it cannot return that jump as replayable 3D motion placed in the 3D scene. We instead keep 3D motion in a separate model that supports arbitrary skeletons, and connect it to a pretrained video model through a small set of cross-attention layers. Together with human and robot video, this yields nearly 200K training pairs. Our evaluations demonstrate that our model animates diverse rigs from text more faithfully than video-first pipelines, and its motion responds to the surrounding scene.

Sources in this cluster