AI TrackerFrontier desk · AEST
Back to news
normalaitech

WorldSonus: Bringing Sound to Worlds

arXiv cs.AI · 7 Oct 2026, 03:50 AEST

WorldSonus is an interactive video-to-audio framework designed for real-time spatial sound synthesis in world models. It addresses challenges such as real-time generation, interactive control, and spatially aligned stereo sound.

Key facts

Read aloud

WorldSonus is an interactive video-to-audio framework designed for real-time spatial sound synthesis in world models. It addresses challenges such as real-time generation, interactive control, and spatially aligned stereo sound. Bringing sound to world models poses three core challenges: real-time generation to keep pace with interactive video streams, interactive control to respond to mid-stream sound instructions, and spatially aligned stereo to reflect scene geometry and camera motion. For real-time generation, WorldSonus employs a streaming causal autoregressive diffusion architecture that synthesizes audio chunks at a low real-time factor RTF of zero point four one. For interactive control, we incorporate an audio-centric captioning pipeline with chunk-indexed prompt scheduling, enabling dynamic manipulation of sound events during generation. Extensive experiments demonstrate that while tailored for world models, WorldSonus generalizes effectively to open-domain video-to-audio benchmarks, matching or outperforming state-of-the-art bidirectional models in both acoustic quality and spatial alignment.

Sources in this cluster