4D-HOF: Hand-Object Flow Matching for Feed-Forward 4D Interaction Reconstruction
arXiv cs.AI · 7 Oct 2026, 03:59 AEST
The 4D-HOF framework offers a novel approach to reconstructing 4D hand-object interactions without the need for costly per-sequence optimization. It utilizes a conditional flow matching model to enhance the accuracy of interaction predictions by correcting errors in real-time.
Key facts
- Existing methods for 4D hand-object reconstruction often rely on costly per-sequence optimization, while generative approaches typically synthesize interactions from random noise, which can lead to unstable interaction prediction.
- We introduce 4D-HOF, a feed-forward framework that reconstructs 4D hand-object interactions from coarse but informative estimates produced by vision foundation models.
- A key advantage of our generative formulation is that it naturally enables test-time guidance within the transport process.
- By training the generative model on diverse datasets, 4D-HOF generalizes robustly to challenging in-the-wild scenarios.
- Experiments on out-of-domain benchmarks show that 4D-HOF achieves state-of-the-art performance, producing more stable and accurate 4D hand-object reconstructions.
Read aloud
The 4D-HOF framework offers a novel approach to reconstructing 4D hand-object interactions without the need for costly per-sequence optimization. It utilizes a conditional flow matching model to enhance the accuracy of interaction predictions by correcting errors in real-time. Existing methods for 4D hand-object reconstruction often rely on costly per-sequence optimization, while generative approaches typically synthesize interactions from random noise, which can lead to unstable interaction prediction. We introduce 4D-HOF, a feed-forward framework that reconstructs 4D hand-object interactions from coarse but informative estimates produced by vision foundation models. A key advantage of our generative formulation is that it naturally enables test-time guidance within the transport process. By training the generative model on diverse datasets, 4D-HOF generalizes robustly to challenging in-the-wild scenarios. Experiments on out-of-domain benchmarks show that 4D-HOF achieves state-of-the-art performance, producing more stable and accurate 4D hand-object reconstructions.
Sources in this cluster
- 4D-HOF: Hand-Object Flow Matching for Feed-Forward 4D Interaction Reconstruction
arXiv cs.AI · 7 Oct 2026, 03:59 AEST