GestureFM : Compact Latent Flow Matching for Low-Latency Co-Speech Body-Motion Generation

<title>Abstract</title> <p>Interactive digital humans require speech-driven body motion that can begin before a complete utterance is available, but many high-quality co-speech generation methods are designed for offline full-sequence inference. This study aims to generate SMPL-X body motion from speech with low first-chunk latency while maintaining competitive motion quality in streaming inference. We propose \ours{}, a compact latent Flow Matching (FM) framework in which a Gesture Variational AutoEncoder (Gesture VAE) with causal decoding compresses 168-dimensional body pose and global translation into 44 latent tokens with a 16-dimensional latent channel, and an audio-conditioned FM Transformer generates motion in the normalised latent space. During streaming inference, the model generates 1 s motion chunks from a 2 s available audio context with a 0.75 s stride and 0.25 s overlap. On the test set, the latent-normalised model achieves an official EMAGE Fr\'echet Gesture Distance (FGD) of 0.272, improving over the unnormalised variant (0.390) and the reproduced EMAGE body-motion checkpoint (2.373), while matching ground-truth Beat Alignment (BA\,{=}\,0.758). The ODE step count has negligible effect on FGD, so \((K\,{=}\,2)\) is recommended as the default, providing 22 ms first-chunk latency at 0.022\((\times)\) real-time factor (RTF) with no quality penalty. In streaming deployment we further observe chunk-boundary velocity discontinuities; a lightweight 7-frame local smoother reduces this boundary velocity jump by 85% (0.700\((\to)\)0.105) while leaving BA stable at 0.762. These results show that GestureFM provides a strong yet efficient latent FM baseline for low-latency body-motion generation.</p>