Abstract
<title>Abstract</title> <p>Entropy coding is the final, lossless stage of every hybrid video codec, and its efficiency directly determines the bitrate at which a given reconstruction quality is achieved. Context-based Adaptive Binary Arithmetic Coding (CABAC), the entropy engine in H.264/AVC, H.265/HEVC, and H.266/VVC, relies on hand-designed context models (154-399 indices), single Bernoulli probability estimates updated by fixed exponential moving average rules, and binarisation that discards the joint statistical structure of multi-valued transform coefficients. These architectural constraints limit probability estimation accuracy within the CABAC framework. Meanwhile, end-to-end neural compression has demonstrated that learned probability models including hyperpriors, autoregressive context models, and Gaussian mixture likelihoods can achieve substantially tighter probability bounds, but these advances remain confined to proprietary, non-standards-compliant codec pipelines. We propose NAE-VC (Neural Arithmetic Encoding for Video Compression), a modular learned entropy coding framework that bridges this gap by replacing only the CABAC module in existing standardised codecs while preserving all other coding tools unchanged. NAE-VC comprises three components: (i) a context aggregation network that fuses spatial features (masked 3×3 convolutions with cross-block boundary augmentation), channel-wise autoregressive features (four-group sequential MLP), and temporal features (motion-compensated reference extraction) through multi-head cross-attention (H = 4, dk = 48) into a unified conditioning vector of dimension D = 192; (ii) a Gaussian Mixture Model (GMM) with K = 3 components that predicts per-coefficient multimodal probability distributions via three parallel linear heads (mean, scale with Softplus, mixing weights with Softmax), discretised over unit quantisation bins for lossless coding; and (iii) a hierarchical context modelling pipeline combining hyperprior side information (1.8 M parameters, 4× spatial reduction) with checkerboard-parallelised spatial context and rANS-based encoding with 16-bit CDF precision. NAE-VC is integrated into the reference software of three codec generations JM-19.0 (H.264/AVC), HM-16.25 (H.265/HEVC), and VTM-17.0 (H.266/VVC) requiring only 8.6 M trainable parameters per model. Comprehensive experiments on six benchmark datasets (UVG, MCL-JCV, HEVC Classes B, C, D, E) under both All-Intra (AI) and Low-Delay P (LDP) configurations demonstrate consistent BD-rate savings across all codecs: −4.82% (H.264), −3.67% (H.265), and −2.41% (H.266) under AI; increasing to −6.15%, −4.93%, and −3.28% under LDP, where the temporal context path provides additional inter-frame conditioning unavailable to CABAC. The monotonically decreasing improvement from H.264 to H.266 is directionally consistent with the hypothesised inverse relationshi between native CABAC sophistication and neural replacement headroom, although other codec-level differences may also contribute. Indirect comparisons against previously reported numbers for Deep-CABAC (−1.65 pp AI) and NN-CABAC (−1.01 pp AI) place NAE-VC in a competitive regime and the gains generalise across PSNR, MS-SSIM, and VMAF metrics (up to −5.47% VMAF BD-rate for H.264), confirming that the improvements arise from genuinely better probability modelling rather than metric-specific optimisation. Ablation analysis reveals that neural probability estimation contributes the largest individual gain (−1.18 pp), followed by the hyperprior (−0.56 pp), GMM mixture modelling (−0.64 pp), and spatial context (−0.49 pp). Cross-generational analysis shows that H.265+NAE-VC closes approximately 25% of the compression gap to native H.266, demonstrating that learned entropy coding provides a meaningful fraction of generational codec gains through modular replacement alone.</p>