An arXiv preprint reports that ACE-Ego-Hand, a system that repurposes a video diffusion model, had substantially lower 3D hand-pose errors than ViDiHand on ARCTIC, HOT3D and held-out HOI4D. Its wrist-aligned error was 30% lower on ARCTIC, 40% lower on HOT3D and 23% lower on HOI4D. The findings concern offline reconstruction of motion through occlusion and out-of-sight gaps. They do not establish the hand's absolute position while it is out of sight.
The biggest differences
On the paper's wrist-aligned pose measure, ACE-Ego-Hand recorded 15.26 millimetres on ARCTIC versus 21.67 for ViDiHand, 12.89 versus 21.51 on HOT3D, and 23.03 versus 30.09 on held-out HOI4D. Those gaps correspond to 30%, 40% and 23% lower error, respectively. HOI4D was held out for zero-shot evaluation, so it was not part of training.
Across ARCTIC, HOT3D and HOI4D, the model led on 27 of 29 reported metrics. Its standard configuration's FAcc, Recall and F1 values were above 0.95, with ARCTIC reaching 1.00. That broad lead means the comparison was not confined to a single pose score.
When the hand disappears
The harder case was the period when a hand was out of sight. Including those frames, the combined error measure was 16.8 millimetres on ARCTIC and 17.3 millimetres on HOT3D for ACE-Ego-Hand, against 31.05 and 44.44 millimetres for ViDiHand. That put ACE-Ego-Hand's error 45.9% lower on ARCTIC and 61.1% lower on HOT3D. But the measure is wrist-aligned, so these figures do not establish the hand's absolute placement during the visual gap.
How the system is built
The method repurposes a video diffusion model as a deterministic geometry encoder. ACE-Ego-Hand combines a Deterministic Clean-Latent Encoder, a Bidirectional Spatiotemporal Decoder and a Ray-Based Camera Solver. Its K-free configuration does not need camera intrinsics supplied at test time or an external detector.
Training combined labeled egocentric video from ARCTIC, HOT3D, H2O and OakInk2 with rendered egocentric Re:InterHand video and image-only FreiHAND and RHD data. HOI4D was held out for zero-shot evaluation.
Evaluation covered five benchmarks: ARCTIC, HOT3D, HOI4D, H2O and OakInk2. The test splits were sequence-disjoint, and each method was run on the same video segments. The authors report that the data came from public research datasets obtained through official channels; no new human-subject data were collected and no dataset was redistributed.
Fast on paper, offline by design
Under the reported single-A100 timing setup, ACE-Ego-Hand processed 63.1 frames per second in its standard configuration and 63.3 frames per second in K-free form, compared with 1.91 frames per second for ViDiHand. That is a 33-fold throughput gap. The figure is a hardware-specific runtime comparison, not evidence that the system can respond inside a live control loop, because the method uses whole-clip context and is offline by construction.
The separately trained K-free model stayed close to the standard model on HOT3D detection: F1 was 0.995 versus 0.996. Its 2D projection error was 6.07 pixels versus 6.42, while its pose error was 1.36 millimetres higher on ARCTIC and 1.09 millimetres higher on H2O. The comparison suggests that avoiding supplied test-time camera intrinsics need not materially change detection performance here, though the pose differences were not zero.
What the component tests found
An ARCTIC feature-source ablation reported a 16.95 millimetre wrist-aligned pose error and a 41.5 millimetre out-of-sight error for the full model. The raw-RGB and VAE-latent alternatives recorded 46.50 and 32.50 millimetres on the pose measure, while the no-LoRA row recorded 23.10 millimetres on pose and 50.1 millimetres on out-of-sight error.
On two additional in-domain datasets, the best-baseline comparison showed a 66% lower wrist-aligned pose error on OakInk2, from 26.520 to 8.988 millimetres, and a 43% lower error on H2O, from 16.219 to 9.223 millimetres. The K-free configuration stayed within 1.1 millimetres of its pose error on both datasets.
The boundary of the result
The practical boundary is clear. ACE-Ego-Hand depends on whole-clip context, so its latency is bounded by clip length and the method is intended for offline data curation rather than closed-loop control. Its out-of-sight scores remain wrist-aligned, leaving absolute placement during the gap unresolved. The work is an arXiv preprint, and its authors report that the code is publicly available.
Paper data and sources
Original title: DreamHand: Repurposing Video Diffusion Models for Occlusion-Robust Egocentric 3D Hand Motion Recovery
Authors: Yufei Liu, Xixi Wang, Hao Li et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text