Skip to yearly menu bar Skip to main content


Poster

G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis

Yufei Ye · Abhinav Gupta · Kris Kitani · Shubham Tulsiani

Arch 4A-E Poster #169
[ ] [ Paper PDF ]
Wed 19 Jun 10:30 a.m. PDT — noon PDT

Abstract:

We propose G-HOP, a denoising diffusion based generative prior for hand-object interactions that allows modeling both the 3D object and a human hand, conditioned on the object category. To learn a 3D spatial diffusion model that can capture this joint distribution, we represent the the human hand via a skeletal distance field to obtain a representation aligned with the (latent) signed distance field for the object. We show that this hand-object prior can then serve as a generic guidance to facilitate other tasks like reconstruction from interaction clip and human grasp synthesis. We believe that our model, trained by aggregating seven diverse real-world interaction datasets spanning across 157 categories, represents a first approach that allows jointly generating both hand and object. Our empirical evaluations demonstrate the benefit from this joint prior in video-based reconstruction and human grasp synthesis, outperforming current task-specific baselines.

Chat is not available.