论文标题
手动相互作用图像生成
Hand-Object Interaction Image Generation
论文作者
论文摘要
在这项工作中，我们致力于一项新任务，即手动相互作用图像生成，该任务旨在有条件地在给定的手，对象及其交互状态下有条件地生成手动图像。在许多潜在的应用程序场景（例如AR/VR游戏和在线购物等）中，这项任务是具有挑战性和值得研究的。为了解决此问题，我们提出了一个新颖的Hogan框架，该框架利用了表现力的模型感知的手动对象表示，并利用其固有的拓扑来构建统一的表面空间。在这个空间中，我们明确考虑相互作用过程中复杂的自我和相互阻塞。在最终图像合成过程中，我们考虑了手和物体的不同特征，并以分裂和综合的方式生成目标图像。为了进行评估，我们构建了一个综合协议，以访问生成图像的保真度和结构保存。在两个大规模数据集（即HO3DV3和DexyCB）上进行了广泛的实验，证明了我们框架的有效性和优越性，既有定量和定性。项目页面可在https://play-with-hoi-generation.github.io/上找到。
In this work, we are dedicated to a new task, i.e., hand-object interaction image generation, which aims to conditionally generate the hand-object image under the given hand, object and their interaction status. This task is challenging and research-worthy in many potential application scenarios, such as AR/VR games and online shopping, etc. To address this problem, we propose a novel HOGAN framework, which utilizes the expressive model-aware hand-object representation and leverages its inherent topology to build the unified surface space. In this space, we explicitly consider the complex self- and mutual occlusion during interaction. During final image synthesis, we consider different characteristics of hand and object and generate the target image in a split-and-combine manner. For evaluation, we build a comprehensive protocol to access both the fidelity and structure preservation of the generated image. Extensive experiments on two large-scale datasets, i.e., HO3Dv3 and DexYCB, demonstrate the effectiveness and superiority of our framework both quantitatively and qualitatively. The project page is available at https://play-with-hoi-generation.github.io/.
