论文标题
基于全面外观的3D注视估算
Learning-by-Novel-View-Synthesis for Full-Face Appearance-Based 3D Gaze Estimation
论文作者
论文摘要
尽管最近基于外观的凝视估计技术取得了进步，但涵盖目标头姿势和凝视分布的训练数据的需求仍然是实用部署的至关重要的挑战。这项工作研究了一种基于单眼3D面部重建的凝视估计训练数据的新方法。与使用多视图重建，照片现实CG模型或生成神经网络的先前作品不同，我们的方法可以操纵和扩展现有培训数据的头部姿势范围，而无需任何其他要求。我们介绍了一个投影匹配的过程，以使重建的3D面部网格与摄像机坐标系保持一致，并使用准确的凝视标签合成面部图像。我们还提出了掩盖引导的凝视估计模型和数据增强策略，以利用合成训练数据进一步提高估计准确性。使用多个公共数据集的实验表明，我们的方法可显着提高具有非重叠凝视分布的挑战跨数据库设置的估计性能。
Despite recent advances in appearance-based gaze estimation techniques, the need for training data that covers the target head pose and gaze distribution remains a crucial challenge for practical deployment. This work examines a novel approach for synthesizing gaze estimation training data based on monocular 3D face reconstruction. Unlike prior works using multi-view reconstruction, photo-realistic CG models, or generative neural networks, our approach can manipulate and extend the head pose range of existing training data without any additional requirements. We introduce a projective matching procedure to align the reconstructed 3D facial mesh with the camera coordinate system and synthesize face images with accurate gaze labels. We also propose a mask-guided gaze estimation model and data augmentation strategies to further improve the estimation accuracy by taking advantage of synthetic training data. Experiments using multiple public datasets show that our approach significantly improves the estimation performance on challenging cross-dataset settings with non-overlapping gaze distributions.
