论文标题
Condies：可控的神经脸部化身
CoNFies: Controllable Neural Face Avatars
论文作者
论文摘要
神经辐射场（NERF）是用于建模2D图像集合中动态3D场景的引人入胜的技术。这些容量表示非常适合综合新型面部表情，但两个问题。首先，可变形的nerf是对象不可知论和模型的整体运动：他们可以重播运动的时间如何随时间变化，但他们不能以可解释的方式改变它。其次，可控的体积表示通常需要耗时的手动注释或3D监督，以便为场景提供语义含义。我们为面部自画像（Condies）提出了可控的神经表示，该神经表示在共同的框架内解决了这两个问题，并且可以依靠自动处理。我们使用自动化的面部动作识别（AFAR）将面部表情描述为动作单位（AU）及其强度的组合。 AUS为系统提供语义位置和控制标签。在表达的视觉和解剖学保真度方面，对新型视图和表达综合的表现优于竞争的方法。
Neural Radiance Fields (NeRF) are compelling techniques for modeling dynamic 3D scenes from 2D image collections. These volumetric representations would be well suited for synthesizing novel facial expressions but for two problems. First, deformable NeRFs are object agnostic and model holistic movement of the scene: they can replay how the motion changes over time, but they cannot alter it in an interpretable way. Second, controllable volumetric representations typically require either time-consuming manual annotations or 3D supervision to provide semantic meaning to the scene. We propose a controllable neural representation for face self-portraits (CoNFies), that solves both of these problems within a common framework, and it can rely on automated processing. We use automated facial action recognition (AFAR) to characterize facial expressions as a combination of action units (AU) and their intensities. AUs provide both the semantic locations and control labels for the system. CoNFies outperformed competing methods for novel view and expression synthesis in terms of visual and anatomic fidelity of expressions.
