论文标题
通过示例：基于示例的图像编辑，扩散模型
Paint by Example: Exemplar-based Image Editing with Diffusion Models
论文作者
论文摘要
语言指导的图像编辑最近取得了巨大的成功。在本文中，我们首次研究了示例引导的图像编辑，以进行更精确的控制。我们通过利用自我监督的培训来解散并重组源图像和示例来实现这一目标。但是，幼稚的方法将引起明显的融合伪像。我们仔细分析了它，并提出了一个信息瓶颈和强大的增强材料，以避免直接复制和粘贴示例图像的微不足道解决方案。同时，为了确保编辑过程的可控性，我们为示例图像设计一个任意形状掩码，并利用无分类器指导来增加与示例图像的相似性。整个框架涉及扩散模型的一个前进，而没有任何迭代优化。我们证明我们的方法实现了令人印象深刻的性能，并可以在具有高保真度的野外图像上进行可控的编辑。
Language-guided image editing has achieved great success recently. In this paper, for the first time, we investigate exemplar-guided image editing for more precise control. We achieve this goal by leveraging self-supervised training to disentangle and re-organize the source image and the exemplar. However, the naive approach will cause obvious fusing artifacts. We carefully analyze it and propose an information bottleneck and strong augmentations to avoid the trivial solution of directly copying and pasting the exemplar image. Meanwhile, to ensure the controllability of the editing process, we design an arbitrary shape mask for the exemplar image and leverage the classifier-free guidance to increase the similarity to the exemplar image. The whole framework involves a single forward of the diffusion model without any iterative optimization. We demonstrate that our method achieves an impressive performance and enables controllable editing on in-the-wild images with high fidelity.
