论文标题
通过生成潜在搜索,无监督的域适应NIR图像的语义分割
Unsupervised Domain Adaptation for Semantic Segmentation of NIR Images through Generative Latent Search
论文作者
论文摘要
与人皮肤相对应的像素的分割是多种应用的重要第一步,从监视到远程光绘画学的心率估计。但是,现有文献仅在EM-Spectrum的可见范围内考虑问题,该问题限制了其在应用程序的关键性更高的低光或无光设置中的效用。为了减轻这个问题,我们考虑了近红外图像的皮肤分割问题。但是,基于深度学习的最先进的细分技术需要大量标记的数据,而这些数据对于当前问题不可用。因此,我们将皮肤分割问题作为独立的无监督域适应性(UDA)的皮肤分割问题,在此我们使用可见范围的红通道中的数据来开发NIR图像上的皮肤分割算法。我们提出了一种独立于目标分割的方法,其中搜索了源域中目标图像的“最近克隆”,并将其用作仅在源域上训练的分割网络中的代理。我们证明了“最近克隆”的存在,并提出了一种通过基于变异推理的深生成模型的潜在空间的优化算法来找到它的方法。我们证明了NIR皮肤分割的疗效,尽管无法访问目标NIR数据,但在NIR域中两个新创建的皮肤分割数据集上的最先进的UDA分割方法对最新的UDA分割方法的功效。此外,我们报告了从合成到CityScapes适应的最新结果,这是无监督的域适应性语义细分的流行环境。该代码和数据集可从https://github.com/ambekarsameer96/glss获得。
Segmentation of the pixels corresponding to human skin is an essential first step in multiple applications ranging from surveillance to heart-rate estimation from remote-photoplethysmography. However, the existing literature considers the problem only in the visible-range of the EM-spectrum which limits their utility in low or no light settings where the criticality of the application is higher. To alleviate this problem, we consider the problem of skin segmentation from the Near-infrared images. However, Deep learning based state-of-the-art segmentation techniques demands large amounts of labelled data that is unavailable for the current problem. Therefore we cast the skin segmentation problem as that of target-independent Unsupervised Domain Adaptation (UDA) where we use the data from the Red-channel of the visible-range to develop skin segmentation algorithm on NIR images. We propose a method for target-independent segmentation where the 'nearest-clone' of a target image in the source domain is searched and used as a proxy in the segmentation network trained only on the source domain. We prove the existence of 'nearest-clone' and propose a method to find it through an optimization algorithm over the latent space of a Deep generative model based on variational inference. We demonstrate the efficacy of the proposed method for NIR skin segmentation over the state-of-the-art UDA segmentation methods on the two newly created skin segmentation datasets in NIR domain despite not having access to the target NIR data. Additionally, we report state-of-the-art results for adaption from Synthia to Cityscapes which is a popular setting in Unsupervised Domain Adaptation for semantic segmentation. The code and datasets are available at https://github.com/ambekarsameer96/GLSS.