论文标题
为3D点云语义和实例分段学习和记忆的代表性原型
Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation
论文作者
论文摘要
3D点云语义和实例细分至关重要，对于3D场景的理解至关重要。由于复杂的结构，点集分布在平衡和多样性上，这似乎是类别失衡和模式失衡。结果，深层网络可以很容易地忘记学习过程中的非主导案例，从而导致性能不令人满意。尽管重新加权可以减少分类良好的示例的影响，但它们在动态训练过程中无法处理非优势模式。在本文中，我们提出了一个由内存的网络，以学习和记住普遍涵盖不同样本的代表性原型。具体而言，引入了一个内存模块，以通过记录迷你批次培训中的模式来减轻遗忘问题。博学的记忆项目始终反映了主要和非主导类别和案例的可解释和有意义的信息。因此，可以通过检索存储的原型来增强扭曲的观察结果和罕见情况，从而提高性能和泛化。基准的详尽实验，即S3DIS和ScannETV2，反映了我们方法在有效性和效率上的优越性。不仅总体准确性，而且非主导阶级也有所提高。
3D point cloud semantic and instance segmentation is crucial and fundamental for 3D scene understanding. Due to the complex structure, point sets are distributed off balance and diversely, which appears as both category imbalance and pattern imbalance. As a result, deep networks can easily forget the non-dominant cases during the learning process, resulting in unsatisfactory performance. Although re-weighting can reduce the influence of the well-classified examples, they cannot handle the non-dominant patterns during the dynamic training. In this paper, we propose a memory-augmented network to learn and memorize the representative prototypes that cover diverse samples universally. Specifically, a memory module is introduced to alleviate the forgetting issue by recording the patterns seen in mini-batch training. The learned memory items consistently reflect the interpretable and meaningful information for both dominant and non-dominant categories and cases. The distorted observations and rare cases can thus be augmented by retrieving the stored prototypes, leading to better performances and generalization. Exhaustive experiments on the benchmarks, i.e. S3DIS and ScanNetV2, reflect the superiority of our method on both effectiveness and efficiency. Not only the overall accuracy but also nondominant classes have improved substantially.
