Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation

这篇论文主要是想利用合成的数据来解决迁移学习的问题。

abstract

Visual Domain Adaptation is a problem of immense importance in computer vision. Previous approaches showcase the inability of even deep neural networks to learn informative representations across domain shift. This problem is more severe for tasks where acquiring hand labeled data is extremely hard and tedious. In this work, we focus on adapting the representations learned by segmentation networks across synthetic and real domains. Contrary to previous approaches that use a simple adversarial objective or superpixel information to aid the process, we propose an approach based on Generative Adversarial Networks (GANs) that brings the embeddings closer in the learned feature space. To showcase the generality and scalability of our approach, we show that we can achieve state of the art results on two challenging scenarios of synthetic to real domain adaptation. Additional exploratory experiments show that our approach: (1) generalizes to unseen domains and (2) results in improved alignment of source and target distributions.

introduction

他们主要的工作也是解决,没有label的domain adaptation问题。和之前FCNs那篇将adversarial loss应用在feature space不同,他们将adversarial loss应用在image space,并是有discriminator导出的。此外,他们还用了生成模型将源域和目标域的分布在feature space里面对齐了。然后,他这里面提到了style space: 其他的很多工作做的是将源域的图做了一个image translation,换句话说就是实现实际的风格迁移,然后在train。

structure


然后结构里面G network和D network就是GAN了。

loss function

感觉对loss function的拆解和分析是理解相关approach的关键。下面我们逐个分析