论文标题
关于转移学习理论：任务多样性的重要性
On the Theory of Transfer Learning: The Importance of Task Diversity
论文作者
论文摘要
我们为通过表示学习提供了新的统计保证，用于转移学习 - 通过学习在不同任务中共享的功能表示来实现时。这使得使用新任务学习的数据远远少于孤立学习它们所需的数据。正式地，我们考虑通过$ f_j \ circ h $参数化的$ t+1 $任务，在一般函数类$ \ mathcal {f} \ circ \ mathcal {h} $中，每个$ f_j $是$ \ m nathcal {f} $ and $ h $ inshen Chum $ nath $ ressancations的每个$ f_j $ cons $ f_j $。让$ c（\ cdot）$表示功能类别的复杂性度量，我们表明，对于多样化的培训任务（1）在第一个$ t $训练任务中学习共享表示所需的示例复杂性为$ c（\ mathcal {h} {h}） + t c（\ mathcal {f} f} $，尽管expections and expectiation and extimients and scriptimate and secrients and secrient and synections and Crienditions and and and and and and and synections（2）仅使用$ c（\ Mathcal {f}）$学习新任务所需的样本复杂性。我们的结果取决于任务多样性的新一般概念（可用于具有一般任务，功能和损失的模型）以及高斯复杂性的新链条规则。最后，我们在文献中的几种重要模型中展示了我们的一般框架的实用性。
We provide new statistical guarantees for transfer learning via representation learning--when transfer is achieved by learning a feature representation shared across different tasks. This enables learning on new tasks using far less data than is required to learn them in isolation. Formally, we consider $t+1$ tasks parameterized by functions of the form $f_j \circ h$ in a general function class $\mathcal{F} \circ \mathcal{H}$, where each $f_j$ is a task-specific function in $\mathcal{F}$ and $h$ is the shared representation in $\mathcal{H}$. Letting $C(\cdot)$ denote the complexity measure of the function class, we show that for diverse training tasks (1) the sample complexity needed to learn the shared representation across the first $t$ training tasks scales as $C(\mathcal{H}) + t C(\mathcal{F})$, despite no explicit access to a signal from the feature representation and (2) with an accurate estimate of the representation, the sample complexity needed to learn a new task scales only with $C(\mathcal{F})$. Our results depend upon a new general notion of task diversity--applicable to models with general tasks, features, and losses--as well as a novel chain rule for Gaussian complexities. Finally, we exhibit the utility of our general framework in several models of importance in the literature.
