论文标题
分类单元：深神经网络培训的轻巧加速器
TaxoNN: A Light-Weight Accelerator for Deep Neural Network Training
论文作者
论文摘要
新兴的智能嵌入式设备依靠深度神经网络（DNN）能够与现实环境进行交互。由于环境条件在及时不断变化，因此这种相互作用具有重新培训DNN的能力。随机梯度下降（SGD）是一种通过在训练数据上优化参数来训练DNNS的广泛使用算法。在这项工作中，首先，我们提出了一种新颖的方法，将训练能力添加到基线DNN加速器（仅推断）中，通过将SGD算法分为简单的计算元素，将训练能力（仅推断）。然后，基于这种启发式方法，我们提出了DNN培训的轻巧加速器Taxonn。分类单元可以通过使用时间拼写方法和低宽度单元重复推理过程中使用的硬件资源来轻松调整DNN权重。我们的实验结果表明，与完整的实施相比，分类单元平均会提高错误分类率0.97％。此外，分类单元提供2.1 $ \ times $节省和1.65 $ \ times $ $ $ $ \ times $ $ \ times $ $ \ times $ $ \ times $。
Emerging intelligent embedded devices rely on Deep Neural Networks (DNNs) to be able to interact with the real-world environment. This interaction comes with the ability to retrain DNNs, since environmental conditions change continuously in time. Stochastic Gradient Descent (SGD) is a widely used algorithm to train DNNs by optimizing the parameters over the training data iteratively. In this work, first we present a novel approach to add the training ability to a baseline DNN accelerator (inference only) by splitting the SGD algorithm into simple computational elements. Then, based on this heuristic approach we propose TaxoNN, a light-weight accelerator for DNN training. TaxoNN can easily tune the DNN weights by reusing the hardware resources used in the inference process using a time-multiplexing approach and low-bitwidth units. Our experimental results show that TaxoNN delivers, on average, 0.97% higher misclassification rate compared to a full-precision implementation. Moreover, TaxoNN provides 2.1$\times$ power saving and 1.65$\times$ area reduction over the state-of-the-art DNN training accelerator.
