论文标题
联合学习超参数从系统角度进行调整
Federated Learning Hyper-Parameter Tuning from a System Perspective
论文作者
论文摘要
联合学习（FL）是一个分布式模型培训范式，可保留客户的数据隐私。它引起了学术界和工业的极大关注。 FL超参数（例如，选定客户的数量和培训通行证的数量）在计算时间，传输时间，计算负载和传输负载方面显着影响训练开销。但是，由于应用程序具有不同的培训偏好，因此目前手动选择FL超参数的实践对FL从业人员造成了沉重的负担。在本文中，我们提出了FedTune，这是一种针对应用程序在FL培训中的多种系统要求量身定制的自动FL高参数调谐算法。 FedTune迭代在FL训练过程中调整FL超参数，并且可以轻松地集成到现有的FL系统中。通过对多种应用和FL聚合算法的FedTune进行广泛的评估，我们表明，与使用固定的FL超参数相比，FedTune轻巧有效，可实现8.48％-26.75％的系统架空降低。本文协助FL从业人员设计高性能的FL培训解决方案。 FedTune的源代码可从https://github.com/datasystech/fedtune获得。
Federated learning (FL) is a distributed model training paradigm that preserves clients' data privacy. It has gained tremendous attention from both academia and industry. FL hyper-parameters (e.g., the number of selected clients and the number of training passes) significantly affect the training overhead in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters imposes a heavy burden on FL practitioners because applications have different training preferences. In this paper, we propose FedTune, an automatic FL hyper-parameter tuning algorithm tailored to applications' diverse system requirements in FL training. FedTune iteratively adjusts FL hyper-parameters during FL training and can be easily integrated into existing FL systems. Through extensive evaluations of FedTune for diverse applications and FL aggregation algorithms, we show that FedTune is lightweight and effective, achieving 8.48%-26.75% system overhead reduction compared to using fixed FL hyper-parameters. This paper assists FL practitioners in designing high-performance FL training solutions. The source code of FedTune is available at https://github.com/DataSysTech/FedTune.
