论文标题
拜占庭式学习在异质数据集上通过铲斗
Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing
论文作者
论文摘要
在拜占庭强大的分布或联合学习中，中央服务器希望通过在多个工人中分布的数据来训练机器学习模型。但是，这些工人中的一小部分可能会偏离规定的算法并发送任意消息。尽管该问题最近受到了重大关注，但大多数当前的防御能力都认为工人具有相同的数据。对于现实的情况，当工人的数据是异质的（非IID）时，我们设计了新的攻击，这些攻击避免了当前的防御力，从而导致绩效的巨大丧失。然后，我们提出了一个简单的存储方案，该方案以微不足道的计算成本调整了现有鲁棒算法到异质数据集。我们在理论上和实验上也验证了我们的方法，表明将铲斗与现有的强大算法相结合是有效的，可以防止挑战性攻击。我们的工作是第一个在现实假设下为非IID拜占庭强大问题确定保证融合的工作。
In Byzantine robust distributed or federated learning, a central server wants to train a machine learning model over data distributed across multiple workers. However, a fraction of these workers may deviate from the prescribed algorithm and send arbitrary messages. While this problem has received significant attention recently, most current defenses assume that the workers have identical data. For realistic cases when the data across workers are heterogeneous (non-iid), we design new attacks which circumvent current defenses, leading to significant loss of performance. We then propose a simple bucketing scheme that adapts existing robust algorithms to heterogeneous datasets at a negligible computational cost. We also theoretically and experimentally validate our approach, showing that combining bucketing with existing robust algorithms is effective against challenging attacks. Our work is the first to establish guaranteed convergence for the non-iid Byzantine robust problem under realistic assumptions.
