论文标题
C-TPE：树木结构化的parzen估计量，具有不平等的限制，用于昂贵的超参数优化
c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimization
论文作者
论文摘要
超参数优化（HPO）对于深度学习算法的强劲性能和现实世界应用程序通常会施加一些约束，例如记忆使用或延迟在性能要求之上。在这项工作中，我们提出了受约束的TPE（C-TPE），这是广泛使用的贝叶斯优化方法，树结构的Parzen估计量（TPE）的扩展，以处理这些约束。我们提出的扩展超出了现有的采集功能和原始TPE的简单组合，而包括解决导致性能差的问题的修改。我们从经验和理论上彻底分析了这些修改，从而提供了有关它们如何有效克服这些挑战的见解。在实验中，我们证明C-TPE在现有方法中表现出最佳的平均等级性能，其统计学意义对81个昂贵的HPO具有不平等约束。由于缺乏基线，我们仅讨论我们方法在附录D中进行严格约束优化的适用性。
Hyperparameter optimization (HPO) is crucial for strong performance of deep learning algorithms and real-world applications often impose some constraints, such as memory usage, or latency on top of the performance requirement. In this work, we propose constrained TPE (c-TPE), an extension of the widely-used versatile Bayesian optimization method, tree-structured Parzen estimator (TPE), to handle these constraints. Our proposed extension goes beyond a simple combination of an existing acquisition function and the original TPE, and instead includes modifications that address issues that cause poor performance. We thoroughly analyze these modifications both empirically and theoretically, providing insights into how they effectively overcome these challenges. In the experiments, we demonstrate that c-TPE exhibits the best average rank performance among existing methods with statistical significance on 81 expensive HPO with inequality constraints. Due to the lack of baselines, we only discuss the applicability of our method to hard-constrained optimization in Appendix D.
