Experimental Design for Optimization of Orthogonal Projection Pursuit Models
Metadata only
Date
2020-04-03Type
- Conference Paper
ETH Bibliography
yes
Altmetrics
Abstract
Bayesian optimization and kernelized bandit algorithms are widely used techniques for sequential black box function optimization with applications in parameter tuning, control, robotics among many others. To be effective in high dimensional settings, previous approaches make additional assumptions, for example on low-dimensional subspaces or an additive structure. In this work, we go beyond the additivity assumption and use an orthogonal projection pursuit regression model, which strictly generalizes additive models. We present a two-stage algorithm motivated by experimental design to first decorrelate the additive components. Subsequently, the bandit optimization benefits from the statistically efficient additive model. Our method provably decorrelates the fully additive model and achieves optimal sublinear simple regret in terms of the number of function evaluations. To prove the rotation recovery, we derive novel concentration inequalities for linear regression on subspaces. In addition, we specifically address the issue of acquisition function optimization and present two domain dependent efficient algorithms. We validate the algorithm numerically on synthetic as well as real-world optimization problems. Show more
Publication status
publishedExternal links
Journal / series
Proceedings of the AAAI Conference on Artificial IntelligenceVolume
Pages / Article No.
Publisher
AAAIEvent
Organisational unit
03908 - Krause, Andreas / Krause, Andreas
Funding
167212 - Scaling Up by Scaling Down: Big ML via Small Coresets (SNF)
815943 - Reliable Data-Driven Decision Making in Cyber-Physical Systems (EC)
More
Show all metadata
ETH Bibliography
yes
Altmetrics