Bayesian Optimization in Wasserstein Spaces

Bayesian Optimization (BO) is a sample efficient approach for approximating the global optimum of black-box and computationally expensive optimization problems which has proved its effectiveness in a wide range of engineering and machine learning problems. A limiting factor in its applications is the difficulty of scaling over 15–20 dimensions. It has been remarked that global optimization problems often have a lower intrinsic dimensionality which can be exploited to construct a feature mapping the original problem into low dimension manifold. In this paper we take a novel approach mapping the original problem into a space of discrete probability distributions endowed with a Wasserstein metric. In this new approach both the Gaussian process model and the acquisition function work in a 1-dimensional Wasserstein (WST) space. The results in the WST space are then mapped back to the original space using a neural network. Computational results show that, at least for high dimension additive test functions, the exploration in the Wasserstein space is significantly more effective.