Neural architecture search for image saliency fusion

Saliency detection methods proposed in the literature exploit different rationales, visual clues, and assumptions, but there is no single best saliency detection algorithm that is able to achieve good results on all the different benchmark datasets. In this paper we show that fusing different saliency detection algorithms together by exploiting neural network architectures makes it possible to obtain better results. Designing the best architecture for a given task is still an open problem since the existing techniques have some limits with respect to the problem formulation, to the search space, and require very high computational resources. To overcome these problems, in this paper we propose a three-step fusion approach. In the first step, genetic programming techniques are exploited to combine the outputs of existing saliency algorithms using a set of provided operations. Having a discrete search space allows us a fast generation of the candidate solutions. In the second step, the obtained solutions are converted into backbone Convolutional Neural Networks (CNNs) where operations are all implemented with differentiable functions, allowing an efficient optimization of the corresponding parameters (in a continuous space) by backpropagation. In the last step, to enrich the expressiveness of the initial architectures, the networks are further extended with additional operations on intermediate levels of the processing that are once again efficiently optimized through backpropagation. Extensive experimental evaluations show that the proposed saliency fusion approach outperforms the state-of-the-art on the MSRAB dataset and it is able to generalize to unseen data of different benchmark datasets.