Nonlinear Regression and Neural Networks

In this paper the authors are concerned with the problem of empirical model building when models from the class of Feedforward Neural Networks are considered. In this case, empirical model building consists of the following four tasks: network's structure selection, variables selection, search for the optimal network's parameter values and model criticism. All these tasks, which are strongly connected, are accomplished by using experimental data and with the aim to provide a model capable to describe the main features of the Data Generating Process. The main concern of the paper is to provide a statistical framework for empirical model building. To this extent a new class of networks, namely the class of $\alpha $-non rejectable networks, is introduced and defined. In particular, the authors emphasize the importance of realizing that searching for the optimal network's parameter values is a nonlinear regression problem, thus the main results from the nonlinear regression framework can be exploited for checking whether a given Feedforward Neural Network belongs to the class of $\alpha $-non rejectable networks. The checking is made through hypothesis testing on model parameters and nonlinear regression analysis provides a method, named profile t function, which enables the computation of exact likelihood regions and to reveal how nonlinear the parameter is. The issues discussed in this paper, as well as the statistical framework proposed to deal with them, albeit developed for Feedforward Neural Networks, apply to other Artificial Neural Networks as well. Several numerical experiments, based on significant case studies, are presented and discussed.