Techniques for evaluating the performance of landslide susceptibility models

Evaluating the performance of landslide susceptibility models is needed to ensure their reliable application to risk management and land-use planning. When results from multiple models are available, a comparison of their performance is necessary to select the model which performs better. In this paper, different techniques to evaluate model performance are discussed and tested using shallow landslide/debris-flow susceptibility models recently presented in the literature (Carrara, A., Crosta, G.B., Frattini, P., 2008. Comparing models of debris-flow susceptibility in the alpine environment. Geomorphology 94 (3–4), 353–378). Moreover, an evaluation technique based on the minimization of costs that may arise from the adoption of the model as a land management regulatory tool, is presented. The results of the application show that simple statistics such as Accuracy, Threat score, Gilbert's skill score, Pierce's skill score, Heidke skill score, and Yule's Q are problematic as they need to split the classified objects into two classes (e.g., stable/unstable) by defining an a-priori value of cutoff susceptibility, which is often not trivial. ROC curves and Success-Rate curves are cutoff-independent and can be used to efficiently visualize and compare the performance of models, but do not explicitly include classification costs. In addition, Success-Rate curves, under certain conditions, can be misleading when applied to grid-cell models. Cost curves include costs and a-priori probabilities, and are suitable for landslide susceptibility model performance evaluation from a practical point of view.