Multimodal Earth Observation Modeling using AI

Earth observation (EO) is critical in many applications involving land security, environmental monitoring, agriculture, urban planning, and disaster management. Given its importance in EO, remote sensing (RS) has become an irreplaceable tool, as it allows for the rapid collection of diverse and accurate data. In recent years, in particular, RS has undergone strong technological innovation due to the use of (1) artificial intelligence and (2) multimodal approaches. The increase in available RS data, in fact, allows the implementation of the most sophisticated and data-hungry state-of-the-art techniques, showing improved accuracy, generalization, and efficiency. At the same time, the availability of different types of modalities allows exploiting the advantages of combining different types of information such as optical, thermal, radar, LiDAR, multispectral, and hyperspectral to describe different soil features. These two tasks together can prompt research in various fields including semantic segmentation, change detection, digital soil mapping, etc. This paper discusses the potential of combining AI techniques and multimodal approaches by collecting a large amount of data, adapted to AI techniques, including spectral and digital elevation models. In particular, the work focuses on two specific use cases: 1) semantic segmentation (SemSeg) and 2) digital soil mapping (DSM). SemSeg aims to classify each pixel in an image into specific object classes. DSM aims to map the soil properties. Both provide valuable information for land management, agriculture, and environmental studies. The results of our experiments confirmed the usefulness of combining AI and multimodal, guiding further developments in EO research.