In this paper we present CMRNet, a realtime approach based on a Convolutional Neural Network (CNN) to localize an RGB image of a scene in a map built from LiDAR data. Our network is not trained in the working area, i. e., CMRNet does not learn the map. Instead it learns to match an image to the map. We validate our approach on the KITTI dataset, processing each frame independently without any tracking procedure. CMRNet achieves 0.27m and 1.07° median localization accuracy on the sequence 00 of the odometry dataset, starting from a rough pose estimate displaced up to 3.5m and 17°. To the best of our knowledge this is the first CNN-based approach that learns to match images from a monocular camera to a given, preexisting 3D LiDAR-map.

Cattaneo, D., Vaghi, M., Ballardini, A., Fontana, S., Sorrenti, D., Burgard, W. (2019). CMRNet: Camera to LiDAR-Map Registration. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC) (pp.1283-1289). Institute of Electrical and Electronics Engineers Inc. [10.1109/ITSC.2019.8917470].

CMRNet: Camera to LiDAR-Map Registration

Daniele Cattaneo
Primo
;
Matteo Vaghi
Secondo
;
Augusto Luis Ballardini;Simone Fontana;Domenico Giorgio Sorrenti
Penultimo
;
2019

Abstract

In this paper we present CMRNet, a realtime approach based on a Convolutional Neural Network (CNN) to localize an RGB image of a scene in a map built from LiDAR data. Our network is not trained in the working area, i. e., CMRNet does not learn the map. Instead it learns to match an image to the map. We validate our approach on the KITTI dataset, processing each frame independently without any tracking procedure. CMRNet achieves 0.27m and 1.07° median localization accuracy on the sequence 00 of the odometry dataset, starting from a rough pose estimate displaced up to 3.5m and 17°. To the best of our knowledge this is the first CNN-based approach that learns to match images from a monocular camera to a given, preexisting 3D LiDAR-map.
slide + paper
deep learning, robotics, localization, sensor fusion
English
2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019
2019
2019 IEEE Intelligent Transportation Systems Conference (ITSC)
9781538670248
2019
1283
1289
8917470
reserved
Cattaneo, D., Vaghi, M., Ballardini, A., Fontana, S., Sorrenti, D., Burgard, W. (2019). CMRNet: Camera to LiDAR-Map Registration. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC) (pp.1283-1289). Institute of Electrical and Electronics Engineers Inc. [10.1109/ITSC.2019.8917470].
File in questo prodotto:
File Dimensione Formato  
08917470.pdf

Solo gestori archivio

Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Dimensione 2.95 MB
Formato Adobe PDF
2.95 MB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/252425
Citazioni
  • Scopus 51
  • ???jsp.display-item.citation.isi??? 40
Social impact