Annotating tables with high-quality semantic labels and linking cells in these tables to entities in reference knowledge graphs (KGs) serve several downstream applications such as knowledge graph construction, data enrichment, data analytics, and so on. There is an increasing interest in combining algorithms for the automatic annotation of tables with interactive applications that let users control and improve the annotations and use them for data enrichment. In this paper, we propose a supervised entity linking network that behaves as a re-ranker and score normalizer of scores returned by off-the-shelves entity retrieval systems. By combining these refined scores with additional signals of reliability of the ranking obtained thereof, we can estimate a confidence score, which tells how reliable is the decision to link a cell to the top-ranked candidate entity. The confidence score is used to decide whether to establish a link or not and to prioritize the cells to be revised by the users to improve the annotation quality. Experiments suggest that the proposed approach provides an effective approach to identifying most critical decisions and supporting link revision within a human-in-the-Ioop data enrichment paradigm.
Avogadro, R., Ciavotta, M., De Paoli, F., Palmonari, M., Roman, D. (2023). Estimating Link Confidence for Human-in-the-Loop Table Annotation. In Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023 (pp.142-149). Institute of Electrical and Electronics Engineers Inc. [10.1109/wi-iat59888.2023.00025].
Estimating Link Confidence for Human-in-the-Loop Table Annotation
Avogadro, Roberto;Ciavotta, Michele;De Paoli, Flavio
;Palmonari, Matteo;
2023
Abstract
Annotating tables with high-quality semantic labels and linking cells in these tables to entities in reference knowledge graphs (KGs) serve several downstream applications such as knowledge graph construction, data enrichment, data analytics, and so on. There is an increasing interest in combining algorithms for the automatic annotation of tables with interactive applications that let users control and improve the annotations and use them for data enrichment. In this paper, we propose a supervised entity linking network that behaves as a re-ranker and score normalizer of scores returned by off-the-shelves entity retrieval systems. By combining these refined scores with additional signals of reliability of the ranking obtained thereof, we can estimate a confidence score, which tells how reliable is the decision to link a cell to the top-ranked candidate entity. The confidence score is used to decide whether to establish a link or not and to prioritize the cells to be revised by the users to improve the annotation quality. Experiments suggest that the proposed approach provides an effective approach to identifying most critical decisions and supporting link revision within a human-in-the-Ioop data enrichment paradigm.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.