Annotating tables with high-quality semantic labels and linking cells in these tables to entities in reference knowledge graphs (KGs) serve several downstream applications such as knowledge graph construction, data enrichment, data analytics, and so on. There is an increasing interest in combining algorithms for the automatic annotation of tables with interactive applications that let users control and improve the annotations and use them for data enrichment. In this paper, we propose a supervised entity linking network that behaves as a re-ranker and score normalizer of scores returned by off-the-shelves entity retrieval systems. By combining these refined scores with additional signals of reliability of the ranking obtained thereof, we can estimate a confidence score, which tells how reliable is the decision to link a cell to the top-ranked candidate entity. The confidence score is used to decide whether to establish a link or not and to prioritize the cells to be revised by the users to improve the annotation quality. Experiments suggest that the proposed approach provides an effective approach to identifying most critical decisions and supporting link revision within a human-in-the-Ioop data enrichment paradigm.

Avogadro, R., Ciavotta, M., De Paoli, F., Palmonari, M., Roman, D. (2023). Estimating Link Confidence for Human-in-the-Loop Table Annotation. In Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023 (pp.142-149). Institute of Electrical and Electronics Engineers Inc. [10.1109/wi-iat59888.2023.00025].

Estimating Link Confidence for Human-in-the-Loop Table Annotation

Avogadro, Roberto;Ciavotta, Michele;De Paoli, Flavio
;
Palmonari, Matteo;
2023

Abstract

Annotating tables with high-quality semantic labels and linking cells in these tables to entities in reference knowledge graphs (KGs) serve several downstream applications such as knowledge graph construction, data enrichment, data analytics, and so on. There is an increasing interest in combining algorithms for the automatic annotation of tables with interactive applications that let users control and improve the annotations and use them for data enrichment. In this paper, we propose a supervised entity linking network that behaves as a re-ranker and score normalizer of scores returned by off-the-shelves entity retrieval systems. By combining these refined scores with additional signals of reliability of the ranking obtained thereof, we can estimate a confidence score, which tells how reliable is the decision to link a cell to the top-ranked candidate entity. The confidence score is used to decide whether to establish a link or not and to prioritize the cells to be revised by the users to improve the annotation quality. Experiments suggest that the proposed approach provides an effective approach to identifying most critical decisions and supporting link revision within a human-in-the-Ioop data enrichment paradigm.
paper
Entity Linking; Entity Reconciliation; Human in the Loop; Machine Learning; Semantic Table Interpretation;
English
22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023 - 26-29 October 2023
2023
Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023
9798350309188
2023
142
149
none
Avogadro, R., Ciavotta, M., De Paoli, F., Palmonari, M., Roman, D. (2023). Estimating Link Confidence for Human-in-the-Loop Table Annotation. In Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023 (pp.142-149). Institute of Electrical and Electronics Engineers Inc. [10.1109/wi-iat59888.2023.00025].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/467025
Citazioni
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 1
Social impact