This paper reports on our participation with the MUCOW test suite at the WMT 2020 news translation task. We introduced MUCOW at WMT 2019 to measure the ability of MT systems to perform word sense disambiguation (WSD), i.e., to translate an ambiguous word with its correct sense. MUCOW is created automatically using existing resources, and the evaluation process is also entirely automated. We evaluate all participating systems of the language pairs English -{ extgreater} Czech, English -{ extgreater} German, and English -{ extgreater} Russian and compare the results with those obtained at WMT 2019. While current NMT systems are fairly good at handling ambiguous source words, we could not identify any substantial progress - at least to the extent that it is measurable by the MUCOW method - in that area over the last year.
Scherrer, Y., Raganato, A., Tiedemann, J. (2020). The MUCOW word sense disambiguation test suite at WMT 2020. In Proceedings of the Fifth Conference on Machine Translation (pp.365-370). Association for Computational Linguistics.
The MUCOW word sense disambiguation test suite at WMT 2020
Raganato, A
;
2020
Abstract
This paper reports on our participation with the MUCOW test suite at the WMT 2020 news translation task. We introduced MUCOW at WMT 2019 to measure the ability of MT systems to perform word sense disambiguation (WSD), i.e., to translate an ambiguous word with its correct sense. MUCOW is created automatically using existing resources, and the evaluation process is also entirely automated. We evaluate all participating systems of the language pairs English -{ extgreater} Czech, English -{ extgreater} German, and English -{ extgreater} Russian and compare the results with those obtained at WMT 2019. While current NMT systems are fairly good at handling ambiguous source words, we could not identify any substantial progress - at least to the extent that it is measurable by the MUCOW method - in that area over the last year.
| File | Dimensione | Formato | |
|---|---|---|---|
|
2020.wmt-1.40.pdf
Solo gestori archivio
Dimensione
249.79 kB
Formato
Adobe PDF
|
249.79 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
