Large Language Models for Machine Translation Quality Annotation: Humans and Models Are Both Challenged

Researchers evaluated large language models (LLMs) against human annotators for machine translation (MT) quality assessment using MQM and ESA schemes across 70 language pairs. Results show LLMs sometimes outperform humans but both face reliability challenges depending on language and annotation type, suggesting potential for collaborative human-LLM approaches.