MODELING COGNITIVE CONSTRAINTS IN SIGNED-TO-SPOKEN TRANSLATION: A MULTIMODAL NEURAL APPROACH

Abstrak

In recent years, the field of sign language translation (SLT) has evolved rapidly due to advances in deep learning and multimodal computing. However, despite this progress, translation from signed to spoken languages remains a complex challenge due to fundamental cognitive, linguistic, and modality-specific constraints. This paper investigates how cognitive factors — including working memory limits, multimodal perception, and temporal alignment — affect the process of translating visual-manual languages into spoken forms. The study also explores how multimodal neural networks, especially transformer-based architectures, can model and mitigate these constraints. Using statistical data and comparative analysis from developed countries (the USA, UK, Germany) and emerging contexts, the paper highlights the current state of SLT research, key datasets, and BLEU score benchmarks, providing recommendations for future development of inclusive AI systems that serve deaf and hard-of-hearing communities worldwide.