Abstract

For the global community of people who are hearing or speech handicapped, sign language is an essential communication tool. While languages like Tamil and Hindi have found their way into Indian technology, Kannada Sign Language (KSL) has not yet been extensively used in digital applications. In order to recognize Kannada sign language, this research suggests a revolutionary machine learning-based method that focuses on both letter and word detection. By gathering a unique dataset of more than 6,000 samples from 20 different participants, our project fills the gap and guarantees resilience against changes in signing conditions and style. In order to minimize computational cost and enable precise gesture capture, we use MediaPipe to extract 21 hand keypoints. Two sophisticated models are integrated into the classification framework: MobileNetV2, which is tuned for static letter recognition, and a 3D Convolutional Neural Network (3D CNN), which is designed for dynamic word gestures. Our experimental results outperform current Indian sign recognition models with an impressive 99.7% letter identification accuracy and an 85% word recognition accuracy. The difficulties in creating datasets, the intricacies of sign variation, and the current small repertoire of 27 words are the main obstacles encountered. However, this study establishes a solid basis for future scalable Kannada sign language recognition systems that can help the deaf people in Karnataka communicate. Future plans call for improving dynamic gesture detection, adding more words to the dictionary, and implementing the system on mobile devices.