Anvitha Infotech
This project is an intelligent communication assistance system that converts Indian Sign Language (ISL) gestures into text and speech in real time. The system uses computer vision and deep learning to recognize hand gestures through …
Communication barriers often create significant challenges for people with hearing and speech impairments, limiting their ability to interact naturally with those unfamiliar with sign language. This gap frequently leads to dependence on interpreters or written communication, which can be inconvenient, time-consuming, and less spontaneous than natural conversation. This project proposes an AI-based Sign Language Recognition and Voice Cloning System that enables seamless interaction between sign language users and non-sign language users, aiming to make everyday communication more direct, accessible, and inclusive.The system captures hand gestures through a webcam and identifies sign language alphabets using a trained deep learning model, allowing real-time interpretation of hand movements without requiring specialized hardware beyond a standard camera. The recognized gestures are converted into text and then transformed into speech using advanced voice cloning technology, generating audio in the user's personalized voice. This personalization adds a meaningful human element to the interaction, allowing the spoken output to reflect the user's own vocal identity rather than a generic synthesized voice, making communication feel more natural and personal.Furthermore, the application includes text-to-sign conversion, allowing normal text to be represented as sign language images, thereby supporting two-way communication where non-sign language users can also convey information back to sign language users in a visually interpretable format. This bidirectional capability ensures that the system supports genuine conversational exchange rather than one-directional translation alone.
The web-based platform is built using Django and integrates MediaPipe, TensorFlow, OpenCV, and Coqui TTS for accurate gesture recognition and realistic speech synthesis. MediaPipe and OpenCV handle real-time hand tracking and image processing, TensorFlow powers the underlying gesture classification model, and Coqui TTS enables high-quality, personalized voice generation, together forming a cohesive pipeline from visual gesture input to spoken audio output.
This solution enhances accessibility, inclusivity, and real-time communication for individuals with hearing and speech disabilities, offering a practical technological bridge that reduces reliance on third-party interpretation and empowers more independent, natural interaction in everyday settings.