Major Project · Web Development

Indian Sign Language Recognition and Personalized Voice Cloning System Project

This project is an intelligent communication assistance system that converts Indian Sign Language (ISL) gestures into text and speech in real time. The system uses computer vision and deep learning to recognize hand gestures through …

Difficulty Level

Advanced

Core Modules

6 Modules

Project Overview

Communication barriers often create challenges for people with hearing and speech impairments. This project proposes an AI-based Sign Language Recognition and Voice Cloning System that enables seamless interaction between sign language users and non-sign language users. The system captures hand gestures through a webcam and identifies sign language alphabets using a trained deep learning model. The recognized gestures are converted into text and then transformed into speech using advanced voice cloning technology, generating audio in the user's personalized voice. Furthermore, the application includes text-to-sign conversion, allowing normal text to be represented as sign language images. The web-based platform is built using Django and integrates MediaPipe, TensorFlow, OpenCV, and Coqui TTS for accurate gesture recognition and realistic speech synthesis. This solution enhances accessibility, inclusivity, and real-time communication for individuals with hearing and speech disabilities.

Module Breakdown 6 Modules

User Management Module

Handles user registration, login, logout, profile management, audio sample upload, and password recovery through OTP verification.

Sign Language Recognition Module

Captures live webcam video and detects hand landmarks using MediaPipe. The trained deep learning model predicts sign language alphabets from hand gestures.

Text and Sentence Generation Module

Converts recognized sign gestures into letters, words, and complete sentences using prediction buffering and gesture stabilization techniques.

Personalized Voice Cloning Module

Generates speech from recognized text using the user's uploaded voice sample, producing natural and personalized audio output.

Text-to-Sign Translation Module

Converts user-entered text into corresponding sign language images for learning and communication purposes.

Real-Time Communication & Monitoring Module

Provides live video streaming, AJAX-based updates, audio retrieval, and communication controls for smooth user interaction.