Two Way Sign Language Translate
December 2020
Inspiration
There are 466 million persons in the world with disabling hearing loss (6.1% of the world’s population) , I have created this app for them and their family so that they can communicate easily. This will allow a person with hearing loss and person who doesn’t knows Sign language to communicate to each other easily. We were also inspired from Hana’s ASL workshop.
What it does
It can do two things:
- Take Voice from User and Convert it to Sign Language in form of a GIF
- It will Take Video from User and convert the Sign Language in Video to Voice using Machine Learning
How we built it
We divided the project into two parts
VOICE/TEXT TO SIGN LANGUAGE CONVERSION:
- Scraping Data from Giphy using Chrome Extension
- Then filtered the gif files and added names to it
- Also added gif files of single alphabets
- Took Voice/Text input from user and split into words and checked if it is present in the GIF filenames. If it is not present then use the Alphabet GIFs for making up words
- Finally Displayed it onto Tkinter App
SIGN LANGUAGE TO VOICE/TEXT CONVERSION:
- Used the ASL Dataset on Kaggle of Alphabets
- Created a CNN algorithm in Tensorflow and trained the model for small data
- Used Live Webcam feed of user hand and predicted the Alphabet from a Region of Interest
- Finally Displayed it onto Tkinter App
Challenges we ran into
1.Tried using Scrapy and Selenium to scrape the Gif files but were unsuccessful
- Creating GUI on Tkinter was hard as we were using it for first time
Accomplishments that we’re proud of
- Successfully created a Multifeatured Desktop App
- Got Accuracy of 95% in predicting Sign Language