|
|
Posted by Okachinepa on 04/09/2025 @
Courtesy of SynEvol
Credit: Florida Atlantic University
For countless deaf and hard-of-hearing people globally, communication obstacles can render daily interactions difficult. Conventional methods, such as sign language interpreters, are frequently limited, costly, and reliant on human presence. In a progressively digital environment, the need for intelligent, supportive technologies that provide immediate, precise, and easy-to-use communication options is on the rise, seeking to close this essential gap.
American Sign Language (ASL) is among the most commonly utilized sign languages, featuring unique hand movements that signify letters, words, and expressions. Current ASL recognition systems frequently face challenges with real-time efficiency, precision, and reliability in various environments.
A significant obstacle in ASL systems is differentiating between gestures that look alike, like "A" and "T" or "M" and "N," frequently resulting in misclassifications. Moreover, the quality of the dataset poses considerable challenges, such as low image resolution, motion blur, uneven lighting, and differences in hand sizes, skin tones, and backgrounds. These elements create bias and limit the model's capacity to generalize across various users and settings.
To address these issues, scholars from the College of Engineering and Computer Science at Florida Atlantic University have created a groundbreaking real-time ASL interpretation system. By merging YOLOv11's object detection capabilities with MediaPipe's accurate hand tracking, the system is able to effectively identify ASL alphabet letters instantaneously. By employing sophisticated deep learning and critical hand point tracking, it converts ASL gestures into text, allowing users to interactively spell names, places, and more with exceptional precision.
Essentially, a built-in webcam functions as a touchless sensor, gathering real-time visual information that is transformed into digital frames for the purpose of gesture analysis. MediaPipe detects 21 keypoints on each hand to form a skeletal framework, whereas YOLOv11 utilizes these points to accurately identify and classify ASL letters.
"What distinguishes this system is that the whole recognition process—from gesture capture to classification—functions smoothly in real-time, irrespective of different lighting conditions or backgrounds," stated Bader Alsharif, the lead author and a Ph.D. candidate in the FAU Department of Electrical Engineering and Computer Science.
"All of this is accomplished with typical, readily available hardware." This highlights the system's practical capability as an easily accessible and scalable assistive technology, rendering it a suitable option for real-world use.
Findings from the research, published in the journal Sensors, validate the system's efficiency, which attained a 98.2% accuracy (mean Average Precision, mAP@0.5) while maintaining low latency. This discovery emphasizes the system's capacity to provide high precision instantly, rendering it an optimal choice for applications that demand quick and dependable performance, including live video processing and interactive technologies.
The ASL Alphabet Hand Gesture Dataset, containing 130,000 images, features diverse hand gestures recorded in various conditions to aid models in achieving better generalization. These conditions encompass a variety of lighting scenarios (bright, dim, and shadowed), different backgrounds (including outdoor and indoor settings), and multiple hand angles and orientations to guarantee durability.
Every image is meticulously labeled with 21 keypoints, showcasing crucial hand features like fingertips, knuckles, and the wrist. These annotations create a basic outline of the hand, enabling models to differentiate between similar gestures with remarkable precision.
Courtesy of SynEvol
Credit: Florida Atlantic University
"This initiative exemplifies how advanced AI can be utilized for the benefit of humanity," stated Imad Mahgoub, Ph.D., co-author and Tecore Professor in the FAU Department of Electrical Engineering and Computer Science.
"By combining deep learning with hand landmark recognition, our team developed a system that not only delivers excellent accuracy but also stays user-friendly and suitable for daily applications." It's a significant advance in the direction of inclusive communication technologies.
In the U.S., around 11 million individuals are deaf, representing about 3.6% of the population, while roughly 15% of American adults (37.5 million) face challenges with hearing.
"The importance of this study is in its ability to change communication for the deaf community by offering an AI-based tool that converts American Sign Language signs into text, facilitating better interactions in education, workplaces, healthcare, and social environments," stated Mohammad Ilyas, Ph.D., co-author and professor in the FAU Department of Electrical Engineering and Computer Science.
"Through the creation of a strong and user-friendly ASL interpretation system, our research aids in the progress of assistive technologies to eliminate obstacles for the deaf and hard of hearing community."
Courtesy of SynEvol
Credit: Florida Atlantic University
Future efforts will aim at enhancing the system's functions to progress from identifying single ASL letters to understanding complete ASL sentences. This would facilitate more natural and smooth communication, enabling users to express complete thoughts and phrases effortlessly.
"This study emphasizes the transformative potential of AI-powered assistive technologies in uplifting the deaf community," stated Stella Batalama, Ph.D., dean of the College of Engineering and Computer Science. "By connecting the communication divide via real-time ASL recognition, this system is crucial in promoting a more inclusive society."
"It enables people with hearing difficulties to connect more smoothly with their surroundings, whether they are presenting themselves, finding their way, or just participating in regular discussions." This technology improves accessibility while also fostering increased social integration, contributing to the development of a more connected and compassionate community for all.
|
|
