Identificação visual em tempo instantâneo da datilologia em Libras: facilitador na comunicação

Abstract

This work proposes a system to facilitate communication between deaf and hearing individuals through the automatic translation of the brazilian sign language (Libras) fingerspelling into text. Using Computer Vision and Deep Learning techniques, the system is capable of recognizing the Libras alphabet from real-time images of human hands. To achieve this goal, the PointNet neural network architecture, specialized in processing three-dimensional data, was employed. A dataset containing images of hands in different positions and angles, representing the Libras alphabet, was used to train the network. The MediaPipe library was crucial for extracting 3D landmarks from the hands in the images, which were used as input for the neural network. After training, the PointNet network demonstrated high accuracy in letter classification, with over 98% accuracy. To enhance user experience, the system was integrated with a spell checker, ensuring greater fluency and naturalness in the translation. Additionally, real-time capture allows for direct user interaction with the system. The results obtained in this work demonstrate the potential of applying Computer Vision and Deep Learning techniques to develop sign language interpretation systems, contributing to the inclusion and communication of deaf individuals