ACADSTAFF UGM
| Title | : | Gesture Recognition in Indonesian Sign Language Using Hybrid Deep Learning Models |
| Author | : |
MUHAMMAD YUSUF DAFFA (1) Wahyono, Ph.D. (2) |
| Date | : | 9 2023 |
| Keyword | : | Sign Language Recognition,BISINDO,Deep Learning,CNN,LSTM,Hybrid Model Sign Language Recognition,BISINDO,Deep Learning,CNN,LSTM,Hybrid Model |
| Abstract | : | Abstract—Sign language comprises unique hand gestures used to communicate between the hearing and hearing-impaired communities, with variations based on regions or countries. Due to the increasing global prevalence of hearing loss, there is a growing need for efficient sign language recognition systems. This paper presents the development of a hybrid CNN-LSTM model specifically designed to recognize static alphabetical gestures in Indonesian Sign Language (BISINDO). Primary data was collected to address the issue of publicly available dataset. These self-collected images underwent augmentation and preprocessing stages prior to model training. The architecture and hyperparameter configuration of the hybrid model were fine-tuned using the Randomized Search CV method. Performance analysis was conducted on both the hybrid model, its constituent models, and state-of-the-art model. The experimental results revealed remarkable performance by the hybrid model with excellent accuracies of 99.60%, 84.87%, and 98.00% on the training, validation, and testing sets respectively, along with a macro average score of 0.98, indicating high precision, recall, and F1-score across all the classes. However, it was observed that the VGG-16 and CNN exhibited slightly superior performance compared to the hybrid model, while the LSTM demonstrated the least favorable performance among the models. |
| Group of Knowledge | : | |
| Level | : | Internasional |
| Status | : |
Published
|
| No | Title | Action |
|---|