Penggunaan Metode Deep Learning untuk Pengembangan Sistem Komunikasi Cerdas bagi Penyandang Disabilitas

NORMA NINGSIH; AFIFAH DWI RAMADHANI; DJOKO SANTOSO; BINTANG DESTA RAMADHANI; ILYASA AYASY EL GHOFIQI

doi:10.26760/mindjournal.v9i2.206-219

Penggunaan Metode Deep Learning untuk Pengembangan Sistem Komunikasi Cerdas bagi Penyandang Disabilitas

NORMA NINGSIH, AFIFAH DWI RAMADHANI, DJOKO SANTOSO, BINTANG DESTA RAMADHANI, ILYASA AYASY EL GHOFIQI

Sari

ABSTRAK

Komunikasi merupakan kebutuhan mendasar bagi mahluk hidup agar dapat berinteraksi dengan lingkungan sekitar. Di dunia, orang-orang disabilitas khususnya tuna rungu dan sulit mendengar sebagian besar berkomunikasi menggunakan bahasa isyarat Penelitian ini mengembangkan sistem klasifikasi Bahasa Isyarat Indonesia (SIBI) menggunakan model Convolutional Neural Network (CNN) VGG16 dan VGG19 yang diintegrasikan dengan aplikasi berbasis web. Sistem ini dirancang untuk membantu komunikasi dengan penyandang disabilitas melalui klasifikasi gerakan tangan secara real-time menggunakan gambar atau webcam. Hasil pengujian menunjukkan bahwa model mencapai akurasi sebesar 96,4% dengan nilai loss 0,1055, menunjukkan performa yang stabil dan generalisasi yang baik tanpa indikasi overfitting. Evaluasi menggunakan confusion matrix menunjukkan distribusi prediksi yang akurat pada 24 kelas isyarat tangan, dengan precision, recall, dan f1-score yang tinggi untuk setiap kelas. Sistem ini diharapkan dapat menjadi alat bantu komunikasi yang efektif bagi penyandang disabilitas dalam kehidupan sehari-hari.

Kata kunci: Deep Learning, VGG16, Klasifikasi, SIBI, Disabilitas

ABSTRACT

Communication is a basic need for living things to interact with their environment. In the world, people with disabilities, especially deaf and hard of hearing, mostly communicate using sign language. This study develops an Indonesian Sign Language (SIBI) classification system using the VGG16 and VGG19 Convolutional Neural Network (CNN) models integrated with a web-based application. This system is designed to assist communication with people with disabilities through real-time hand gesture classification using images or webcams. The test results show that the model achieves an accuracy of 96.4% with a loss value of 0.1055, indicating stable performance and good generalization without any indication of overfitting. A confusion matrix evaluation shows an accurate prediction distribution across 24 hand gesture classes, with high precision, recall, and f1-score for each class. This system is expected to be an effective communication tool for people with disabilities in everyday life.

Keywords: Deep Learning, VGG16, Classification, SIBI, Disability

Teks Lengkap:

PDF

Referensi

Alrowais, F., Marzouk, R., Al-Wesabi, F. N., & Hilal, A. M. (2023). Hand Gesture Recognition for Disabled People Using Bayesian Optimization with Transfer Learning. Intelligent Automation and Soft Computing, 36(3), 3325–3342. https://doi.org/10.32604/iasc.2023.036354

Barto, Andrew. G., & Sutton, Richard. S. (2015). Reinforcement learning: An introduction (2nd ed.). The MIT Press.

Bintang, Al. (2022). Sistem Isyarat Bahasa Indonesia (SIBI). Https://Www.Kaggle.Com/Datasets/Alvinbintang/Sibi-Dataset.

Chung, H.-Y., Chung, Y.-L., & Tsai, W.-F. (2019). An efficient hand gesture recognition system based on deep CNN. IEEE International Conference on Industrial Technology (ICIT), 853–858.

Dwi Nurhayati, O., Eridani, D., & Hafiz Tsalavin, M. (2022). SISTEM ISYARAT BAHASA INDONESIA (SIBI) METODE CONVOLUTIONAL NEURAL NETWORK SEQUENTIAL SECARA REAL TIME A REAL-TIME INDONESIAN LANGUAGE SIGN SYSTEM USING THE CONVOLUTION NEURAL NETWORK METHOD. Jurnal Teknologi Informasi Dan Ilmu Komputer (JTIIK), 9(4), 819–828. https://doi.org/10.25126/jtiik.202294787

El-Din, S. A. E., & El-Ghany, M. A. A. (2020). Sign Language Interpreter System: An alternative system for machine learning. Proceedings of the 2nd Novel Intel- Ligent and Leading Emerging Sciences Conference, 332–337.

Izzah, A., & Suciati, N. (2014). Translation of Sign Language Using Generic Fourier Descriptor and Nearest Neighbour.

International Journal on Cybernetics & Informatics, 3(1), 31–41. https://doi.org/10.5121/ijci.2014.3104

Latif, A., Apriani, E., Syahwildan, M., & Purnamasari, P. (2023). Development of Artificial Intelligence (AI): A Bibliometric Analysis Approach. Jurnal Info Sains, 13(3), 1031–1043. http://ejournal.seaninstitute.or.id/index.php/InfoSains

Obi, Y., Claudio, K. S., Budiman, V. M., Achmad, S., & Kurniawan, A. (2022). Sign language recognition system for communicating to people with disabilities. Procedia Computer Science, 216, 13–20. https://doi.org/10.1016/j.procs.2022.12.106

Soekarta, R., Aras, S., & Ahmad Nur Aswad. (2023). Hyperparameter Optimization of CNN Classifier for Music Genre Classification. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 7(5), 1205–1210. https://doi.org/10.29207/resti.v7i5.5319

Weiss, K., Khoshgoftaar, T. M., & Wang, D. D. (2016). A survey of transfer learning. Journal of Big Data, 3(1). https://doi.org/10.1186/s40537-016-0043-6

WHO. (2024, February 2). Deafness and hearing loss. Https://Www.Who.Int/News-Room/Fact-Sheets/Detail/Deafness-and-Hearing-Loss.

Wijiyanto, W., Pradana, A. I., Sopingi, S., & Atina, V. (2024). Teknik K-Fold Cross Validation untuk Mengevaluasi Kinerja Mahasiswa. Jurnal Algoritma, 21(1). https://doi.org/10.33364/algoritma/v.21-1.1618

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? In International Journal of Educational Technology in Higher Education (Vol. 16, Issue 1). Springer Netherlands. https://doi.org/10.1186/s41239-019-0171-0

DOI: https://doi.org/10.26760/mindjournal.v9i2.206-219