Journal of Intelligent Communication

Article

Developing an Intelligent Recognition System for British Sign Language: A Step Towards Inclusive Communication

Downloads

https://doi.org/10.54963/jic.v4i1.351
Isiaq, S. O., & Stephen, S. (2025). Developing an Intelligent Recognition System for British Sign Language: A Step Towards Inclusive Communication. Journal of Intelligent Communication, 4(1), 42–57. https://doi.org/10.54963/jic.v4i1.351
Volume 4 Issue 1 (2025): In Progress
Copyright (c) 2025 Sakirulai Olufemi Isiaq, Shany Stephen
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright

The authors shall retain the copyright of their work but allow the Publisher to publish, copy, distribute, and convey the work.

License

Journal of Intelligent Communication (JIC)  publishes accepted manuscripts under Creative Commons Attribution 4.0 International (CC BY 4.0). Authors who submit their papers for publication by Journal of Intelligent Communication (JIC) agree to have the CC BY 4.0 license applied to their work, and that anyone is allowed to reuse the article or part of it free of charge for any purpose, including commercial use. As long as the author and original source is properly cited, anyone may copy, redistribute, reuse and transform the content.

Authors

  • Sakirulai Olufemi Isiaq
    Creative Computing Institute, University of Arts London, London WC1V 7EY, UK https://orcid.org/0000-0003-3622-7680
  • Shany Stephen Department of Computing, Solent University, Southampton SO14 0YN, UK

Effective communication is crucial for ensuring inclusivity, yet the hard-of-hearing community faces significant barriers due to a shortage of qualified interpreters. British Sign Language (BSL), officially recognised in the UK, relies on hand gestures, facial expressions, and body movements. However, limited interpreter availability necessitates technological solutions to bridge the communication gap between signers and non-signers. This study proposes a real-time, vision-based BSL recognition system using computer vision and deep learning to interpret fingerspelling and six commonly used BSL words. The system employs OpenCV for video capture, MediaPipe for hand feature extraction, and Long Short-Term Memory (LSTM) networks for sign classification. A dataset incorporating left- and right-handed signers achieved a 94.23% accuracy rate for 26 fingerspelling gestures and 99.07% for six words. To enhance usability, a graphical user interface was developed, enabling seamless real-time interaction. These findings demonstrate the potential of AI-driven sign language recognition to improve accessibility and foster more inclusive communication for the hard-of-hearing community.

Keywords:

British Sign Language; Realtime Recognition; Media Pipe; Hard‑of‑Hearing; LSTM; OpenCV

References

  1. Van Cleve, J.V.; Crouch, B.A. A Place of Their Own: Creating the Deaf Community in America. Gallaudet University Press: Washington, DC, USA, 1989.
  2. NRCPD. Registered Sign Language Interpreters in the UK. Available from: https://www.nrcpd.org.uk/registration-figures (accessed on 4 January 2025).
  3. Cui, R.; Liu, H.; Zhang, C. Recurrent convolutional neural networks for continuous sign language recognition by staged optimization. In Proceedings of The 2017 IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21–26 July 2017.
  4. Park, H.; Lee, Y.; Ko, J. Enabling real-time sign language translation on mobile platforms with on-board depth cameras. Proc. ACM. Interact. Mob. Wearable Ubiquitous Technol. 2021, 5, 1–30.
  5. Patel, P.; Patel, N. Vision Based Real-time Recognition of Hand Gestures for Indian Sign Language using Histogram of Oriented Gradients Features. Int. J. Next-Gener Comput. 2019, 10, 92–102.
  6. Tanvir, M.; Alam, M.S.; Saha, D.K.; et al. Real-Time Recognition of Bangla Sign Language Characters: A Computer Vision Based Approach Using Convolutional Neural Network. In Proceedings of The 2021 3rd International Conference on Electrical & Electronic Engineering (ICEEE), Rajshahi, Bangladesh, 22–24 December 2021.
  7. Liwicki, S.; Everingham, M. Automatic recognition of fingerspelled words in british sign language. In Proceedings of The 2009 IEEE computer society conference on computer vision and pattern recognition workshops, Miami, FL, USA, 20–25 June 2009.
  8. Dardas, N.H.; Georganas, N.D. Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques. IEEE Trans. Instrum. Meas. 2011, 60, 3592–3607. DOI: https://doi.org/10.1109/TIM.2011.2161140
  9. Wadhawan, A., Kumar, P. Deep learning-based sign language recognition system for static signs. Neural. Comput. appli. 2020, 32, 7957–7968.
  10. Rambhau, P.P. Recognition of two hand gestures of word in British sign language (BSL). Int. J. Sci. Res. Publi. 2013, 3, 1–5.
  11. Bird, J. J.; Ekárt, A.; Faria, D. R. British Sign Language recognition via late fusion of computer vision and Leap Motion with transfer learning to American Sign Language. Sensors 2020, 20, 5151. DOI: https://doi.org/10.3390/s20185151
  12. Buckley, N.; Sherrett, L.; Secco, E.L. A CNN sign language recognition system with single & double-handed gestures. In Proceedings of The 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC), Madrid, Spain, 12–16 July 2021.
  13. Hameed, H., et al. Privacy-preserving British Sign Language recognition using deep learning. In Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, Scotland, United Kingdom, 11–15 July 2022.
  14. Adão, T.; Oliveira, J.; Shahrabadi, S.; et al. Empowering deaf-hearing communication: Exploring Synergies between predictive and generative AI-based strategies towards (Portuguese) sign language interpretation. J. Imaging 2023, 9, 235.
  15. IBM. What is artificial intelligence (AI)? Available Online: https://www.ibm.com/cloud/learn/what-is-artificial-intelligence (accessed on 4 February 2025).
  16. Al-Ahdal, M.E.; Nooritawati, M.T. Review in sign language recognition systems. In Proceedings of The 2012 IEEE Symposium on Computers & Informatics (ISCI), Penang, Malaysia, 18–20 March 2012.
  17. Google. MediaPipe: Holistic. Available Online: https://mediapipe.dev (accessed on 10 August 2024).
  18. Halder, A.; Tayade, A. Real-time vernacular sign language recognition using mediapipe and machine learning. Int. J. Res. Publi. Rev. 2021, 2, 9–17.
  19. Duy Khuat, B.; Thai Phung, D.; Thi Thu Pham, H.; et al. Vietnamese sign language detection using Mediapipe. In Proceedings of The 2021 10th International Conference on Software and Computer Applications, Kuala Lumpur, Malaysia; 23–26 February 2021.
  20. Alvin, A.; Shabrina, N.H. Ryo, A.; Christian, E. Hand gesture detection for sign language using neural network with mediapipe. Ultima Comput. J. Sist. Komp. 2021, 13, 57–62
  21. Adhikary, A.. Real-time sign language recognition using MediaPipe and deep learning. Procedia Comput. Sci. 2023, 218, 1384–1393.
  22. Subramanian, B.; Olimov, B.; Naik, S. M.; et al. An integrated mediapipe-optimized GRU model for Indian sign language recognition. Sci. Rep. 2022, 12, 11964. DOI: https://doi.org/10.1038/s41598-022-15998-7
  23. Google. Pose detection. In ML Kit - Google for Developers. Retrieved from: https://developers.google.com/ml-kit/vision/pose-detection (Accessed on 4 March 2025)
  24. Google AI Edge. Hand Landmarks Detection Guide. Available online: https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker (accessed on 2 August 2025).
  25. Bradski, G.; Kaehler, A. Learning OpenCV: Computer Vision with the OpenCV Library, 1st ed. O'Reilly Media: Publisher location, USA, 2008; pp. 102–105.
  26. Fingerspelling Alphabet. Available online: https://www.british-sign.co.uk/fingerspelling-alphabet-charts/ (accessed on 4 July 2022).
  27. Han, J.; Kamber, M.; Pei, J. Data Mining: Concepts and Techniques, 3rd ed. Morgan Kaufmann: Publisher location, USA, 2011; pp. 83–120.
  28. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning. MIT Press: Cambridge, MA, USA, 2016; p. 163.
  29. Chollet, F. Deep Learning with Python, 2nd ed. Manning Publications: Shelter Island, NY, USA, 2021.
  30. Yang, S., Yu, X., Zhou, Y. Lstm and gru neural network performance comparison study: Taking yelp review dataset as an example. In Proceedings of The 2020 International workshop on electronic communication and artificial intelligence (IWECAI), Shanghai, China, 12–14 June 2020.
  31. Sharma, S.; Sharma, S.; Athaiya, A. Activation functions in neural networks. Towards Data Sci. 2017, 6, 310–316.
  32. Vijay, K.; Krithiga, P.; Kavirakesh, S.; et al. Early Detection of Diabetic Retinopathy Using Deep Convoulutional Neural Network. In Artificial Intelligence and Machine Learning in Big Data Processing; Geetha, R., Dao, N.N., Khalid, S., Eds.; Springer Nature: Cham, Switzerland, 2023; pp. 315–327.
  33. He, H.; Ma, Y. Imbalanced Learning: Foundations, Algorithms and Applications, 1st ed. John Wiley & Sons: Hoboken, NJ, USA, 2013; pp. 43–59.
  34. Adhikary, S.; Talukdar, A. K.; Sarma, K. K. A vision-based system for recognition of words used in indian sign language using mediapipe. In Proceedings of The 2021 sixth international conference on image information processing (ICIIP), Shimla, India, 26--28 November 2021.