Journal of Intelligent Communication

Article

Interactive Conversational AI with IoT Devices for Enhanced Human-Robot Interaction

Downloads

https://doi.org/10.54963/jic.v3i1.317
Secco, E. L. (2025). Interactive Conversational AI with IoT Devices for Enhanced Human-Robot Interaction. Journal of Intelligent Communication, 3(1). https://doi.org/10.54963/jic.v3i1.317
Volume 3 Issue 1: March 2024
Copyright (c) 2024 Emanuele Lindo Secco
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright

The authors shall retain the copyright of their work but allow the Publisher to publish, copy, distribute, and convey the work.

License

Journal of Intelligent Communication (JIC)  publishes accepted manuscripts under Creative Commons Attribution 4.0 International (CC BY 4.0). Authors who submit their papers for publication by Journal of Intelligent Communication (JIC) agree to have the CC BY 4.0 license applied to their work, and that anyone is allowed to reuse the article or part of it free of charge for any purpose, including commercial use. As long as the author and original source is properly cited, anyone may copy, redistribute, reuse and transform the content.

Authors

Significance - The rapid advancements in conversational AI and IoT technologies have opened up new possibilities for human-machine interaction. Despite the progress, a gap exists in integrating these two fields to create more centralized, intuitive, and engaging user experiences. Current integrations typically consist of specialised hardware-software pairs that do not fully leverage the capabilities of advanced conversational models, thereby limiting their applicability. This research proposes a general solution to bridge the capabilities of various IoT devices with the oversight and control abilities of AI language models, enhancing the potential for more versatile and natural IoT-AI-human interactions.

Aim and Approach - This research presents the design and development of an IoT system operated by an AI language model and conversationally managed by humans to operate robots. Based on this setup, the initial goal is to create a framework for interactively controlling a robotic arm. The approach involves using a Raspberry Pi as a central control system and ChatGPT API to manage conversations and execute given commands.

Results - The developed IoT-AI system demonstrated efficient and reliable human-robot interaction. It effectively captures user voice inputs, processes them through advanced AI models, and generates appropriate commands for the robotic arm, achieving an average voice-to-motion latency of 5.5 seconds. While some latency and voice recognition challenges exist, the overall performance confirms the viability of using conversational AI for natural and intuitive robotic control.

Conclusions - This research successfully integrates conversational AI with IoT devices, resulting in a more user-centric and efficient human-robot interaction. The system highlights the significant potential of precisely translating natural language commands into robotic actions, enhancing user experience and operational efficiency.

Keywords:

Artificial Intelligence; Intelligent Communication; Conversational AI; IoT; Human Machine Interaction; Human Robot Interaction keyword.

References

  1. Improving Language Understanding by Generative Pre-Training. Available Online: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf (Accessed 21 January 2025).
  2. Language Models are Few-Shot Learners. Available Online: https://arxiv.org/pdf/2005.14165 (Accessed 21 January 2025).
  3. Gubbi, J.; Buyya, R.; Marusic, S; Palaniswami, M. Internet of Things (IoT): A vision, architectural elements, and future directions. Future Gener. Comp. Syst. 2013, 29, 1645–1660
  4. Al-Fuqaha, A.; Guizani, M.; Mohammadi, M.; et al. Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications. IEEE Commun. Surv. Tutorials 2015, 17, 2347–2376.
  5. Piyare, R. & Tazil, M., 2011. Bluetooth Based home automation system using cell phone. In Proceedings of the 2011 IEEE 15th International Symposium on Consumer Electronics, Singapore, 14–17 June 2011. DOI: http://dx.doi.org/10.1109/ISCE.2011.5973811.
  6. Becerik-Gerber, B. et al. The field of human building interaction for convergent research and innovation for intelligent built environments. Sci Rep 2022, 12, 22092.
  7. Masreliez, L. THE FRAGMENTED SMART HOME: A comprehensive analysis of available interoperable solutions to connect wireless smart home communications. Available Online: https://www.diva-portal.org/smash/get/diva2:1584387/FULLTEXT01.pdf (Accessed 21 January 2025).
  8. Smart Talk: How organizations and consumers are embracing voice and chat assistants. Available online: https://www.capgemini.com/wp-content/uploads/2019/09/Report-%E2%80%93-Conversational-Interfaces_Web-Final.pdf (Accessed 21 January 2025)..
  9. Z Isherwood, EL Secco, A Raspberry Pi computer vision system for self-driving cars, Computing Conference, 2, 910-924, 2022, DOI: 10.1007/978-3-031-10464-0
  10. GPT-3.5 Turbo fine-tuning and API updates. Available Online: https://openai.com/index/gpt-3-5-turbo-fine-tuning-and-api-updates/ (Accessed 21 January 2025).
  11. Liu, D. & Cao, J., 2022. Determinants of Collaborative Robots Innovation Adoption in Small and Medium-Sized Enterprises: An Empirical Study in China. Appl. Sci. 2022, 12, 10085
  12. Guo, X., Shen, Z., Zhang, Y. & Wu, T. Review on the Application of Artificial Intelligence in Smart Homes. Smart Cities 2019, 2, 402–420.
  13. Thakare, V.; Khire, G.; Kumbhar, M. Artificial Intelligence (AI) and Internet of Things (IoT) in Healthcare: Opportunities and Challenges. ECS Trans 2022. 107, 7941.
  14. Roller, S.; et al. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Online,19 to 23 April 2021.
  15. Secco, E.L.; et al. Human-like Robotic Hands for Biomedical Applications and Beyond. Front. Robot. AI 2024, 11, 1414971.
  16. Bell, D.; Secco, E.L. Design of a 3D-Printed Accessible and Affordable Robotic Arm and a User-Friendly Graphical User Interface. In Fourth Congress on Intelligent Systems; Kumar, S., K., B., Kim, J.H., Bansal, J.C., Eds.; Springer: Singapore, 2023; vol 868, 195–205
  17. Innes, M.; Secco, E.L. An Understanding of How Technology Can Assist in the Epidemic of Medicine Nonadherence with the Development of a Medicine Dispenser. Eur. J. Appl. Sci. 2023, 11, 522–550
  18. Hutton, J. Secco, E.L. Development of an interface for real time control of a dexterous robotic hand, using MYO muscle sensor, Acta Sci. Comp. Sci. 2023, 5.3: 25-36 .
  19. Latif, B.; Buckley, N.; Secco, E.L. Hand Gesture & Human-Drone Interaction. In Intelligent Systems and Applications; Arai, K. Eds.;Springer:Cham, country, 2022; 544, 299-308.
  20. Manolescu, D.; Mutinda, B.; Secco, E.L. Human–robot interaction via wearable device - A wireless glove system for remote control of 7-DoF robotic arm, Acad. Eng. 2024, 1(3)
  21. Mowlai, M.; Mahdavimanshadi, M.; Sayyadzadeh, I. Adapting Transformer-Based Multi-Style Networks for Human Pose Prediction with a Custom Data Pipeline in Industrial Human-Robot Collaboration. In Proceedings of the 2024 Systems and Information Engineering Design Symposium, Charlottesville, VA, USA, 03 May 2024.
  22. Secco, E.L.; McHugh, D.D.; Buckley, N. A CNN-based Computer Vision Interface for Prosthetics’ Control. In Wireless Mobile Communication and Healthcare; Gao, X., Jamalipour, A., Guo, L., Eds.; Springer: Cham, 2022; 440, pp. 41–59.
  23. Chu, T.S.; Chua, A.Y.; Secco, E.L. A Study on Neuro Fuzzy Algorithm Implementation on BCI-UAV Control Systems. ASEAN Eng. J. 2022, 12, 75–81.