Journal of Intelligent Communication

Article

Multi UAV Cooperative Reconnaissance based on Dynamic Programming VDN Algorithm

VDN algorithm strategy decision for UAV 3D trajectory in environment

Downloads

https://doi.org/10.54963/jic.v4i1.238
Huang, J., Yang, Z., Li, J., Wu, S., Zhang, X., & Li, B. (2024). Multi UAV Cooperative Reconnaissance based on Dynamic Programming VDN Algorithm. Journal of Intelligent Communication, 4(1), 44–62. https://doi.org/10.54963/jic.v4i1.238
Volume 4 Issue 1 (2024): In Progress
Copyright (c) 2024 Jingyi Huang, Ziyi Yang, Jiarui Li, Shuying Wu, Xinyu Zhang, Bo Li
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright

The authors shall retain the copyright of their work but allow the Publisher to publish, copy, distribute, and convey the work.

License

Journal of Intelligent Communication (JIC)  publishes accepted manuscripts under Creative Commons Attribution 4.0 International (CC BY 4.0). Authors who submit their papers for publication by Journal of Intelligent Communication (JIC) agree to have the CC BY 4.0 license applied to their work, and that anyone is allowed to reuse the article or part of it free of charge for any purpose, including commercial use. As long as the author and original source is properly cited, anyone may copy, redistribute, reuse and transform the content.

Authors

  • Jingyi Huang School of Electronics and Information, Northwestern Polytechnical University, Xi'an 710072, China
  • Ziyi Yang School of Electronics and Information, Northwestern Polytechnical University, Xi'an 710072, China
  • Jiarui Li School of Electronics and Information, Northwestern Polytechnical University, Xi'an 710072, China
  • Shuying Wu School of Electronics and Information, Northwestern Polytechnical University, Xi'an 710072, China
  • Xinyu Zhang School of Electronics and Information, Northwestern Polytechnical University, Xi'an 710072, China
  • Bo Li
    School of Electronics and Information, Northwestern Polytechnical University, Xi'an 710072, China

This paper proposes a multi agent value decomposition network (VDN) based multi UAV collaborative reconnaissance and control method to address the issue of insufficient strategies for multi UAV collaborative reconnaissance and control. By designing corresponding algorithm networks and training processes, the goal of autonomy, collaboration, and intelligence among multiple unmanned aerial vehicle systems has been achieved, assisting unmanned aerial vehicle combat forces in achieving collaborative operations and decision-making. This article uses AirSim as the simulation verification environment to verify the effectiveness of the proposed algorithm. The experimental results show that the algorithm proposed in this paper can achieve multi UAV collaborative reconnaissance tasks in complex environments, providing an intelligent solution for UAV collaborative control.

Keywords:

reinforcement learning dynamic programming UAV collaborative reconnaissance VDN algorithm
(This article belongs to the Topical Collection "Intelligent Decision and Control of Unmanned Systems".)

References

  1. Li, B.; Yang, Z.Y.; Jia, Z.R.; Ma, H. A. Unsupervised Learning Neural Network for UAV Regional Reconnaissance Path Planning. J. Northwestern Polytechnical Univ. 2021, 39, 77–84.
  2. Zhou, X.Q. The Development Status and Trends of Foreign Electronic Warfare Drones. Ship Electron. Countermeasures 2003, 26, 6–9+19.
  3. Chen, Z.J.; Wei, J.Z.; Wang, Y.X.; Zhou, R. UAV Autonomous Control Levels and System Structure. Acta Aeronautica et Astronautica Sinica 2011, 32, 1075–1083.
  4. Cao, J.H.; Gao, X.G. Intelligent Command and Control System for Multi UAV Collaborative Operations. Firepower Command Control. 2003, 28, 4.
  5. Chen, H.; Wang, X.M.; Jiao, Y.S.; Li, Y.A. UAV Coverage Trajectory Planning Algorithm for Convex Polygonal Regions. J. Aeronaut. 2010, 31, 1802–1808.
  6. Fu, X.W.; Wei, G.W.; Gao, X.G. Multi UAV Collaborative Area Search Algorithm in Uncertain Environments. Syst. Eng. Electron. Technol. 2016, 38, 821–827.
  7. Wang, D.; Zhang, G.Z.; Mu, W.D. Multi UAV collaborative combat communication self-organizing network technology. Aviat. Missile 2012, 1, 59–63.
  8. Merino, L.; Caballero, F.; Ferruz, J.; Wiklund, J.; Forssén, P.; Ollero, A. Multi UAV Cooperative Perception Techniques. In Multiple Heterogeneous Unmanned Aerial Vehicles; Ollero, A., Maza, I., Eds.; Springer Tracts in Advanced Robotics: Berlin, Germany; Volume 37, pp. 67–110.
  9. Hu, P.L.; Zhao, C.H.; Hu, J.W. Reject Collaborative Perception and Autonomous Control of Unmanned Aerial Vehicle Clusters in the Environment. In Proceedings of the 40th China Control Conference, Shanghai, China, 26–28 July 2021.
  10. Li, C.C. Collaborative Perception and Visualization of Three-Dimensional Complex Environments. Master Dissertation, Xi'an University of Electronic Science and Technology, Xi’an, China, 1 May 2020.
  11. Merino, L.; Caballero, F.; Dios, J.R.M. A Cooperative Perception System for Multiple UAVs: Application to Automatic Detection of Forest Fires. J. Field Rob. 2006, 23, 165–184.
  12. Zhong, S.B.; Zhu, W.; Peng, L.; Huang, X.B. Research on the Key Technology System of Collaborative Perception. CN Emerg. Manag. 2021, 12, 52–55.
  13. Chen, Y. Research on Planning and Simulation of Collaborative Reconnaissance Tasks for Drone Clusters. Master Dissertation, Nanjing University of Aeronautics and Astronautics, Nanjing, China, 1 March 2021.
  14. Rasmussen, S.J.; Shima, T. Branch and Bound Tree Search for Assigning Cooperative UAVs to Multiple Tasks. In Proceedings of the 2006 American Control Conference, Minneapolis, MN, USA, 14–16 June 2006.
  15. Azam, Md A.; Shankarachary, R. Decentralized Formation Shape Control of UAV Swarm Using Dynamic Programming. Signal Process., Sens./Inf. Fusion Target Recogn. XXIX 2020, 11423, 69–76.
  16. Pang, Q.W.; Li, W.G.; Li, Y.K.; Hu, Y.J.; Jia, H.X. Multi UAV Collaborative Reconnaissance Trajectory Planning Based on Improved Genetic Algorithm. CN J. Inertial Technol. 2020, 28, 248–255.
  17. Kang, X.C.; He, G.J.; Chen, F.; Li. X.G. A Discrete Firefly Algorithm for Solving the Task Allocation Problem of Unmanned Aerial Vehicle ISR. J. Missile Guidance 2019, 39, 131–134+138.
  18. Lin, J.C.; Jia, G.W.; Hou, Z.X. Research on Task Assignment of Heterogeneous UAV Formation in the Anti-radar Combat. Sys. Eng. Electron. 2018, 40, 1986–1992.
  19. Tian, Z.; Wang, X.F. Cooperative Multiple Task Assignment for Heterogeneous Multi-UAVs with Multi-Chromosome Genetic Algorithm. Flight Dyn. 2020, 9, 687.
  20. Maza, I.; Ollero, A. Multiple UAV Cooperative Searching Operation Using Polygon Area Decomposition and Efficient Coverage Algorithms. In Distributed Autonomous Robotic Systems 6, 1st ed.; Alami, R., Chatila, R., Asama, H., Eds.; Springer Tokyo: Tokyo, Japan, 2007; Volume 1, pp. 221–230.
  21. Agarwal, A.; Hiot, L.M.; Nghia, N.T. Parallel Region Coverage Using Multiple UAVs. In Proceedings of the 2006 IEEE Aerospace Conference, Big Sky, MT, USA, 4–11 March 2006.
  22. Yao, Y.; Li, Q.; Chen, X. Optimization of the Application of A* Algorithm in Path Planning. Microelectron. Comput. 2017, 34, 51–55.
  23. Chen, J.Y.; Hu, K.K.; Li, Y.W. Research on UAV Multi-point Navigation Algorithm Based on MBRRT*. Comput. Sci. 2018, 45, 85–90.
  24. Pehlivanoglu, Y.V. A New Vibrational Genetic Algorithm Enhanced with a Voronoi Diagram for Path Planning of Autonomous UAV. Aerosp. Sci. Technol. 2012, 16, 47–55.
  25. Wang, C.; Dong, H.L.; Gu, X.S. Improved Particle Swarm Optimization Algorithm and its Application Path Planning. Control Eng. CN 2019, 26, 1466–1471.
  26. Zhou, Q.; Zhang, R.; Suo, X.J. Genetic Algorithm for UAV Trajectory Planning with Timing Constraints. Aeronautical Comput. Tech. 2016, 46, 93–96.
  27. Li X.G.; Cai, Y.L. Unmanned Aerial Vehicle Path Planning Based on Improved Ant Colony Algorithm. Flight Mech. 2017, 35, 52–56.
  28. Li, Y.Q. Route Planning for Multi UAV Collaborative Area Surveillance Based on Genetic Algorithm and Deep Reinforcement Learning. Master Dissertation, Xi'an University of Electronic Science and Technology, Xi'an, China, 1 June 2018.
  29. Xu, T.H. Research on Deep Reinforcement Learning Method for Autonomous Collaborative Reconnaissance of Drone Clusters. Master Dissertation, National University of Defense Technology, Changsha, China, 1 October 2019.
  30. Zhang, F.Z.; Zhu, Y. Task Allocation Method for Collaborative Reconnaissance of Multiple Unmanned Aerial Vehicles in Complex Environments. J. Sys. Simul. 2022, 34, 2293–2302.
  31. Li, B.; Huang, J.Y.; Wan, K.F.; Song, C. A Review of Research on the Application of UAV System Based on Deep Reinforcement Learning. Tacti-cal Missile Technol. 2023, 1, 58–68.
  32. Zhao, Y.; Guo, J.F.; Zheng, H.X.; Bai, C.C. A Reinforcement Learning Based Collision Avoidance Computational Guidance Method for Multiple Unmanned Aerial Vehicles. Navigation Positioning Timing 2021, 8, 31–40.
  33. Fan, L.T. Research on Multi UAV Collaborative Task Planning Algorithm Based on Reinforcement Learning. Master Dissertation, Henan University of Science and Technology, Luoyang, China, 1 May 2019.
  34. Value-decomposition Networks for Cooperative Multi-Agent Learning. Available online: https://arxiv.org/abs/1706.05296 (accessed on 1 March 2024).
  35. Zhou, Y. Research on 3D Obstacle Avoidance Algorithm for Unmanned Aerial Vehicles Based on Airsim Simulation Platform. Master Dissertation, University of Electronic Science and Technology, Sichuan, China, 15 March 2020.