ISBN: 978-981-18-7950-0 DOI: 10.18178/wcse.2023.06.031
From Words to Flight: Integrating OpenAI ChatGPT with PX4/Gazebo for Natural Language-Based Drone Control
Abstract—The rapid progress of large language models in recent years has opened new opportunities for human-robot interaction. In this paper, we propose a novel approach to control drones using natural language commands by integrating OpenAI ChatGPT with the PX4/Gazebo simulator. The proposed system enables users to interact with the simulator using everyday language, allowing them to control the drone's actions with ease, eliminating the need for extensive training in drone piloting. We discuss the implementation details, including the validation of the ChatGPT-generated commands and their translation into executable actions in the simulator. To the best of our knowledge, this is the first proposal of a verification and validation system for commands generated by ChatGPT and LLMs in general. Furthermore, we discuss the crafting of effective prompts and the essential criteria for doing so. Our approach demonstrates promising results in terms of both usability and reliability, paving the way for further research on natural languagebased control systems for robotics applications.
Index Terms—LLM, ChatGPT, PX4, Gazebo, UAV, AI-assisted control, prompt engineering, Commands validation
Mohamed Lamine Tazir, Matei Mancas, Thierry Dutoit
ISIALAB, Numediart Institute, University of Mons, BELGIUM
Cite: Mohamed Lamine Tazir, Matei Mancas, Thierry Dutoit, "From Words to Flight: Integrating OpenAI ChatGPT with PX4/Gazebo for Natural Language-Based Drone Control" Proceedings of 2023 the 13th International Workshop on Computer Science and Engineering (WCSE 2023), pp. 215-222, June 16-18, 2023.