Precision landing with reinforcement learning

Deep reinforcement learning for drone precision landing, docker container for simulation in Gazebo-ROS2 dashing with PX4-Autopilot controller.

Project developed, starting from a problem of DRAFT PoliTO, as part of the course “AI in Industry” of the University of Bologna.

Table of contents

1. Setup
2. Usage
   1. Train
   2. Test
   3. Code changes
   4. Speed-up
3. Torch to ONNX
4. Acknowledgments
5. References

Setup

Install Docker.

Modify the absolute paths in “run_docker.sh” to reflect the position of the repository on your computer.

Add your user to the “docker” group

sudo groupadd docker
sudo gpasswd -a $USER docker
newgrp docker  # Or log out/in

Build and start the docker, it will take same time, from the root of the repository run:

• For ROS2 Dashing:

   ./run_docker.sh build  # if an error occurs, delete the docker and run again this command with "sudo"
   

• For ROS2 Foxy:

   ./run_docker.sh build_foxy  # if an error occurs, delete the docker and run again this command with "sudo"
   

Don’t worry if this message appears “bash: /src/shared/ros_packages/install/setup.bash: No such file or directory”, you just need to follow the rest of the setup and the following times it won’t happen again.

Once it has finished, in the docker run the following commands, in order to build the packages:

cd /src/shared/ros_packages
colcon build
source install/setup.bash

Usage

To start the docker run:

• For ROS2 Dashing:

   ./run_docker.sh run
   

• For ROS2 Foxy:

   ./run_docker.sh run_foxy
   

Train

Open 2 terminals and run the docker in each one of them, as explained above.

In the first one run:

cd /src/shared
./launch_train_ddpg.sh

In the second one run:

ros2 run px4_ros_extended gazebo_runner.py --train

Parameters

In “shared/ros_packages/px4_ros_extended/src_py/params.yaml” you can set a few parameters, all of them but “train_window_reward” and “test_window_reward”, that are used by the jupyter notebook, are used by the agent.

Pay particular attention to the model name, the target dimension and the observation and action shape, as the test script will need to know these values.

Among these parameters area also those that let you change the way in which the target moves, such as its maximum and minimum velocity or the type of movement, i.e. “linear” or “circular”, and the area in which the agent can move.

Plots and Models

In the folder “shared/logs” are saved the rewards in pickle files, you can have a look at the jupyter notebook “shared/Log Analysis.ipynb” to use them and print a few plots.

In the folder “shared/models” are saved the actor and critic models, the files with “best” in the name contain the best weights found during evaluation; the number at the start of the name represents the unique-id of the session.

Test

Open 2 terminals and run the docker in each one of them, as explained above.

In the first one run:

ros2 run px4_ros_extended gazebo_runner.py --test --headless

File truncated at 100 lines see the full file