Mobile Robot DRL Navigation

A ROS2 framework for DRL autonomous navigation on mobile robots with LiDAR.

Table of contents

• Introduction
• Installation
  • Docker Installation (recommended)
  • Manual Installation
• Training
  • Loading a Stored Model
  • Optional Configuration
  • Utilities
• Physical Robot
• Troubleshooting

Introduction

This repository contains a ROS2 and PyTorch framework for developing and experimenting with deep reinforcement learning for autonomous navigation on mobile robots. Models are trained in simulation and evaluated either in simulation or on a real-world robot. The robot model used in this repository is based on the turtlebot3. However, the framework can be used for any robot model that can provide LiDAR and odometry information and work with linear velocity messages.

Below are a few examples of what the current framework can be used for:

• Train, store, load, and evaluate a navigation agent in simulation in different environments
• Deploy an existing model on a real robot to perform navigation and obstacle avoidance
• Evaluate the effect of different hyperparameters on training time and performance
• Experiment with additional capabilities (backward motion, frame stacking)
• Implement your own DRL algorithm (currently includes: DQN, DDPG, TD3)

Installation

Docker Installation (Recommended)

In order to greatly simplify the installation process and get up and running quickly it is recommended to use Docker. Docker can be seen as a lightweight VM that allows you to run applications within an isolated container making it easy to install all of the dependencies.

First, install docker

Now, in order to use your GPU within the docker container to run the machine learning models, we need to complete a few extra simple steps. You should already have the nvidia driver installed on your system.

Nvidia Container Toolkit

The next thing we need to do is install the nvidia container toolkit which is a piece of software that will allow us to use our GPU within the docker container. The installation steps are listed below.

First, setup the package repository and the GPG key:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

Then install the container toolkit:

sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit

Configure the Docker daemon to recognize the NVIDIA Container Runtime:

sudo nvidia-ctk runtime configure --runtime=docker

And lastly, restart the Docker daemon to complete the installation after setting the default runtime:

sudo systemctl restart docker

At this point, a working setup can be tested by running a base CUDA container:

sudo docker run --rm --runtime=nvidia --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi

Build and run container

Now build the container specified in the Dockerfile, which may take 10-20 minutes:

docker build -t turtlebot3_drlnav .

Lastly, we need to give our docker container permission to run GUIs:

xhost +local:docker

Now that everything is set up, start the container:
(Replace /PATH/TO/REPO with the path where you downloaded this repository on your machine)

docker run -it --gpus all --privileged   --env NVIDIA_VISIBLE_DEVICES=all   --env NVIDIA_DRIVER_CAPABILITIES=all  --env DISPLAY=${DISPLAY}  --env QT_X11_NO_MITSHM=1  --volume /tmp/.X11-unix:/tmp/.X11-unix -v /PATH/TO/REPO/turtlebot3_drlnav:/home/turtlebot3_drlnav   --network host turtlebot3_drlnav

And that’s it! we don’t need to install any other dependencies thanks to docker.

While inside the container first build our application:

File truncated at 100 lines see the full file