Versatile modular neural locomotion control with fast learning

Code for M. Thor et al., Versatile modular neural locomotion control with fast learning, 2021, submitted to NMI.

Drawing inspiration from animal locomotion, we propose a simple yet versatile modular neural control structure with fast learning. The key advantages of our approach are that behavior-specific control modules can be added incrementally to obtain increasingly complex emergent locomotion behaviors, and that neural connections interfacing with existing modules can be quickly and automatically learned.

Content:

• System Requirements
• Code overview
• Install
• Run the controller with learned weights
• Run learning algorithm
• License

System Requirements

This code has been tested with the following hardware and software:

• Intel® Core™ i9-9900K CPU @ 3.60GHz × 16
• GeForce RTX 2080
• Ubuntu 18.04.4 LTS
• coppeliaSim V4.1.0
• Vortex physics engine¹
• Python 3.7.7
• ROS melodic

¹ The Vortex physics engine requires a license (which is free for researchers). Alternatively, you can use the Newton physics engine, but for that, we cannot guarantee successful behaviors. Especially, the wall and pipe climbing behaviors does not work well with the Newton physics engine. We hypothesize that this is because the behavior was learned with the Vortex physics engine and because the behavior requires the higher complexity of that engine. To test the remaining behaviors an modified version of the advanced environment called Advanced_newton_env.ttt has been made.

Code overview

The following explains the content of the six main directories:

• data: Contains data from running the simulation and PI^BB algorithm (both data used in the result section (in the storage directory) as well as job files used during learning). Additionally, it contains the learned weight sets (as .json files) for the base controller and eight behavior-specific modules presented in the paper. For a more detailed explaination see the README.md file in the data directory.
• interfaces: Contains .lua files for interfacing with and setting up the simulation. It also contains the build_dir for cmake.
• machine_learning: Contains all the code necessary for the PI^BB learning algorithm as well as bash scripts for running the simulation.
• neural_controllers: Contains all the code nesseary for the locomotion controller (i.e., the CPG-RBF network).
• simulations: Contains coppeliaSim simulation environments (including the MORF hexapod robot).
• utils: Contains additional utilities needed by the controller implementation and simulation interface.

Install

Install time will take 15-30 minutes.

First, we need to set up the simulation (coppeliaSim):

1. Download coppeliaSim EDU from the downloads page (tested on V4.1.0 / Ubuntu 18.04).
2. Clone this repository to your local machine (optional: set $FRAMEWORK_PATH to the path for the directory containing the cloned repository).

git clone https://github.com/MathiasThor/CPG-RBFN-framework.git

1. Extract the downloaded .zip file into a directory as many times as you need “simulation workers” (i.e., the number of simulations running in parallel. We used four workers in all our experiments).
2. Rename the extracted coppeliaSim directories as: VREP1, VREP2, VREP3, VREP4, etc. (optional: set $VREP_WORKER_PATH to the path for the directory containing the workers).
3. In remoteApiConnections.txt in each of the VREP# directories, change portIndex1_port so that VREP1 has 19997, VREP2 has 19996, VREP3 has 19995, VREP4 has 19994, etc.
4. Copy libv_repExtRosInterface.so into each of the worker directories from the utils directory.

cp $FRAMEWORK_PATH/CPG-RBFN-framework/utils/libv_repExtRosInterface.so $VREP_WORKER_PATH/VREP1/

1. Install the required python libraries (matplotlib, jupyter, drawnow, and numpy).

cd $FRAMEWORK_PATH/CPG-RBFN-framework/

sudo apt install python3-pip

pip3 install -r requirements.txt

1. Install LIB GSL.

sudo apt-get install libgsl-dev

The neural controllers use ROS to communicate with coppeliaSim. Therefore, make sure that ros-xxx-desktop-full (tested on melodic) is installed (ROS install guide)

Run the controller with learned weights

1. Start an ROS core.

roscore

1. Start the simulation workers. In this example, we will only need a single worker.

cd $VREP_WORKER_PATH/VREP1/

./coppeliaSim.sh $FRAMEWORK_PATH/CPG-RBFN-framework/simulations/Advanced_env.ttt

1. Build the locomotion controller.

cd $FRAMEWORK_PATH/CPG-RBFN-framework/interfaces/morf/sim/build_dir

rm CMakeCache.txt

cmake .

make

1. Run with the learned weights presented in the paper for 400s.

File truncated at 100 lines see the full file