BTGenBot

This work presents a novel approach to generating behavior trees for robots using lightweight large language models (LLMs) with a maximum of 7 billion parameters. The study demonstrates that it is possible to achieve satisfying results with compact LLMs when fine-tuned on a specific dataset. The key contributions of this research include the creation of a finetuning dataset based on existing behavior trees using GPT-3.5 and a comprehensive comparison of multiple LLMs (namely llama2, llama-chat, and code-llama) across nine distinct tasks. To be thorough, we evaluated the generated behavior trees using static syntactical analysis, a validation system, a simulated environment, and a real robot. Furthermore, this work opens the possibility of deploying such solutions directly on the robot, enhancing its practical applicability. Findings from this study demonstrate the potential of LLMs with a limited number of parameters in generating effective and efficient robot behaviors.

Release for the paper BTGenBot: Behavior Tree Generation for Robotic Tasks with Lightweight LLMs, published in 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

Preprint available on arxiv: https://arxiv.org/abs/2403.12761
Paper: https://ieeexplore.ieee.org/document/10802304

Dataset, llama-2-7b-chat and codellama-7b-instruct LoRA adapters available on HuggingFace.

Authors: Riccardo Andrea Izzo, Gianluca Bardaro and Matteo Matteucci
Location: AIRLab (The Artificial Intelligence and Robotics Lab of Politecnico di Milano)

Overview

• bt_client: client designed to interpret and execute LLM-generated behavior trees directly on robot
• bt_generator: demo notebook to load our fine-tuned models for generating behavior trees
• bt_validator: validator that assess the overall correctness of the LLM-generated trees
• dataset: our instruction-following dataset used to fine-tune the models
• lora_adapters: LoRA adapters for the base models, used in the notebook to load the fine-tuned version
• prompt: outcomes of prompts run on both LlamaChat and CodeLlama models, both in zero-shot and one-shot scenarios

Setup

bt_generator

Create a conda environment (or equivalent virtualenv):

conda create -n btgenbot python==3.10

Install dependencies:

pip install -r requirements.txt

bt_client/bt_validator

Create a colcon workspace and clone this repository in your ROS2 workspace
Build:

colcon build

Required ROS2 dependencies:

• BehaviorTree.CPP: available here
• BehaviorTree.ROS2: available here
• igus_rebel_commander: available here, required only by bt_client for the task involving arucos and arm activity
• aruco_interfaces: available here, required only by bt_client for the task involving arucos and arm activity

Tested on a Linux computer with Ubuntu 22.04 and ROS2 Humble

bt_client

Client usage

• Select the task and the corresponding behavior tree in XML format from the ones available in /bt_xml
• Modify config/tree.yaml configuration file with the file name in the tree_name field:

tree_name: "main_tree.xml"

To add a new behavior tree, follow these steps:

1. Create an XML file representing the behavior tree
2. Place the XML file in the /bt_xml folder
3. Specify the name of the XML file in the config/tree.yaml configuration file

• Build and source the package

colcon build
source install/setup.bash

• Launch the client and execute the selected behavior tree

ros2 launch bt_client bt.launch.py

Node functionalities

Keep in mind that the system offers a range of pre-defined node functionalities. For instance, the “MoveTo” action facilitates the sending of a navigation goal to the Nav2 server, utilizing the goal specified within the behavior tree XML.

Moreover, locations for testing purposes are outlined in the location.yaml configuration file. These locations are pre-defined and serve as references for navigation tasks.

It is possible to add further actions with

factory.registerNodeType<ACTION_NAME>("ACTION_NAME");

File truncated at 100 lines see the full file