English | [简体中文](README_zh-CN.md)
RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-World
📰 Release
- [2025/01/08] 🔥 We release data collection method.
- [2024/12/04] 🔥 We release the RoboMatrix supervised fine-tuning (SFT) dataset containing 1,500 high-quality human-annotated demonstration videos.
Demos
Dynamic Adversarial Interaction
https://private-user-images.githubusercontent.com/35285052/392642975-b78e28aa-45c2-4bb0-9e70-b6a08c678f85.mp4
Hardware Preparation
We use robots from DJI’s RoboMaster series as the hardware platform, including the Engineering Robot (EP) and
the Warrior Robot (S1). These two forms of robots share some common components, including the mobile chassis, monocular RGB camera, audio module, and controller. Additionally, each robot is equipped with a unique set of components to perform specific tasks, such as the target shooting capability of the S1 robot and the target grasping capability of the EP robot.
We modified the EP robot by mounting the camera above the robot to prevent the camera’s viewpoint from changing with the movement of the robotic arm. See 3D_Printing for the parts of the designed camera mount.
We use BEITONG ASURA 2PRO+ GAMEPAD NEARLINK VERSION as the controller for robot teleoperation.
Environment Configuration
We developed RoboMatrix using the ROS2 framework on Ubuntu 20.04. You can follow the official installation guidance to complete the installation of the Foxy distro of ROS2 and the necessary tools. In addition, we passed the test on Ubuntu 22.04 (ROS2 Humble), which may provide some reference for you if you want to install RoboMatrix in a later version of Ubuntu.
We provide a general installation procedure for ROS2, this might give you some help. If you already have it installed on your system, please skip this step.
ROS2 Installation
### Step 1: Set UTF-8
Open a terminal, check weather your system supports UTF-8.
```bash
locale
```
If not support (no output in terminal), please install.
```bash
sudo locale-gen en_US en_US.UTF-8
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
export LANG=en_US.UTF-8
```
### Step 2: Set Ubuntu Universe
Open a terminal, check weather your system supports Ubuntu Universe.
```bash
apt-cache policy | grep universe
```
If not support (no output in terminal), please install.
```bash
sudo apt install software-properties-common
sudo add-apt-repository universe
```
### Step 3: Add ROS2 software source and key
```bash
sudo apt update && sudo apt install curl gnupg2 lsb-release
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null
```
### Step 4: Get ROS2
Install the specified version of ROS2, using **Foxy** as an example.
```bash
sudo apt update
sudo apt install ros-foxy-desktop
```
### Step 5: Source bash
```bash
echo "source /opt/ros/foxy/setup.bash" >> ~/.bashrc
source .bashrc
```
### Step 6: Test demo
Open a terminal, start talker node.
```bash
ros2 run demo_nodes_cpp talker
```
Open a new terminal, start listener node.
```bash
File truncated at 100 lines [see the full file](https://github.com/waynemao/robomatrix/tree/main/README.md)
ROS Index
