Unofficial Python or ROS2 library for Manus Meta Gloves

While Manus Meta natively supports C++ SDK, there is no Python or ROS2 bindings or retargeting code to robot hands such as LEAP Hand.

This repo has tools which we use with the LEAP Hand in many projects and demos including Bimanual Dexterity for Complex Tasks Please see Bidex for more videos.

The code is similar to the released C++ SDK by Manus, but adds ZMQ bindings to get the useful glove data in ROS2 and then a Pybullet based SDLS retargeter which converts this data to LEAP Hands.

This is tested on our Quantum Mocap Metagloves on SDK 2.4 but should work on other Manus gloves.

Manus SDK Setup

This sends the useful Manus data from the SDK to Python/ROS2 nodes, specifically RawSkeletonInfo is the fingertip data from MANUS and Ergonomics data is the approximate human skeleton joint angle data estimated by MANUS.

Pick EITHER Ubuntu Standalone (Only an Ubuntu machine required), or with Windows as the Core gloves server and then connect to it with a second Ubuntu client machine.

Ubuntu only is more useful for robotics, but Windows is better supported by Manus Core.

You need a license from Manus to use the SDK to use this software.

Option 1: Ubuntu Standalone

• Install cppzmq. (This is a dependency for our version of the Manus SDK) I recommend this guide from cppzmq:

    sudo apt-get install libzmq3-dev
    download and unzip the lib, cd to directory
    mkdir build
    cd build
    cmake ..
    sudo make -j4 install
    

• Follow Manus’ instructions and install “Core Integrated” dependencies without Docker.
• Instead of compiling the default sdk you download from them, compile our version of the SDK but use the same instructions. (Note -lzmq has been added to the makefile that we provide) Make (Option 1) works well.

Now to run our version of the MANUS SDK:

• Run the C++ (not minimal client) SDK with ./SDKClient_Linux.out
• Pick standalone 1 Core Integrated by pressing 1.
• The data should stream successfully via ZMQ.
• Run the Python or the ROS2 script on the Linux machine. See those sections to setup Python or ROS2.
• Check the python/ROS2 ZMQ address is tcp://127.0.0.1:8000 (This is default) whenever you run using just one computer.

Option 2: Windows Manus Core -> Ubuntu

• Install Manus Core 2.4 and setup the gloves there
• Install the requirements from the Manus SDK normally
• Download this repo and open the solution file in Visual Studio.
• Install cppzmq. On windows I recommend this guide from cppzmq:
  • Build cppzmq via vcpkg. This does an out of source build and installs the build files

```git clone https://github.com/Microsoft/vcpkg.git cd vcpkg ./bootstrap-vcpkg.sh # bootstrap-vcpkg.bat for Powershell ./vcpkg integrate install ./vcpkg install cppzmq

```

• Set the IP address of the Windows machine in the python/ROS SDK. I recommend you put them on the same local network and set them to two different static IPs.
  • Connect ethernet cable between the two computers
  • On the Windows machine and Ubuntu machines set the IP as the below images.
  • Make sure the Python/ROS2 file matches the IP address you select for the Windows machine and the IPs on each machine are different.

  

• NOTE, currently Windows has a different joint ordering than Linux, please see read_and_send_zmq.py for details on how to switch between.

Now to run the SDK:

• Compile and run in Visual Studio or run the .exe that is generated (not minimal client). Select 2 for data to come local from Core. Otherwise make sure Manus Core is closed and then select 1 to run as SDK standalone.
• Run the Python or the ROS2 script on the Linux machine. See the next section to setup Python or ROS2.

Python Setup

• This directly prints the MANUS Glove data out on a Python file as an example.
• Install Python zmq by doing pip install pyzmq
• The glove serial numbers in the Python file should match your gloves. (Number is on Manus Core or printed by the SDK)
• To run, launch our MANUS SDK first and then run python minimal_example.py
• If successful the data should be printing to the terminal.

  Python + LEAP Hand

• This demo integrates MANUS gloves with the LEAP Hand Python SDK. Check the LEAP Hand for more details.
• First install LEAP Hand:
  • pip install dynamixel_sdk numpy
  • Copy the leap_hand_utils folder from the LEAP Hand Python SDK next to the LEAP_hand_example.py in this repository so that it can be imported correctly.
• Run our MANUS SDK, then connect and power your LEAP Hand and then run: python LEAP_Hand_example.py
• If successful, the LEAP Hand will come to life similar to our conference demo as seen on the bottom of https://leaphand.com/
• Note that this does not match the pinch grasps between the two hands but instead copies joint angles directly from the MANUS skeleton to LEAP Hand for simplicity. The thumb will never be that good using this mode due to the differing kinematics of the two hands.

Manus has now released an official version of the ROS2 library on their website!

ROS2 + LEAP Hand Retargeting Setup

It is useful to retarget the glove data to robot hands to perform similar fingertip grasping between the two hands.

• Inspired by Robotic Telekinesis and Dexcap, this code retargets and solves for robot joint angles that matches the pinch grasps between the human and robot hands.
• This retargeter takes the Manus Glove fingertip data, runs SDLS from Pybullet and then returns joint angles for the robot.
• We provide examples using LEAP Hand, but you can also import your own URDF/MJCF into Pybullet.
• All of these nodes are in Python so they are easy to use.

ROS2 Setup

• If not already installed, follow ROS2 Installation Instructions to install ROS2.
• The code we provide is two ROS2 Node/Packages:
  • Glove gets the glove data
  • Telekinesis performs the inverse kinematics

File truncated at 100 lines see the full file