robotlearningblock

Welcome to the project repository for the Internet-connected task board, an electronic real-world benchmarking platform for evaluating robot manipulation skills. This project, known as the Digital Robotic Judge, or DR.J, contains the complete design of competition task boards used in the Robothon Grand Challenge and the euROBIN Manipulation Skill Versatility Challenge to remotely assess manipulation performances over the Internet. Trial attempts are recorded by an on-board microcontroller which verifies the execution time and successful task completion through performance circuits. Task board telemetry is automatically published on a web dashboard. The best automated robot solutions are collected, ranked, and verified with an expert jury over a video conference call. The top copmetition results are posted on a leaderboard and developers are encouraged to share their project code in the form of competition submissions so that others can reproduce and improve on their results.

Industrial use cases collected through a survey are incorporated into the design of portable electronic task boards and made available to the community of robot developers to solve and put forth their best automated solution. New task board designs are developed annually. Share your use case to be considered in upcoming designs with the Use Case Survey Google Form.

If you find this work helpful, please cite our IEEE-RAM paper.

The concept of a crowdsourced robot solution for real-world use cases with the DR.J pipeline is shown below.

Quick Links

• IEEE-RAM Paper Digital Robot Judge: DOI: 10.1109/MRA.2023.3336473
• 1-min Paper Highlight Video
• ERF 2022 Hackathon: Franka Emika Tactile Manipulation Benchmark Challenge: Highlight Video
• Robothon Grand Challenge Competition automatica website
• IROS 2024 Competiton: euROBIN Manipulation Skill Versatility Challenge
• Links to Web Dashboards for Deployed Task Boards:
  • TBv2023: Teams Single Board View
  • TBv2023: Subtask Completion Times
  • TBv2021: Teams Single Board View
• Task Board Telmetry Emulator in Python (browser-based) Replit
• Simulated Task Board Model on Mujoco for Offline Skill Development Multiverse Github Project
• Request a physical task board Task Board Request Google Form
• Propose new tasks for future board designs Use Case Survey Google Form

Project Description

The goal of the project is to provide an objective tool for measuring manipulation performance for both humans and robots. A microcontroller on the task box monitors the state of the mounted manipulation elements and reports state change events to a cloud-hosted dashboard to aggregate performance data. Device telemetry, including accelerometer data, is regularly logged to provide insight into the device’s usage. Manipulation tasks are designed with objects that can be electronically verified to be in their designated start and goal positions. Trial protocols define sequences of actions on the task board and allow designers to create test scenarios. The execution time of subtasks within a protocol are recorded and automatically published on a web dashboard for consistent and convenient comparison with previous trial attempts. A modular framework for adding new task board objects was selected to encourage community members to design and build their own trial protocols.

An example demonstration of the internet-connected task board can be viewed on YouTube. Here is the very first test run with a Franka Emika Panda robot in the lab YouTube.

A screenshot of a web dashboard from the 2021 Robothon Grand Challenge competition.

Just for fun: The name is borrowed from the legendary basketball star Julius “DR.J” Irving.

A little history on the smart task board development…

The internet-connected task board was first featured during the internationl robot manipulation competition, Robothon Grand Challenge at automatica in 2021. The design of the task board has been expanded and reimplemented for the competition again in 2022 and has also been picked up by the pan-European robotics excellence collaboration project, euROBIN.

Here is an example setup of a exemplary robot testing setup with the task board.

The electronic task board allows for the direct comparison between a human and a robot performance to quantitatively measure the gap.

Here is the trial protocol for the task board version TBv2021 used in the Robothon Grand Challenge in 2021 and 2022 to guide roboticists to develop reusable manipulation skills for e-waste sorting and disassembly.

A new task board design (TBv2023) was designed and built to develop manipulation skills in performing electronic device assessments with a multimeter. The design was also adapted to meed the needs of the euROBIN network and was used in the Robothon Grand Challenge in 2023.

As part of the competition, teams were challenged to demonstrate the manipulation skills of their robot platform on an e-waste object of their choice in the “Bring-Your-Own-Device (BYOD) Challenge”. Below are some examples of the successful demonstrations.

Quick Start to connect a Task Board to the Web Dashboard

The microcontroller (orange component) has 3 Buttons and a USB-C charge/programming port:

• Power Button (side closest to USB-C),
• Button A (next to screen, labeled M5),
• Button B (side furthest from USB-C).

Turning on the Task Board

1. Toggle Power Manually = Hold Power Button for 2 seconds to turn on and 6 seconds to turn off.
2. Plugging in the uController will automatically turn on the device.

Connect board to the web dashboard over the internet with your preferred WiFi Network using your laptop or smartphone

1. Connect to Task Board SSID: “AutoConnectAP-task-board-###” where ### refers to the number on the sticker on the side of the box (Password: “robothon”)

2. A configuration screen should automatically open in a browser on your device. Otherwise, open web browser and go to URL: “192.168.4.1”.

3. Click the ”Configure” button to setup a new WiFi network connection.

4. Select Desired WiFi from the scanned list.

5. Enter your SSID’s WiFi password.

6. Click the “Save” Button. The new credentials will be saved to the microcontrollers EEPROM memory for future connections. The Task Board will attempt to connect to new WiFi. After it has successfully connected to the Internet over the new network the Home Screen will be displayed on the screen.

Begin running experiments with the task board

1. Ensure the task board components are on their starting position prior to starting. The uController will display an alert on the screen if it detects an incorrect starting configuration.
2. Press Button A to start recording a trial run! Trial runs will be automatically recorded to the web dashboard. Scan the QR code on the sticker for more information.
3. Reset Trial Clock after a trialèPress Button B to reset the clock to zero.

Some Other Helpful Tips:

• Offline/Local Mode: Use your task board without the internet by holding Button A down when powering on the device.

File truncated at 100 lines see the full file