Overview

This interface enables a user to remotely teleoperate a Stretch robot through a web browser. This website can be set up to teleoperate the robot remotely from anywhere in the world with an internet connection, or simply eyes-off teleop from the next room on a local network. The codebase is built on ROS2, WebRTC, Nav2, and TypeScript.

Setup & Installation

The interface is compatible with the Stretch RE1, RE2 and SE3. It currently only supports Ubuntu 22.04 and ROS2 Humble. Upgrade your operating system if necessary (instructions) and create/update the Stretch ROS2 Humble workspace (instructions). This will install all package dependencies and install the web teleop interface.

Audio configuration

We provide a convenience script ./configure_audio.sh to configure your audio devices. Depending on desired audio configuration, you may need to pass in speaker and mic names (see the comment in the script itself for usage details). Below, we detail additional commands that could help configure your audio.

You may need to re-run audio configuration every time you re-start your Stretch.

Audio Devices

For robot-to-operator audio streaming, the web interfaces uses the robot’s system default microphone if one or more external microphones are plugged in, or the built-in robot microphone if no external microphones are plugged in.

For operator-to-robot text-to-speech, the web interface uses the system default speaker.

However, note the system defaults can change when you (un)plug audio devices (e.g., sometimes (un)plugging a mic can cause the system default speaker to change). Thus, it is best practice to always check.

In the below instructions, replace <sink/source> with sink for a speaker and source for a microphone. Note that this won’t work if you’re using X-11 forwarding:

1. List all speakers/microphones: pactl list short <sink/source>s
2. Get the default: pactl get-default-<sink/source>
3. Set the default: pactl set-default-<sink/source> <device-name> where <device-name> is one of the names listed from the above command.
   1. The built-in default speaker will be called something like alsa_output.pci-0000_00_1f.3.analog-stereo, and the built-in default mic will be called something like alsa_input.usb-SEEED_ReSpeaker_4_Mic_Array__UAC1.0_-00.multichannel-input.

Audio volume/gain

1. To unmute the speaker: amixer set 'Master unmute
2. To set the speaker volume to 100%: amixer sset 'Master' 100% (Note that there is also a physical volume knob on the head of the Stretch3 that must be turned up.)
3. To unmute the mic: amixer set 'Capture' cap
4. To set the mic gain to 100%: amixer sset 'Capture' 100%

Installing Beta Teleop Cameras

To install the Beta teleop cameras, plug one camera in and run the following command:

REx_camera_set_symlink.py --list

You should see an output similar to:

For use with S T R E T C H (R) from Hello Robot Inc.
---------------------------------------------------------------------

Found the following Video Devices:


Intel(R) RealSense(TM) Depth Ca (usb-0000:00:0d.0-1):
Ports: ['/dev/video0', '/dev/video1', '/dev/video2', '/dev/video3', '/dev/video4', '/dev/video5', '/dev/media0', '/dev/media1']


USB CAMERA: USB CAMERA (usb-0000:00:14.0-1.1.1):
Ports: ['/dev/video6', '/dev/video7', '/dev/media2']

Note, it is important to make sure the cameras are not plugged in at the same time because they will have the same name: USB CAMERA and you will not be able to differentiate between the two. Next, set the camera symlink by running the following command:

REx_camera_set_symlink.py --port <PORT> --symlink <SYMLINK_NAME>

Replace <PORT> with the 0th element in the ports list for the USB CAMERA outputted by REx_camera_set_symlink.py --list command. In the example above, that would be /dev/video6. Replace <SYMLINK_NAME> with hello-navigation-camera or hello-gripper-camera for the navigation and gripper camera respectively. For example, if we were setting up the navigation camera the command would look similar to:

REx_camera_set_symlink.py --port /dev/video6 --symlink hello-navigation-camera

Repeat this process for both cameras, then run:

ll /dev/hello-*

and verify the symlinks are setup correctly.

Launching the Interface

First, navigate to the folder containing the codebase using:

colcon_cd stretch_web_teleop

Next, launch the interface:

./launch_interface

If you’d like to launch the interface with a map run:

./launch_interface -m maps/<NAME_OF_MAP>.yaml

File truncated at 100 lines see the full file