GStreamer ROS 2 Babel Fish

This project provides GStreamer elements for bridging ROS 2 image topics with GStreamer pipelines. It allows for high-performance streaming of image data between ROS 2 and GStreamer, supporting both raw and compressed image formats.

For a detailed understanding of the system’s design and component interactions, please refer to the Architecture Overview.

Features

• rbfimagesrc: GStreamer source element that subscribes to ROS 2 image topics.
  • Supports sensor_msgs/msg/Image (raw) and sensor_msgs/msg/CompressedImage.
  • Automatically handles format conversion.
  • Configurable QoS settings.
• rbfimagesink: GStreamer sink element that publishes to ROS 2 image topics.
  • Automatically determines whether to publish sensor_msgs/msg/Image (raw) or sensor_msgs/msg/CompressedImage (e.g., JPEG, PNG) based on the input caps.
  • Zero-copy publishing where possible (using shared memory / loaned messages if supported).

[!NOTE] Both elements use GstReferenceTimestampMeta with timestamp/x-unix to attach and recover the timestamp of the ROS image, enabling the processing of image buffers without losing time information (as long as all processing elements preserve it).

Installation

This package is a ROS 2 package. You can build it using colcon.

1. Create a ROS 2 workspace (if you haven’t already):

    mkdir -p ~/ros2_ws/src
    cd ~/ros2_ws/src
    

1. Clone the repository:

    git clone https://github.com/StefanFabian/gstreamer_ros_babel_fish
    

1. Install dependencies using rosdep:

    cd ~/ros2_ws
    rosdep update
    rosdep install --from-paths src --ignore-src -r -y
    

1. Build the package:

    colcon build --packages-select gstreamer_ros_babel_fish
    

1. Source the workspace:

    source install/setup.bash
    

Usage

After sourcing the workspace again, the GStreamer elements rbfimagesrc and rbfimagesink should be available. You can verify this with:

gst-inspect-1.0 rbfimagesrc
gst-inspect-1.0 rbfimagesink

Example Pipelines

1. Display a ROS Image Topic

Subscribe to a ROS topic /camera/image_raw and display it using autovideosink.

gst-launch-1.0 rbfimagesrc topic=/camera/image_raw ! videoconvert ! autovideosink

2. Stream Video to ROS

Generate a test video pattern and publish it to /test_video.

gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,framerate=30/1 ! rbfimagesink topic=/test_video sync=true

3. Publish Compressed Images

Encode video as JPEG and publish to /compressed_video/compressed.

gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,framerate=30/1 ! jpegenc ! rbfimagesink topic=/compressed_video sync=true

[!NOTE] When receiving compressed data (image/jpeg, image/png), rbfimagesink automatically publishes to <topic>/compressed.

4. Roundtrip (ROS -> GStreamer -> ROS)

Subscribe to /input_topic, process it (e.g., flip), and publish to /output_topic.

File truncated at 100 lines see the full file