Data Tamer Tools

Tools for working with DataTamer data in ROS2, providing Foxglove integration and MCAP data storage capabilities.

Overview

This package provides five main tools:

• Foxglove Relay: Real-time visualization of DataTamer data through Foxglove Studio
• MCAP Sink: Store DataTamer data in MCAP (MessagePack) format for efficient storage and playback
• MCAP Converter: Convert DataTamer-encoded MCAP files to Protobuf-encoded MCAP files for Foxglove compatibility
• Rosout Logger: Record ROS log messages (/rosout) to MCAP files with Foxglove Log schema
• Log Rotation Coordinator: Synchronize log directory rotation across multiple MCAP loggers

Tools

1. Foxglove Relay (foxglove_relay)

A ROS2 component that bridges DataTamer data to Foxglove Studio for real-time visualization.

Features:

• Automatically discovers DataTamer topics (/data_tamer/schema, /data_tamer/schemas, /data_tamer/snapshot)
• Supports both JSON and Protocol Buffers encoding for Foxglove visualization
• Converts DataTamer schemas to JSON Schema or Protocol Buffers descriptor format
• Creates Foxglove WebSocket server for real-time data streaming
• Supports soft eviction of idle channels (closes channels but keeps schemas)
• Configurable eviction policies and discovery intervals
• Optional ROS log relaying to Foxglove

Prerequisites:

• Your DataTamer channels must use DataTamer::ROS2PublisherSink to publish data to ROS2 topics
• The relay automatically discovers and subscribes to the standard DataTamer ROS2 topics

Usage:

# Launch as ROS2 component
ros2 run data_tamer_tools foxglove_relay

# Or with custom parameters
ros2 run data_tamer_tools foxglove_relay --ros-args \
  -p host:=127.0.0.1 \
  -p port:=8765 \
  -p use_protobuf:=true \
  -p eviction_ttl_sec:=900 \
  -p eviction_period_sec:=30 \
  -p discovery_sec:=5 \
  -p enable_rosout:=true

Parameters:

WebSocket Server Configuration:

• host (string, default: “127.0.0.1”): WebSocket server host address
• port (int, default: 8765): WebSocket server port number

Data Encoding:

• use_protobuf (bool, default: true): Use Protocol Buffers encoding instead of JSON
  • true: Data is serialized using Protocol Buffers for better performance and smaller payloads
  • false: Data is serialized as JSON strings for human readability

Channel Management:

• eviction_ttl_sec (int, default: 900): Time-to-live for idle channels in seconds
  • Channels that haven’t received data for this duration will be soft-evicted (closed but schema kept)
• eviction_period_sec (int, default: 30): Interval in seconds for checking stale channels
  • How often the relay checks for channels that should be evicted
• discovery_sec (int, default: 5): Interval in seconds for topic discovery
  • How often the relay scans for new DataTamer topics to subscribe to

ROS Log Integration:

• enable_rosout (bool, default: true): Enable relaying of ROS logs to Foxglove
  • true: ROS logs from /rosout topic are relayed to Foxglove Log channel
  • false: Disable ROS log relaying
• rosout_topic (string, default: “/rosout”): ROS topic name for log messages

Foxglove Studio Connection:

1. Open Foxglove Studio
2. Add a “Raw Messages” panel
3. Connect to ws://localhost:8765 (or your configured host:port)
4. Select the desired DataTamer channel to visualize

Encoding Notes:

• Protocol Buffers mode (use_protobuf:=true): More efficient, smaller payloads, better performance
• JSON mode (use_protobuf:=false): Human-readable data, easier to debug, larger payloads
• Both modes provide the same data visualization capabilities in Foxglove Studio

2. MCAP Sink

Stores DataTamer data in MCAP (MessagePack) format for efficient storage and playback.

Features:

• Multiple compression options (None, LZ4, Zstd)
• Configurable chunk sizes
• Thread-safe operations
• Integration with DataTamer sink system
• Log rotation support: When initialized with a ROS node, automatically subscribes to rotation topic
• Synchronized rotation: Multiple MCAP sinks can rotate to new directories simultaneously

Basic Usage:

```cpp #include <data_tamer_tools/sinks/mcap_sink.hpp>

File truncated at 100 lines see the full file