suave_runner

Docker

Build the headless image from the repository root:

docker build -t suave-headless:dev -f docker/dockerfile-suave-headless .

Run the local image with NVIDIA GPU support:

docker run -it --rm --gpus all --runtime=nvidia --name suave_runner -e DISPLAY=$DISPLAY -e QT_X11_NO_MITSHM=1 -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=all -v /dev/dri:/dev/dri -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime:ro suave-headless:dev

docker run -it --rm --gpus all --runtime=nvidia --name suave_runner -e DISPLAY=$DISPLAY -e QT_X11_NO_MITSHM=1 -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=all -v /dev/dri:/dev/dri -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime:ro -v $HOME/ros_workspaces/suave_ws/src/suave:/home/ubuntu-user/suave_ws/src/suave suave-headless:dev

The published image is available as ghcr.io/kas-lab/suave-headless:main.

docker run -it --rm --gpus all --runtime=nvidia --name suave_runner -e DISPLAY=$DISPLAY -e QT_X11_NO_MITSHM=1 -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=all -v $HOME/suave/results:/home/ubuntu-user/suave/results -v /dev/dri:/dev/dri -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime:ro ghcr.io/kas-lab/suave-headless:main

Batch runner

To run several campaigns sequentially in one go, use the generic run_batch node, configured through a YAML manifest (see config/runner/batch_campaigns.yml for a runnable example covering exp1-exp3 and extended_exp1-extended_exp3 with the bt, metacontrol, random, and none managing systems bundled with this package; a third-party managing system’s own runner config can be added the same way, from its own repo):

ros2 launch suave_runner run_batch_launch.py

Or directly, pointing at any manifest:

ros2 run suave_runner run_batch \
  --ros-args --params-file config/runner/batch_campaigns.yml

Note: running every campaign back-to-back can take several hours to a few days, depending on the machine and num_runs per experiment – plan to leave it running unattended (e.g. in screen/tmux) rather than waiting on it.

This creates a timestamped batch directory under ~/suave/results/batches/ (e.g. batch_20260913_104200/) containing:

• state.json – the batch manifest (schema_version: 2): the campaign list (name, package, config_file), written once when the batch starts. Progress is not stored there; it is derived from each campaign’s runs.json
• run_batch.log – the overall orchestration log
• <campaign_name>.log – combined stdout/stderr for each campaign (e.g. exp1.log)
• campaigns/<campaign_name>/ – each campaign’s own results directory, in the same layout suave_runner produces for a single campaign

Checking whether a batch needs to be resumed: a batch finished cleanly if run_batch.log ends with All campaigns completed successfully.. If it stops early – Ctrl+C, a crash, a failed campaign – the log ends instead with a Batch interrupted; ... or Batch finished with N unfinished campaign(s): ... line followed by the exact resume command to use. A campaign is done when the number of "status": "completed" entries in its campaigns/<name>/runs.json equals the number of runs its runner config declares. suave batch list prints the completed runs per campaign, and diagnose_batch prints completed/expected counts.

Resuming an interrupted batch: pass the batch’s state.json as the resume_state_file parameter; campaigns whose runs have all completed are skipped, and an unfinished campaign picks up from its runs.json instead of starting over:

ros2 run suave_runner run_batch --ros-args \
  -p resume_state_file:=~/suave/results/batches/<batch_dir>/state.json

To run a custom subset or order of campaigns, write your own manifest (or edit config/runner/batch_campaigns.yml) and pass it via --params-file before starting a new batch. -p fail_fast:=true stops the batch as soon as a campaign fails instead of continuing to the next one, and -p dry_run:=true prints the ros2 run command for each campaign without executing anything.

Diagnosing batch failures

diagnose_batch reads a batch’s or campaign’s result files and reports which runs failed, why, and where the evidence is. It only reads files, so it works on a finished batch, a running one, or a copy on another machine:

```bash ros2 run suave_runner diagnose_batch ~/suave/results/batches//state.json ros2 run suave_runner diagnose_batch ~/suave/results/batches/ ros2 run suave_runner diagnose_batch ~/suave/results/ ros2 run suave_runner diagnose_batch --json

File truncated at 100 lines see the full file