Planning Data Analyzer

Overview

This package provides offline evaluation tools for trajectory planning performance analysis from recorded rosbag data.

Evaluation Modes

1. Open Loop: Evaluate prediction accuracy against ground truth trajectories
2. OR Scene: Evaluate override regression scenarios (LIVE vs HISTORICAL)

Quick Start

Open Loop Evaluation

ros2 run autoware_planning_data_analyzer autoware_planning_data_analyzer_node --ros-args \
  -p bag_path:=~/my_bag \
  -p evaluation.mode:=open_loop \
  -p trajectory_topic:=/planning/trajectory \
  -p json_output_path:=~/results.json

OR Scene Evaluation

# Using the unified script
./install/autoware_planning_data_analyzer/share/autoware_planning_data_analyzer/scripts/run_evaluation.sh \
  -b ~/result_bag.mcap \
  -m or_scene \
  -t /planning/diffusion_planner/trajectory \
  -o ~/results \
  --live \
  --viz

Multi-Model Pipeline

# Evaluate multiple models from DLR results
./install/autoware_planning_data_analyzer/share/autoware_planning_data_analyzer/scripts/multi_model_pipeline.sh \
  --scenario ~/dataset/scenario.yaml \
  --trajectory /planning/model_a/trajectory \
  --trajectory /planning/model_b/trajectory \
  --output ~/comparison \
  --viz

Output Files

• evaluation_result.json - Detailed metrics (ADE, FDE, TTC, etc.) and summary statistics
• evaluation_output.bag/ - Evaluation metrics as rosbag for visualization
• debug_images/*.png - Trajectory visualization (with --viz flag)

Parameters

Key parameters in config/offline_evaluation.param.yaml:

• evaluation.mode: Evaluation mode (open_loop or or_scene)
• evaluation_interval_ms: Sampling interval (default: 100ms)
• sync_tolerance_ms: Time synchronization tolerance (default: 100ms)
• trajectory_topic: Trajectory topic to evaluate
• or_scene_evaluation.*: OR scene specific settings

Utility Scripts

Located in scripts/ directory:

• Bag Processing: merge_bags.py, rename_bag_topic.py, add_prefix_to_bag.py
• Ground Truth: add_gt_trajectory_to_bag.py
• OR Analysis: detect_or_and_route.py, evaluate_or_segments.py
• Visualization: generate_or_visualization.py
• Comparison: compare_live_historical.py

See scripts/README.md for detailed documentation.

For More Information

• Offline evaluation scripts: scripts/README.md
• Configuration: config/offline_evaluation.param.yaml
• Launch files: launch/

Changelog for package autoware_planning_data_analyzer

0.4.0 (2025-11-16)

0.3.0 (2025-08-11)

• chore: sync files (#214)
• Contributors: awf-autoware-bot[bot]

0.2.0 (2025-03-24)

0.1.0 (2025-01-28)

• unify version to 0.0.0

• feat(autoware_planning_data_analyzer)!: replace tier4_debug_msgs with tier4_internal_debug_msgs (#199)

• chore: sync files (#11) Co-authored-by: github-actions <<github-actions@github.com>> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

• refactor(global_parameter_loader): prefix package and namespace with autoware (#159)

• refactor(glog_component): prefix package and namespace with autoware (#158)

• perf(planning_data_analyzer): improve performance of weight grid search (#116)

  • perf(planning_data_analyzer): improve weight search logic
  • perf: multithread

  * style(pre-commit): autofix ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

• feat(planning_data_analyzer): add new package (#103)

  • feat(planning_data_analyzer): add new package
  • style(pre-commit): autofix
  • refactor: use map
  • style(pre-commit): autofix
  • fix: print
  • style(pre-commit): autofix
  • refactor: small change
  • feat: weight grid search
  • feat: add time delta
  • style(pre-commit): autofix
  • fix: typo

  * fix: ignore spell check for foxglove json ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

• Contributors: Esteve Fernandez, Ryohsuke Mitsudome, Satoshi OTA, Yutaka Kondo, awf-autoware-bot[bot]