Planning Evaluator

Purpose

This package provides nodes that generate various metrics to evaluate the quality of planning.

Metrics can be published in real time and saved to a JSON file when the node is shut down:

• metrics_for_publish:
  • Metrics listed in metrics_for_publish are calculated and published to the topic.
• metrics_for_output:
  • Metrics listed in metrics_for_output are saved to a JSON file when the node shuts down (if output_metrics is set to true).
  • These metrics include statistics derived from metrics_for_publish and additional information such as parameters and descriptions.

Metrics

All possible metrics are defined in the Metric enumeration located in:

• include/autoware/planning_evaluator/metrics/metric.hpp
• include/autoware/planning_evaluator/metrics/output_metric.hpp

These files also provide string conversions and human-readable descriptions for use in output files.

Metric Classifications

By Data Type

1. Statistics-based Metrics:
   • Calculated using autoware_utils::Accumulator, which tracks minimum, maximum, mean, and count values.
   • Sub-metrics: /mean, /min, /max, and /count.
2. Value-based Metrics:
   • Metrics with a single value.
   • Sub-metrics: /value.
   • Some metrics with older implementations use the statistics-based format of /mean, /min, /max, but all values are the same.
3. Count-based Metrics:
   • Count occurrences of specific events over a time period.
   • Sub-metrics: /count, or /count_in_duration.

By Purpose

1. Trajectory Metrics
2. Trajectory Deviation Metrics
3. Trajectory Stability Metrics
4. Trajectory Obstacle Metrics
5. Modified Goal Metrics
6. Planning Factor Metrics
7. Steering Metrics
8. Blinker Metrics
9. Other Information

Detailed Metrics

Trajectory Metrics

Evaluates the current trajectory T(0) itself. Metrics are calculated and published when a trajectory is received.

Implemented metrics

• curvature: Statistics of curvature at each trajectory point.
  • Sub-metrics to publish: /mean, /min, /max.
  • Sub-metrics to output: The same as above but take the published data as data point instead of each trajectory point.
• point_interval: Statistics of distances between consecutive trajectory points.
  • Sub-metrics to publish: /mean, /min, /max.
  • Sub-metrics to output: The same as above but take the published data as data point instead of each trajectory point.
• relative_angle: Statistics of angles between consecutive trajectory points.
  • Parameters: trajectory.min_point_dist_m (minimum distance between points).
  • Sub-metrics to publish: /mean, /min, /max.
  • Sub-metrics to output: The same as above but take the published data as data point instead of each trajectory point.

• resampled_relative_angle: Similar to relative_angle, but considers a point at a fixed distance (e.g., half the vehicle length) for angle calculation from the current point as the next point to calculate the relative angle, instead of using the immediately adjacent point.

• length: Total trajectory length.
  • Sub-metrics: Value-based metric, but using the statistics-based format of /mean, /min, /max with the same value.
  • Sub-metrics to output: /mean, /min, /max for the published data.
• duration: Expected driving time to travel the trajectory.
  • Sub-metrics: Value-based metric, but using the statistics-based format of /mean, /min, /max with the same value.
  • Sub-metrics to output: /mean, /min, /max for the published data.
• velocity: Statistics of velocity at each trajectory point.
  • Sub-metrics to publish: /mean, /min, /max.
  • Sub-metrics to output: The same as above but take the published data as data point instead of each trajectory point.
• acceleration: Statistics of acceleration at each trajectory point.
  • Sub-metrics to publish: /mean, /min, /max.
  • Sub-metrics to output: The same as above but take the published data as data point instead of each trajectory point.
• jerk: Statistics of jerk at each trajectory point.
  • Sub-metrics to publish: /mean, /min, /max.
  • Sub-metrics to output: The same as above but take the published data as data point instead of each trajectory point.

Trajectory Deviation Metrics

Evaluates the trajectory deviation by comparing the trajectory T(0) and the reference trajectory.

File truncated at 100 lines see the full file

Changelog for package autoware_planning_evaluator

0.50.0 (2026-02-14)

• Merge remote-tracking branch 'origin/main' into humble

• fix(autoware_planning_evaluator): sort predicted paths (#12034) fix predicted path sorting bug

• refactor(evaluator): migrate deprecated getClosestLanelet() (#11987)

• fix(autoware_planning_evaluator): a pet bug (#11950) fix pet bug

• chore(planning_evaluator): on the worst only (#11932) on the worst only

• feat(autoware_planning_evaluator): new obstacle metrics (#11761)

  • tmp save
  • remove some draft code
  • add new implements
  • polish code, need to update readme
  • pre-commit
  • update readme
  • fix cppcheck
  • fix unit test bug, and add test cases for ttc, drac.
  • cry to fix ci building error

  * refactor code ---------

• docs(planning_evaluator): revise general terminology (#11833)

• Contributors: Kem (TiankuiXian), Mamoru Sobue, Ryohsuke Mitsudome, Zulfaqar Azmi

0.49.0 (2025-12-30)

• Merge remote-tracking branch 'origin/main' into prepare-0.49.0-changelog

• fix: resolve clock type mismatch in tf2 lookups with simulation time (#11523)

  * fix: resolve clock type mismatch in tf2 transform lookups Replace rclcpp::Time(0) with tf2::TimePointZero in lookupTransform calls to fix clock type conflicts when using simulation time. The issue:

  • rclcpp::Time(0) creates a time with SYSTEM_TIME clock type
  • When nodes run with use_sim_time:=true, transforms use ROS_TIME clock
  • This causes clock type mismatch errors in tf2 lookups

  - Error: "Lookup would require extrapolation into the past" The fix:

  • tf2::TimePointZero is clock-type agnostic
  • Correctly represents "get latest available transform"

  - Also replaced rclcpp::Duration::from_seconds() with tf2::durationFromSec() This bug affects transform lookups in critical safety and planning components, causing runtime errors when simulation time is enabled. Affected packages:

  • autoware_autonomous_emergency_braking
  • autoware_planning_evaluator
  • autoware_freespace_planner

  * style(pre-commit): autofix ---------Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> Co-authored-by: Shumpei Wakabayashi <<42209144+shmpwk@users.noreply.github.com>>

• Contributors: Ryohsuke Mitsudome, ralwing

0.48.0 (2025-11-18)

• Merge remote-tracking branch 'origin/main' into humble

• feat(planning_evaluator): refactor the obstacle_distance and obstacle_ttc metric (#11478)

  • tmp save
  • refactor and pre-commit
  • tmp save
  • fix start point bug and apply deceleration lower bound
  • remove unused launch parm
  • polish code
  • fix unit tests

  * update readme ---------

File truncated at 100 lines see the full file