Hunav Evaluator

A ROS2 python package to compute different human-aware navigation metrics about the simulations performed with the HuNav Simulator.

Tested in ROS2 Humble

Dependencies

• numpy
• hunav_msgs
• hunav_agent_manager

Description

This package subscribes to a set of topic published by the Hunav Agent Manager:

• /human_states this topic publishes messages of type hunav_msgs/msg/Agents with the status of the human agents for each time step.
• /robot_states this topic publishes messages of type hunav_msgs/msg/Agent with the status of the robot for each time step.

This information is stored and then is employed to compute a set of metrics related to the social robot navigation.

In order to compute the metrics, the user must specify a set of parameters in a yaml file. The evaluator will read this file and will compute the metrics indicated in it. The results will be stored in a csv file in the path indicated in the parameters.

To run the evaluator, the user must execute the following command:

ros2 launch hunav_evaluator hunav_evaluator.launch.py

This will start the evaluator node, which will subscribe to the topics and compute the metrics according to the parameters specified in the yaml file.

The evaluator recording can be started/stopped by using a ROS2 service. The service is called /hunav_start_recording and it takes a hunav_msgs/srv/StartEvaluation request with the following parameters:

• experiment_tag A string that will be used to tag the experiment. This tag will be used to generate the output files with the results of the metrics.
• robot_goal A geometry_msgs/msg/PoseStamped message with the goal position of the robot. This is used to compute some metrics related to the goal position.
• run_id An integer that will be used to identify the run of the experiment. This is useful to distinguish between different runs of the same experiment.

The evaluator will start recording the data and will compute the metrics according to the parameters specified in the yaml file. The recording can be stopped by calling the service /hunav_stop_recording . This service is an empty service that will stop the recording and will compute the final metrics.

Configuration and Parameters

The user must specify the set of metrics to be computed and some parameters about the metrics generation. This values must be indicated in the yaml file metrics.yaml, placed in the config directory. After compilation this file is moved to the install directory of the ROS workspace.

Global parameters

• frequency . Indicate the frequency of capturing the data (it must be slower than data publishing). If the value is set to zero, the data is captured at the same frequency than it is published.
• result_file . Full path and name of the output text file. If a new experiment is performed and the file already exists, the evaluator will open the file and add the computed metrics in a new row at the end of the file. This allows to repeat experiments and to store the results as rows in the same file for easier post-processing. The parameter can be changed during the experiment resetting this parameter.
• metrics . List with the names of the metrics to be computed. The user can just comment/uncomment the desired metrics.

Metrics configuration

Currently, the metrics implemented are those employed in our previous work [1] and the SEAN simulator [2]. The metrics from the SocNavBench [3], Crowdbot [4] and the compilation indicated in the work of Gao et al. [5], are being studied and included.

The user can configure the metrics to be computed in two different ways:

• By editing the metrics yaml file, and changing the boolean parameters of the metrics (true to compute the metric, false to ignore it).
• By using a RViz2 panel to select/unselect the metrics and to store them. See the hunav_rviz2_panel for more information on how to use the panel.

Example snippet of the metrics configuration file:

```yaml metrics:

  # Pérez-Higueras, N., Caballero, F. & Merino, L. 
  # Teaching Robot Navigation Behaviors to Optimal RRT Planners. 
  # Int.J. of Soc. Robotics 10, 235–249 (2018). https://doi.org/10.1007/s12369-017-0448-1
  time_to_reach_goal: true
  path_length: false
  cumulative_heading_changes: true
  avg_distance_to_closest_person: true
  minimum_distance_to_people: true
  intimate_space_intrusions: true
  personal_space_intrusions: true
  social_space_intrusions: true
  group_intimate_space_intrusions: true
  group_personal_space_intrusions: true
  group_social_space_intrusions: true
  
  
  # N. Tsoi et al., "SEAN 2.0: Formalizing and Generating Social Situations
  # for Robot Navigation," in IEEE Robotics and Automation Letters, vol. 7,
  # no. 4, pp. 11047-11054, Oct. 2022, doi: 10.1109/LRA.2022.3196783.
  completed: true
  minimum_distance_to_target: true
  final_distance_to_target: true
  robot_on_person_collision: false
  person_on_robot_collision: false
  time_not_moving: true
  
  # SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigation
  #ABHIJAT BISWAS, ALLAN WANG, GUSTAVO SILVERA, AARON STEINFELD, and HENNY AD-MONI, Carnegie Mellon University
  avg_robot_linear_speed: true
  avg_robot_angular_speed: true
  avg_acceleration: false
  avg_overacceleration: false
  
  # Learning a Group-Aware Policy for Robot Navigation
  # Kapil Katyal ∗1,2 , Yuxiang Gao ∗2 , Jared Markowitz 1 , Sara Pohland 3 , Corban Rivera 1 , I-Jeng Wang 1 , Chien-Ming Huang 2 
  avg_pedestrian_velocity: true

File truncated at 100 lines see the full file