buildfarm perf tests

Purpose

This package defines some tests. On one hand it invokes perf_test from Apex.AI’s performance_test package. This allows you to test performance and latency of several ROS 2 RMW implementations. On the other hand we are evaluating the additional overhead caused by a single pub/sub topic or one process spinning and detect potential leaks related to theses activities.

• There is a test for each RMW:

  • CycloneDDS (standalone test)
  • FastRTPS (standalone test)
  • rmw_connext_cpp
  • rmw_cyclonedds_cpp
  • rmw_fastrtps_cpp
  • rmw_fastrtps_dynamic_cpp
  • rmw_opensplice_cpp

Test 1 - Performance Test (Apex.AI)

In this test we are running the Performance Test provided by Apex.AI. Right now we have our own fork because there are some pending pull requests in the official gitlab repository.

In this test we are measurement:

• Average latency
• CPU usage (provided by Apex.AI tool)
• Sent/Received messages per second
• Total lost messages
• Max resident set size

We are generating two plots per measurement

• One per-build
• Other over time

The test only measures the latency between the same RMW implementation

Test 2 - Simple pub/sub

In this case we are testing one publisher and one subscriber in different processes sending a 1kArray at 5Hz. This will allow us to evaluate additional overhead caused by a single pub/sub topic and detect leaks related to this activity.

We measure for both publisher and subscriber:

• Average round trip
• CPU usage ( readed from the filesystem )
• Total lost messages
• Received/Sent messages per second
• Physical memory
• Resident anonymous memory
• Virtual memory

Again we plot measurement:

• One per-build
• Other over time

Test 3 - Node spinning

This test creates one process and spins for 1 minute to evaluate ROS 2 overhead and detect obvious leaks.

We measure:

• CPU usage ( readed from file system )
• Physical memory
• Resident anonymous memory
• Virtual memory

Again we plot measurement:

• One per-build
• Other over time

Build

1. Install ROS 2 (https://index.ros.org/doc/ros2/Installation/).

File truncated at 100 lines see the full file

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.