autoware_tensorrt_plugins

Purpose

The autoware_tensorrt_plugins package extends the operations available in TensorRT via plugins.

Algorithms

The following operations are implemented:

Sparse convolutions

We provide a wrapper for spconv (please see the correspondent package for details about the algorithms involved). This requires the installation of spconv_cpp which is automatically installed in autoware’s setup script. If needed, the user can also build and install it using the repository’s instructions.

Argsort

We provide an implementation for the Argsort operation as a plugin since the TopK TensorRT implementation has limitations in the size of elements it can handle.

BEV Pool

We provide a wrapper for the bev_pool operation presented in BEVFusion. Please refer to the original paper for specific details.

Scatter operations

We provide a wrapper for the segment_csr operation presented in torch_scatter. Please refer to the original code for specific details.

The SegmentCSR TensorRT plugin supports a 2D source tensor and a 1D CSR row pointer:

• src: row-major tensor with shape [num_rows, num_cols]
• indptr: int64 tensor with shape [num_segments + 1]
• output: tensor with shape [num_segments, num_cols]

Each output row reduces the contiguous source rows src[indptr[s] : indptr[s + 1]] independently for every column. Higher-rank/batched CSR inputs are not part of this plugin contract; callers should flatten or reshape inputs before invoking the plugin.

Unique

While ONNX supports the unique operation, TensorRT does not provide an implementation. For this reason we implement Unique as CustomUnique to avoid name classes. The implementation mostly follows torch_scatter implementation. Please refer to the original code for specific details.

Multi-Scale Deformable Attention

The MultiScaleDeformableAttentionPlugin implements the multi-scale deformable attention mechanism introduced in Deformable DETR. This operation is crucial for vision transformers that need to attend to multiple scales and spatial locations efficiently.

Key features:

• Supports multi-scale feature maps with different resolutions
• Enables learning of sampling offsets and attention weights
• Optimized CUDA implementation for efficient GPU execution
• Supports both FP32 and FP16 precision

Inputs:

1. value: Feature maps at different scales (B, L, M, D)
2. spatial_shapes: Spatial dimensions of each scale (N, 2)
3. level_start_index: Starting indices for each scale (N,)
4. sampling_loc: Learned sampling locations (B, Q, M, L, P, 2)
5. attn_weight: Learned attention weights (B, Q, M, L, P)

Output:

• Attended features (B, Q, M*D)

Rotate

The RotatePlugin provides efficient image rotation functionality with support for different interpolation methods. This is useful for data augmentation and geometric transformations in perception pipelines.

Key features:

• Supports bilinear and nearest neighbor interpolation
• Arbitrary rotation angles around a specified center point
• Optimized CUDA kernels for both FP32 and FP16 precision
• Handles boundary conditions properly

Inputs:

1. input: Input image tensor (C, H, W)
2. angle: Rotation angle in degrees (scalar)
3. center: Center of rotation (2,)

Output:

• Rotated image with same dimensions as input

Parameters:

• interpolation: Interpolation mode (0 = bilinear, 1 = nearest)

Select and Pad

The SelectAndPadPlugin enables conditional selection and padding of tensor elements based on flags. This is particularly useful for dynamic batching scenarios where sequences have variable lengths.

Key features:

• Efficiently selects valid elements based on boolean flags
• Pads output to a fixed size with invalid tokens
• Uses CUB library for optimized GPU selection operations
• Supports both FP32 and FP16 precision

File truncated at 100 lines see the full file

Changelog for package autoware_tensorrt_plugins

0.53.0 (2026-09-29)

• Merge remote-tracking branch 'origin/main' into prepare-0.53.0-changelog

• feat(autoware_ptv3): support multisweep densification with time-lag features (#13300)

  • feat(autoware_ptv3): support multisweep densification with time-lag features
  • feat(autoware_ptv3): support none source reconstruction mode
  • fix(autoware_ptv3): publish every output cloud in input order
  • perf(autoware_ptv3): drop the per-frame stream sync from sweep aggregation
  • fix(autoware_ptv3): mark the current sweep explicitly instead of inferring it from the lag
  • fix(autoware_ptv3): reject unusable frames when they are cached instead of throwing mid-inference
  • refactor(autoware_ptv3): create the TF buffer and listener only when densification is enabled
  • fix(autoware_ptv3): advertise the filtered cloud only when filter classes are configured
  • refactor(autoware_ptv3): drop the duplicated frame count, config copies and cache walks
  • fix(autoware_ptv3): pass identity for the current frame instead of a pose round trip

  * chore: add voxelizer to the cspell dictionary ---------

• chore(pre-commit): update clang-format to v22.1.5 (#13126)

  • chore(pre-commit): update clang-format to v22.1.5

  * style(pre-commit): autofix ---------

• Contributors: Amadeusz Szymko, Mete Fatih Cırıt, Ryohsuke Mitsudome

0.52.0 (2026-06-30)

• Merge remote-tracking branch 'origin/main' into tmp/bot/bump_version_base

• perf(autoware_tensorrt_plugins): keep SegmentCSR allocation-free (#12555) Initialize the SegmentCSR output buffer directly instead of allocating, filling, copying, and freeing a scratch base buffer on every launch. Co-authored-by: Copilot <<223556219+Copilot@users.noreply.github.com>>

• feat(autoware_tensorrt_plugins): support fused bias/activation in ImplicitGemmPlugin (#12658)

  • feat(autoware_tensorrt_plugins): add fused activation and optional bias support to ImplicitGemmPlugin

  - Add act_type field to ImplicitGemmParameters for fused activation (kNone/kReLU/kSigmoid/kLeakyReLU)

  - Support optional 6th input for per-channel fused bias (BN-folded); the plugin now accepts 5 (no bias) or 6 (with bias) inputs

  - Pass act_alpha, act_beta, act_type, and bias tensor to ConvGemmOps::implicit_gemm instead of hardcoded kNone/empty

  • Track num_plugin_inputs_ across clone/configurePlugin/onShapeChange

  - Extend parse_fields lambda to accept act_type as optional 7th ONNX attribute (backward-compatible with 6-field ONNX graphs)

  • chore: fix naming
  • chore: fix implicit gemm
  • chore: clean code

  * chore: fix naming ---------

• refactor(autoware_tensorrt_plugins): remove SegmentCSR nD indptr path (#12739) Remove the unused n-dimensional indptr launcher path and offset helper now that the plugin contract is explicitly 1D. Co-authored-by: Copilot <<223556219+Copilot@users.noreply.github.com>>

• fix(autoware_tensorrt_plugins): fix implicitgemm build and runtime error (#12734)

  • fix(autoware_tensorrt_plugins): fix kBUILD/kRUNTIME deserialization and type issues
  • refactor(autoware_tensorrt_plugins): extract field parsing into lambda in createPlugin

  * chore: fix cspell ---------

File truncated at 100 lines see the full file