ROS Index
|
rai_open_set_vision package from rai reporai_bringup rai_nomad rai_open_set_vision |
ROS Distro
|
Package Summary
| Tags | No category tags. |
| Version | 0.1.0 |
| License | Apache-2.0 |
| Build type | AMENT_PYTHON |
| Use | RECOMMENDED |
Repository Summary
| Description | RAI is a vendor-agnostic agentic framework for robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more. |
| Checkout URI | https://github.com/robotecai/rai.git |
| VCS Type | git |
| VCS Version | development |
| Last Updated | 2025-07-25 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | ai robotics ros2 vlm multimodal embodied-artificial-intelligence embodied-agent embodied-ai o3de llm generative-ai ai-agents-framework embodied-agents robotec |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Package Description
Additional Links
Maintainers
- Kajetan Rachwał
Authors
RAI Open Set Vision
This package provides a ROS2 Node which is an interface to the Idea-Research GroundingDINO Model. It allows for open-set detection.
Installation
In your workspace you need to have an src folder containing this package rai_open_set_vision and the rai_interfaces package.
Preparing the GroundingDINO
Add required ROS dependencies:
rosdep install --from-paths src --ignore-src -r
Build and run
In the base directory of the RAI package install dependencies:
poetry install --with openset
Source the ros installation
source /opt/ros/${ROS_DISTRO}/setup.bash
Run the build process:
colcon build --symlink-install
Source the environment
source setup_shell.sh
Run the GroundedSamAgent and GroundingDinoAgent agents.
python run_vision_agents.py
Agents create two ROS 2 Nodes: grounding_dino and grounded_sam using ROS2Connector.
These agents can be triggered by ROS2 services:
-
grounding_dino_classify:rai_interfaces/srv/RAIGroundingDino -
grounded_sam_segment:rai_interfaces/srv/RAIGroundedSam
[!TIP]
If you wish to integrate open-set vision into your ros2 launch file, a premade launch file can be found in
rai/src/rai_bringup/launch/openset.launch.py
[!NOTE] The weights will be downloaded to
~/.cache/raidirectory.
RAI Tools
rai_open_set_vision package contains tools that can be used by RAI LLM agents
enhance their perception capabilities. For more information on RAI Tools see
Tool use and development tutorial.
GetDetectionTool
This tool calls the grounding dino service to use the model to see if the message from the provided camera topic contains objects from a comma separated prompt.
[!TIP]
you can try example below with rosbotxl demo binary. The binary exposes
/camera/camera/color/image_rawand/camera/camera/depth/image_rawtopics.
Example call
```python from rai_open_set_vision.tools import GetDetectionTool from rai.communication.ros2 import ROS2Connector, ROS2Context
with ROS2Context(): connector=ROS2Connector(node_name=”test_node”) x = GetDetectionTool(connector=connector)._run( camera_topic=”/camera/camera/color/image_raw”, object_names=[“chair”, “human”, “plushie”, “box”, “ball”], )
File truncated at 100 lines see the full file
Package Dependencies
| Deps | Name |
|---|---|
| vision_msgs | |
| rai_interfaces | |
| ament_copyright | |
| ament_pep257 |
System Dependencies
| Name |
|---|
| python3-pytest |
Dependant Packages
Launch files
- launch/example_communication_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- dino_weights_path [default: ]
- sam_weights_path [default: ]
- image_path
- launch/gdino_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
- launch/gsam_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
Messages
Services
Plugins
Recent questions tagged rai_open_set_vision at Robotics Stack Exchange
|
|
rai_open_set_vision package from rai reporai_bringup rai_nomad rai_open_set_vision |
ROS Distro
|
Package Summary
| Tags | No category tags. |
| Version | 0.1.0 |
| License | Apache-2.0 |
| Build type | AMENT_PYTHON |
| Use | RECOMMENDED |
Repository Summary
| Description | RAI is a vendor-agnostic agentic framework for robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more. |
| Checkout URI | https://github.com/robotecai/rai.git |
| VCS Type | git |
| VCS Version | development |
| Last Updated | 2025-07-25 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | ai robotics ros2 vlm multimodal embodied-artificial-intelligence embodied-agent embodied-ai o3de llm generative-ai ai-agents-framework embodied-agents robotec |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Package Description
Additional Links
Maintainers
- Kajetan Rachwał
Authors
RAI Open Set Vision
This package provides a ROS2 Node which is an interface to the Idea-Research GroundingDINO Model. It allows for open-set detection.
Installation
In your workspace you need to have an src folder containing this package rai_open_set_vision and the rai_interfaces package.
Preparing the GroundingDINO
Add required ROS dependencies:
rosdep install --from-paths src --ignore-src -r
Build and run
In the base directory of the RAI package install dependencies:
poetry install --with openset
Source the ros installation
source /opt/ros/${ROS_DISTRO}/setup.bash
Run the build process:
colcon build --symlink-install
Source the environment
source setup_shell.sh
Run the GroundedSamAgent and GroundingDinoAgent agents.
python run_vision_agents.py
Agents create two ROS 2 Nodes: grounding_dino and grounded_sam using ROS2Connector.
These agents can be triggered by ROS2 services:
-
grounding_dino_classify:rai_interfaces/srv/RAIGroundingDino -
grounded_sam_segment:rai_interfaces/srv/RAIGroundedSam
[!TIP]
If you wish to integrate open-set vision into your ros2 launch file, a premade launch file can be found in
rai/src/rai_bringup/launch/openset.launch.py
[!NOTE] The weights will be downloaded to
~/.cache/raidirectory.
RAI Tools
rai_open_set_vision package contains tools that can be used by RAI LLM agents
enhance their perception capabilities. For more information on RAI Tools see
Tool use and development tutorial.
GetDetectionTool
This tool calls the grounding dino service to use the model to see if the message from the provided camera topic contains objects from a comma separated prompt.
[!TIP]
you can try example below with rosbotxl demo binary. The binary exposes
/camera/camera/color/image_rawand/camera/camera/depth/image_rawtopics.
Example call
```python from rai_open_set_vision.tools import GetDetectionTool from rai.communication.ros2 import ROS2Connector, ROS2Context
with ROS2Context(): connector=ROS2Connector(node_name=”test_node”) x = GetDetectionTool(connector=connector)._run( camera_topic=”/camera/camera/color/image_raw”, object_names=[“chair”, “human”, “plushie”, “box”, “ball”], )
File truncated at 100 lines see the full file
Package Dependencies
| Deps | Name |
|---|---|
| vision_msgs | |
| rai_interfaces | |
| ament_copyright | |
| ament_pep257 |
System Dependencies
| Name |
|---|
| python3-pytest |
Dependant Packages
Launch files
- launch/example_communication_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- dino_weights_path [default: ]
- sam_weights_path [default: ]
- image_path
- launch/gdino_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
- launch/gsam_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
Messages
Services
Plugins
Recent questions tagged rai_open_set_vision at Robotics Stack Exchange
|
|
rai_open_set_vision package from rai reporai_bringup rai_nomad rai_open_set_vision |
ROS Distro
|
Package Summary
| Tags | No category tags. |
| Version | 0.1.0 |
| License | Apache-2.0 |
| Build type | AMENT_PYTHON |
| Use | RECOMMENDED |
Repository Summary
| Description | RAI is a vendor-agnostic agentic framework for robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more. |
| Checkout URI | https://github.com/robotecai/rai.git |
| VCS Type | git |
| VCS Version | development |
| Last Updated | 2025-07-25 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | ai robotics ros2 vlm multimodal embodied-artificial-intelligence embodied-agent embodied-ai o3de llm generative-ai ai-agents-framework embodied-agents robotec |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Package Description
Additional Links
Maintainers
- Kajetan Rachwał
Authors
RAI Open Set Vision
This package provides a ROS2 Node which is an interface to the Idea-Research GroundingDINO Model. It allows for open-set detection.
Installation
In your workspace you need to have an src folder containing this package rai_open_set_vision and the rai_interfaces package.
Preparing the GroundingDINO
Add required ROS dependencies:
rosdep install --from-paths src --ignore-src -r
Build and run
In the base directory of the RAI package install dependencies:
poetry install --with openset
Source the ros installation
source /opt/ros/${ROS_DISTRO}/setup.bash
Run the build process:
colcon build --symlink-install
Source the environment
source setup_shell.sh
Run the GroundedSamAgent and GroundingDinoAgent agents.
python run_vision_agents.py
Agents create two ROS 2 Nodes: grounding_dino and grounded_sam using ROS2Connector.
These agents can be triggered by ROS2 services:
-
grounding_dino_classify:rai_interfaces/srv/RAIGroundingDino -
grounded_sam_segment:rai_interfaces/srv/RAIGroundedSam
[!TIP]
If you wish to integrate open-set vision into your ros2 launch file, a premade launch file can be found in
rai/src/rai_bringup/launch/openset.launch.py
[!NOTE] The weights will be downloaded to
~/.cache/raidirectory.
RAI Tools
rai_open_set_vision package contains tools that can be used by RAI LLM agents
enhance their perception capabilities. For more information on RAI Tools see
Tool use and development tutorial.
GetDetectionTool
This tool calls the grounding dino service to use the model to see if the message from the provided camera topic contains objects from a comma separated prompt.
[!TIP]
you can try example below with rosbotxl demo binary. The binary exposes
/camera/camera/color/image_rawand/camera/camera/depth/image_rawtopics.
Example call
```python from rai_open_set_vision.tools import GetDetectionTool from rai.communication.ros2 import ROS2Connector, ROS2Context
with ROS2Context(): connector=ROS2Connector(node_name=”test_node”) x = GetDetectionTool(connector=connector)._run( camera_topic=”/camera/camera/color/image_raw”, object_names=[“chair”, “human”, “plushie”, “box”, “ball”], )
File truncated at 100 lines see the full file
Package Dependencies
| Deps | Name |
|---|---|
| vision_msgs | |
| rai_interfaces | |
| ament_copyright | |
| ament_pep257 |
System Dependencies
| Name |
|---|
| python3-pytest |
Dependant Packages
Launch files
- launch/example_communication_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- dino_weights_path [default: ]
- sam_weights_path [default: ]
- image_path
- launch/gdino_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
- launch/gsam_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
Messages
Services
Plugins
Recent questions tagged rai_open_set_vision at Robotics Stack Exchange
|
|
rai_open_set_vision package from rai reporai_bringup rai_nomad rai_open_set_vision |
ROS Distro
|
Package Summary
| Tags | No category tags. |
| Version | 0.1.0 |
| License | Apache-2.0 |
| Build type | AMENT_PYTHON |
| Use | RECOMMENDED |
Repository Summary
| Description | RAI is a vendor-agnostic agentic framework for robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more. |
| Checkout URI | https://github.com/robotecai/rai.git |
| VCS Type | git |
| VCS Version | development |
| Last Updated | 2025-07-25 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | ai robotics ros2 vlm multimodal embodied-artificial-intelligence embodied-agent embodied-ai o3de llm generative-ai ai-agents-framework embodied-agents robotec |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Package Description
Additional Links
Maintainers
- Kajetan Rachwał
Authors
RAI Open Set Vision
This package provides a ROS2 Node which is an interface to the Idea-Research GroundingDINO Model. It allows for open-set detection.
Installation
In your workspace you need to have an src folder containing this package rai_open_set_vision and the rai_interfaces package.
Preparing the GroundingDINO
Add required ROS dependencies:
rosdep install --from-paths src --ignore-src -r
Build and run
In the base directory of the RAI package install dependencies:
poetry install --with openset
Source the ros installation
source /opt/ros/${ROS_DISTRO}/setup.bash
Run the build process:
colcon build --symlink-install
Source the environment
source setup_shell.sh
Run the GroundedSamAgent and GroundingDinoAgent agents.
python run_vision_agents.py
Agents create two ROS 2 Nodes: grounding_dino and grounded_sam using ROS2Connector.
These agents can be triggered by ROS2 services:
-
grounding_dino_classify:rai_interfaces/srv/RAIGroundingDino -
grounded_sam_segment:rai_interfaces/srv/RAIGroundedSam
[!TIP]
If you wish to integrate open-set vision into your ros2 launch file, a premade launch file can be found in
rai/src/rai_bringup/launch/openset.launch.py
[!NOTE] The weights will be downloaded to
~/.cache/raidirectory.
RAI Tools
rai_open_set_vision package contains tools that can be used by RAI LLM agents
enhance their perception capabilities. For more information on RAI Tools see
Tool use and development tutorial.
GetDetectionTool
This tool calls the grounding dino service to use the model to see if the message from the provided camera topic contains objects from a comma separated prompt.
[!TIP]
you can try example below with rosbotxl demo binary. The binary exposes
/camera/camera/color/image_rawand/camera/camera/depth/image_rawtopics.
Example call
```python from rai_open_set_vision.tools import GetDetectionTool from rai.communication.ros2 import ROS2Connector, ROS2Context
with ROS2Context(): connector=ROS2Connector(node_name=”test_node”) x = GetDetectionTool(connector=connector)._run( camera_topic=”/camera/camera/color/image_raw”, object_names=[“chair”, “human”, “plushie”, “box”, “ball”], )
File truncated at 100 lines see the full file
Package Dependencies
| Deps | Name |
|---|---|
| vision_msgs | |
| rai_interfaces | |
| ament_copyright | |
| ament_pep257 |
System Dependencies
| Name |
|---|
| python3-pytest |
Dependant Packages
Launch files
- launch/example_communication_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- dino_weights_path [default: ]
- sam_weights_path [default: ]
- image_path
- launch/gdino_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
- launch/gsam_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
Messages
Services
Plugins
Recent questions tagged rai_open_set_vision at Robotics Stack Exchange
|
|
rai_open_set_vision package from rai reporai_bringup rai_nomad rai_open_set_vision |
ROS Distro
|
Package Summary
| Tags | No category tags. |
| Version | 0.1.0 |
| License | Apache-2.0 |
| Build type | AMENT_PYTHON |
| Use | RECOMMENDED |
Repository Summary
| Description | RAI is a vendor-agnostic agentic framework for robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more. |
| Checkout URI | https://github.com/robotecai/rai.git |
| VCS Type | git |
| VCS Version | development |
| Last Updated | 2025-07-25 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | ai robotics ros2 vlm multimodal embodied-artificial-intelligence embodied-agent embodied-ai o3de llm generative-ai ai-agents-framework embodied-agents robotec |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Package Description
Additional Links
Maintainers
- Kajetan Rachwał
Authors
RAI Open Set Vision
This package provides a ROS2 Node which is an interface to the Idea-Research GroundingDINO Model. It allows for open-set detection.
Installation
In your workspace you need to have an src folder containing this package rai_open_set_vision and the rai_interfaces package.
Preparing the GroundingDINO
Add required ROS dependencies:
rosdep install --from-paths src --ignore-src -r
Build and run
In the base directory of the RAI package install dependencies:
poetry install --with openset
Source the ros installation
source /opt/ros/${ROS_DISTRO}/setup.bash
Run the build process:
colcon build --symlink-install
Source the environment
source setup_shell.sh
Run the GroundedSamAgent and GroundingDinoAgent agents.
python run_vision_agents.py
Agents create two ROS 2 Nodes: grounding_dino and grounded_sam using ROS2Connector.
These agents can be triggered by ROS2 services:
-
grounding_dino_classify:rai_interfaces/srv/RAIGroundingDino -
grounded_sam_segment:rai_interfaces/srv/RAIGroundedSam
[!TIP]
If you wish to integrate open-set vision into your ros2 launch file, a premade launch file can be found in
rai/src/rai_bringup/launch/openset.launch.py
[!NOTE] The weights will be downloaded to
~/.cache/raidirectory.
RAI Tools
rai_open_set_vision package contains tools that can be used by RAI LLM agents
enhance their perception capabilities. For more information on RAI Tools see
Tool use and development tutorial.
GetDetectionTool
This tool calls the grounding dino service to use the model to see if the message from the provided camera topic contains objects from a comma separated prompt.
[!TIP]
you can try example below with rosbotxl demo binary. The binary exposes
/camera/camera/color/image_rawand/camera/camera/depth/image_rawtopics.
Example call
```python from rai_open_set_vision.tools import GetDetectionTool from rai.communication.ros2 import ROS2Connector, ROS2Context
with ROS2Context(): connector=ROS2Connector(node_name=”test_node”) x = GetDetectionTool(connector=connector)._run( camera_topic=”/camera/camera/color/image_raw”, object_names=[“chair”, “human”, “plushie”, “box”, “ball”], )
File truncated at 100 lines see the full file
Package Dependencies
| Deps | Name |
|---|---|
| vision_msgs | |
| rai_interfaces | |
| ament_copyright | |
| ament_pep257 |
System Dependencies
| Name |
|---|
| python3-pytest |
Dependant Packages
Launch files
- launch/example_communication_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- dino_weights_path [default: ]
- sam_weights_path [default: ]
- image_path
- launch/gdino_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
- launch/gsam_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
Messages
Services
Plugins
Recent questions tagged rai_open_set_vision at Robotics Stack Exchange
|
|
rai_open_set_vision package from rai reporai_bringup rai_nomad rai_open_set_vision |
ROS Distro
|
Package Summary
| Tags | No category tags. |
| Version | 0.1.0 |
| License | Apache-2.0 |
| Build type | AMENT_PYTHON |
| Use | RECOMMENDED |
Repository Summary
| Description | RAI is a vendor-agnostic agentic framework for robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more. |
| Checkout URI | https://github.com/robotecai/rai.git |
| VCS Type | git |
| VCS Version | development |
| Last Updated | 2025-07-25 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | ai robotics ros2 vlm multimodal embodied-artificial-intelligence embodied-agent embodied-ai o3de llm generative-ai ai-agents-framework embodied-agents robotec |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Package Description
Additional Links
Maintainers
- Kajetan Rachwał
Authors
RAI Open Set Vision
This package provides a ROS2 Node which is an interface to the Idea-Research GroundingDINO Model. It allows for open-set detection.
Installation
In your workspace you need to have an src folder containing this package rai_open_set_vision and the rai_interfaces package.
Preparing the GroundingDINO
Add required ROS dependencies:
rosdep install --from-paths src --ignore-src -r
Build and run
In the base directory of the RAI package install dependencies:
poetry install --with openset
Source the ros installation
source /opt/ros/${ROS_DISTRO}/setup.bash
Run the build process:
colcon build --symlink-install
Source the environment
source setup_shell.sh
Run the GroundedSamAgent and GroundingDinoAgent agents.
python run_vision_agents.py
Agents create two ROS 2 Nodes: grounding_dino and grounded_sam using ROS2Connector.
These agents can be triggered by ROS2 services:
-
grounding_dino_classify:rai_interfaces/srv/RAIGroundingDino -
grounded_sam_segment:rai_interfaces/srv/RAIGroundedSam
[!TIP]
If you wish to integrate open-set vision into your ros2 launch file, a premade launch file can be found in
rai/src/rai_bringup/launch/openset.launch.py
[!NOTE] The weights will be downloaded to
~/.cache/raidirectory.
RAI Tools
rai_open_set_vision package contains tools that can be used by RAI LLM agents
enhance their perception capabilities. For more information on RAI Tools see
Tool use and development tutorial.
GetDetectionTool
This tool calls the grounding dino service to use the model to see if the message from the provided camera topic contains objects from a comma separated prompt.
[!TIP]
you can try example below with rosbotxl demo binary. The binary exposes
/camera/camera/color/image_rawand/camera/camera/depth/image_rawtopics.
Example call
```python from rai_open_set_vision.tools import GetDetectionTool from rai.communication.ros2 import ROS2Connector, ROS2Context
with ROS2Context(): connector=ROS2Connector(node_name=”test_node”) x = GetDetectionTool(connector=connector)._run( camera_topic=”/camera/camera/color/image_raw”, object_names=[“chair”, “human”, “plushie”, “box”, “ball”], )
File truncated at 100 lines see the full file
Package Dependencies
| Deps | Name |
|---|---|
| vision_msgs | |
| rai_interfaces | |
| ament_copyright | |
| ament_pep257 |
System Dependencies
| Name |
|---|
| python3-pytest |
Dependant Packages
Launch files
- launch/example_communication_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- dino_weights_path [default: ]
- sam_weights_path [default: ]
- image_path
- launch/gdino_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
- launch/gsam_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
Messages
Services
Plugins
Recent questions tagged rai_open_set_vision at Robotics Stack Exchange
|
|
rai_open_set_vision package from rai reporai_bringup rai_nomad rai_open_set_vision |
ROS Distro
|
Package Summary
| Tags | No category tags. |
| Version | 0.1.0 |
| License | Apache-2.0 |
| Build type | AMENT_PYTHON |
| Use | RECOMMENDED |
Repository Summary
| Description | RAI is a vendor-agnostic agentic framework for robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more. |
| Checkout URI | https://github.com/robotecai/rai.git |
| VCS Type | git |
| VCS Version | development |
| Last Updated | 2025-07-25 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | ai robotics ros2 vlm multimodal embodied-artificial-intelligence embodied-agent embodied-ai o3de llm generative-ai ai-agents-framework embodied-agents robotec |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Package Description
Additional Links
Maintainers
- Kajetan Rachwał
Authors
RAI Open Set Vision
This package provides a ROS2 Node which is an interface to the Idea-Research GroundingDINO Model. It allows for open-set detection.
Installation
In your workspace you need to have an src folder containing this package rai_open_set_vision and the rai_interfaces package.
Preparing the GroundingDINO
Add required ROS dependencies:
rosdep install --from-paths src --ignore-src -r
Build and run
In the base directory of the RAI package install dependencies:
poetry install --with openset
Source the ros installation
source /opt/ros/${ROS_DISTRO}/setup.bash
Run the build process:
colcon build --symlink-install
Source the environment
source setup_shell.sh
Run the GroundedSamAgent and GroundingDinoAgent agents.
python run_vision_agents.py
Agents create two ROS 2 Nodes: grounding_dino and grounded_sam using ROS2Connector.
These agents can be triggered by ROS2 services:
-
grounding_dino_classify:rai_interfaces/srv/RAIGroundingDino -
grounded_sam_segment:rai_interfaces/srv/RAIGroundedSam
[!TIP]
If you wish to integrate open-set vision into your ros2 launch file, a premade launch file can be found in
rai/src/rai_bringup/launch/openset.launch.py
[!NOTE] The weights will be downloaded to
~/.cache/raidirectory.
RAI Tools
rai_open_set_vision package contains tools that can be used by RAI LLM agents
enhance their perception capabilities. For more information on RAI Tools see
Tool use and development tutorial.
GetDetectionTool
This tool calls the grounding dino service to use the model to see if the message from the provided camera topic contains objects from a comma separated prompt.
[!TIP]
you can try example below with rosbotxl demo binary. The binary exposes
/camera/camera/color/image_rawand/camera/camera/depth/image_rawtopics.
Example call
```python from rai_open_set_vision.tools import GetDetectionTool from rai.communication.ros2 import ROS2Connector, ROS2Context
with ROS2Context(): connector=ROS2Connector(node_name=”test_node”) x = GetDetectionTool(connector=connector)._run( camera_topic=”/camera/camera/color/image_raw”, object_names=[“chair”, “human”, “plushie”, “box”, “ball”], )
File truncated at 100 lines see the full file
Package Dependencies
| Deps | Name |
|---|---|
| vision_msgs | |
| rai_interfaces | |
| ament_copyright | |
| ament_pep257 |
System Dependencies
| Name |
|---|
| python3-pytest |
Dependant Packages
Launch files
- launch/example_communication_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- dino_weights_path [default: ]
- sam_weights_path [default: ]
- image_path
- launch/gdino_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
- launch/gsam_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
Messages
Services
Plugins
Recent questions tagged rai_open_set_vision at Robotics Stack Exchange
|
|
rai_open_set_vision package from rai reporai_bringup rai_nomad rai_open_set_vision |
ROS Distro
|
Package Summary
| Tags | No category tags. |
| Version | 0.1.0 |
| License | Apache-2.0 |
| Build type | AMENT_PYTHON |
| Use | RECOMMENDED |
Repository Summary
| Description | RAI is a vendor-agnostic agentic framework for robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more. |
| Checkout URI | https://github.com/robotecai/rai.git |
| VCS Type | git |
| VCS Version | development |
| Last Updated | 2025-07-25 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | ai robotics ros2 vlm multimodal embodied-artificial-intelligence embodied-agent embodied-ai o3de llm generative-ai ai-agents-framework embodied-agents robotec |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Package Description
Additional Links
Maintainers
- Kajetan Rachwał
Authors
RAI Open Set Vision
This package provides a ROS2 Node which is an interface to the Idea-Research GroundingDINO Model. It allows for open-set detection.
Installation
In your workspace you need to have an src folder containing this package rai_open_set_vision and the rai_interfaces package.
Preparing the GroundingDINO
Add required ROS dependencies:
rosdep install --from-paths src --ignore-src -r
Build and run
In the base directory of the RAI package install dependencies:
poetry install --with openset
Source the ros installation
source /opt/ros/${ROS_DISTRO}/setup.bash
Run the build process:
colcon build --symlink-install
Source the environment
source setup_shell.sh
Run the GroundedSamAgent and GroundingDinoAgent agents.
python run_vision_agents.py
Agents create two ROS 2 Nodes: grounding_dino and grounded_sam using ROS2Connector.
These agents can be triggered by ROS2 services:
-
grounding_dino_classify:rai_interfaces/srv/RAIGroundingDino -
grounded_sam_segment:rai_interfaces/srv/RAIGroundedSam
[!TIP]
If you wish to integrate open-set vision into your ros2 launch file, a premade launch file can be found in
rai/src/rai_bringup/launch/openset.launch.py
[!NOTE] The weights will be downloaded to
~/.cache/raidirectory.
RAI Tools
rai_open_set_vision package contains tools that can be used by RAI LLM agents
enhance their perception capabilities. For more information on RAI Tools see
Tool use and development tutorial.
GetDetectionTool
This tool calls the grounding dino service to use the model to see if the message from the provided camera topic contains objects from a comma separated prompt.
[!TIP]
you can try example below with rosbotxl demo binary. The binary exposes
/camera/camera/color/image_rawand/camera/camera/depth/image_rawtopics.
Example call
```python from rai_open_set_vision.tools import GetDetectionTool from rai.communication.ros2 import ROS2Connector, ROS2Context
with ROS2Context(): connector=ROS2Connector(node_name=”test_node”) x = GetDetectionTool(connector=connector)._run( camera_topic=”/camera/camera/color/image_raw”, object_names=[“chair”, “human”, “plushie”, “box”, “ball”], )
File truncated at 100 lines see the full file
Package Dependencies
| Deps | Name |
|---|---|
| vision_msgs | |
| rai_interfaces | |
| ament_copyright | |
| ament_pep257 |
System Dependencies
| Name |
|---|
| python3-pytest |
Dependant Packages
Launch files
- launch/example_communication_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- dino_weights_path [default: ]
- sam_weights_path [default: ]
- image_path
- launch/gdino_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
- launch/gsam_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
Messages
Services
Plugins
Recent questions tagged rai_open_set_vision at Robotics Stack Exchange
|
|
rai_open_set_vision package from rai reporai_bringup rai_nomad rai_open_set_vision |
ROS Distro
|
Package Summary
| Tags | No category tags. |
| Version | 0.1.0 |
| License | Apache-2.0 |
| Build type | AMENT_PYTHON |
| Use | RECOMMENDED |
Repository Summary
| Description | RAI is a vendor-agnostic agentic framework for robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more. |
| Checkout URI | https://github.com/robotecai/rai.git |
| VCS Type | git |
| VCS Version | development |
| Last Updated | 2025-07-25 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | ai robotics ros2 vlm multimodal embodied-artificial-intelligence embodied-agent embodied-ai o3de llm generative-ai ai-agents-framework embodied-agents robotec |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Package Description
Additional Links
Maintainers
- Kajetan Rachwał
Authors
RAI Open Set Vision
This package provides a ROS2 Node which is an interface to the Idea-Research GroundingDINO Model. It allows for open-set detection.
Installation
In your workspace you need to have an src folder containing this package rai_open_set_vision and the rai_interfaces package.
Preparing the GroundingDINO
Add required ROS dependencies:
rosdep install --from-paths src --ignore-src -r
Build and run
In the base directory of the RAI package install dependencies:
poetry install --with openset
Source the ros installation
source /opt/ros/${ROS_DISTRO}/setup.bash
Run the build process:
colcon build --symlink-install
Source the environment
source setup_shell.sh
Run the GroundedSamAgent and GroundingDinoAgent agents.
python run_vision_agents.py
Agents create two ROS 2 Nodes: grounding_dino and grounded_sam using ROS2Connector.
These agents can be triggered by ROS2 services:
-
grounding_dino_classify:rai_interfaces/srv/RAIGroundingDino -
grounded_sam_segment:rai_interfaces/srv/RAIGroundedSam
[!TIP]
If you wish to integrate open-set vision into your ros2 launch file, a premade launch file can be found in
rai/src/rai_bringup/launch/openset.launch.py
[!NOTE] The weights will be downloaded to
~/.cache/raidirectory.
RAI Tools
rai_open_set_vision package contains tools that can be used by RAI LLM agents
enhance their perception capabilities. For more information on RAI Tools see
Tool use and development tutorial.
GetDetectionTool
This tool calls the grounding dino service to use the model to see if the message from the provided camera topic contains objects from a comma separated prompt.
[!TIP]
you can try example below with rosbotxl demo binary. The binary exposes
/camera/camera/color/image_rawand/camera/camera/depth/image_rawtopics.
Example call
```python from rai_open_set_vision.tools import GetDetectionTool from rai.communication.ros2 import ROS2Connector, ROS2Context
with ROS2Context(): connector=ROS2Connector(node_name=”test_node”) x = GetDetectionTool(connector=connector)._run( camera_topic=”/camera/camera/color/image_raw”, object_names=[“chair”, “human”, “plushie”, “box”, “ball”], )
File truncated at 100 lines see the full file
Package Dependencies
| Deps | Name |
|---|---|
| vision_msgs | |
| rai_interfaces | |
| ament_copyright | |
| ament_pep257 |
System Dependencies
| Name |
|---|
| python3-pytest |
Dependant Packages
Launch files
- launch/example_communication_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- dino_weights_path [default: ]
- sam_weights_path [default: ]
- image_path
- launch/gdino_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]
- launch/gsam_launch.xml
- ~ Copyright 2024 Robotec.ai ~ ~ Licensed under the Apache License, Version 2.0 (the "License"); ~ you may not use this file except in compliance with the License. ~ You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, software ~ distributed under the License is distributed on an "AS IS" BASIS, ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ~ See the License for the specific language governing permissions and ~ limitations under the License.
-
- weights_path [default: ]