ROS Index
No version for distro humble. Known supported distros are highlighted in the buttons above.
No version for distro jazzy. Known supported distros are highlighted in the buttons above.
No version for distro kilted. Known supported distros are highlighted in the buttons above.
No version for distro rolling. Known supported distros are highlighted in the buttons above.
Repository Summary
| Description | ROS 2 node for open-vocabulary object detection using NanoOWL. |
| Checkout URI | https://github.com/nvidia-ai-iot/ros2-nanoowl.git |
| VCS Type | git |
| VCS Version | master |
| Last Updated | 2024-03-08 |
| Dev Status | UNKNOWN |
| Released | UNRELEASED |
| Tags | No category tags. |
| Contributing |
Help Wanted (-)
Good First Issues (-) Pull Requests to Review (-) |
Packages
| Name | Version |
|---|---|
| ros2_nanoowl | 0.0.0 |
README
ROS2 NanoOWL
ROS2 node for open-vocabulary object detection using NanoOWL.
NanoOWL optimizes OWL-ViT to run real-time on NVIDIA Jetson Orin with TensorRT. This project provides a ROS 2 package for object detection using NanoOWL.
Setup
- Set up your Isaac ROS development environment following instructions here.
- Clone required projects under
${ISAAC_ROS_WS}/src:
cd ${ISAAC_ROS_WS}/src
git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git
git clone https://github.com/NVIDIA-AI-IOT/ROS2-NanoOWL.git
git clone https://github.com/NVIDIA-AI-IOT/nanoowl
git clone https://github.com/NVIDIA-AI-IOT/torch2trt
git clone --branch humble https://github.com/ros2/demos.git
- Launch the docker container using the
run_dev.shscript:
cd ${ISAAC_ROS_WS}/src/isaac_ros_common
./scripts/run_dev.sh
- Install dependencies:
- Pytorch: The Isaac ROS development environment that we set up in step 1 comes with PyTorch preinstalled. Check your PyTorch version using the interactive Python interpreter by running python from terminal, and these commands:
import torch
torch.__version__
-
NVIDIA TensorRT: If you’re developing on an NVIDIA Jetson, TensorRT is pre installed as part of JetPack. Verify the installation by running python from terminal, and then this command in the interactive Python interpreter:
import tensorrt. If it says ‘ModuleNotFound’, try the following command and check again following the steps above:
sudo apt-get install python3-libnvinfer-dev
If this fails, run the following command and try again:
sudo apt-get install apt-utils
In case the 'ModuleNotFound' error still shows up - The python bindings to tensorrt are available in ```dist-packages ```, which may not be visible to your environment. We add ```dist-packages ``` to ```PYTHONPATH ``` to make this work:
export PYTHONPATH=/usr/lib/python3.8/dist-packages:$PYTHONPATH
If ```tensorrt ``` is still not installed, try the following command:
pip install pycuda
-
Torchvision: Identify which version of torchvision is compatible with your PyTorch version from here. Clone and install that specific version from source in your workspace’s src folder:
git clone –-branch <version> https://github.com/pytorch/vision.git. For example:
cd ${ISAAC_ROS_WS}/src
git clone --branch v0.13.0 https://github.com/pytorch/vision.git
cd vision
pip install .
Verify that torchvision has been installed correctly using the interactive Python interpreter by running python from terminal, and these commands:
cd ../
import torchvision
torchvision.__version__
If it says ‘ModuleNotFound’, try each of the following and check again following the steps above:
sudo apt install nvidia-cuda-dev
pip install ninja
sudo apt-get install ninja-build
- Transformers library:
pip install transformers
- Matplotlib:
pip install matplotlib
- torch2trt: Enter the torch2trt repository cloned in step 2 and install the package:
cd ${ISAAC_ROS_WS}/src/torch2trt
pip install .
- NanoOWL: Enter the NanoOWL repository cloned in step 2 and install the package:
cd ${ISAAC_ROS_WS}/src/nanoowl
pip install .
-
cam2image:
We want to use the image_tools package from the
demosrepository that we cloned to take input from an attached usb camera. Build and source this package from your workspace:
```
File truncated at 100 lines see the full file
CONTRIBUTING
No version for distro galactic. Known supported distros are highlighted in the buttons above.
No version for distro iron. Known supported distros are highlighted in the buttons above.
No version for distro melodic. Known supported distros are highlighted in the buttons above.
No version for distro noetic. Known supported distros are highlighted in the buttons above.