interbotix_perception_modules

Overview

This package contains Python modules along with supporting level ROS nodes to perform perception with any robotic arm platform. Using the tools here along with a color/depth camera, robotic arm, and AprilTag, it is possible to have the arm pick and place small objects from a tabletop-type platform using camera feedback. Examples showcasing this can be found in the interbotix_XXXXX_perception ROS package in any arm-related repository.

Structure

Below is a list and short description of each helper module. Over time, this list will grow to include others.

• apriltag - small API to interact with AprilTag’s apriltag_ros_single_image_server_node node; it allows users to snap a picture of an AprilTag and returns its pose relative to the camera frame. Note that the module is only meant to be used to capture static AR tags or to find a static transform with the help of the tag. For AR tag tracking, the apriltag_ros_continuous_node should be used. No module exists for that yet. To import, include from interbotix_perception_modules.apriltag import InterbotixAprilTagInterface in your Python script.

  • camera_info_topic - ROS parameter that specifies the absolute topic name where the camera calibration info is published
  • camera_color_topic - ROS parameter that specifies the absolute topic name where the camera color images are published

• armtag - small API that builds upon the apriltag module; it calculates a static transform (usually from the camera’s frame to an arm’s base_link frame) using the help of an AprilTag located on the arm (at a location specified in the URDF). To import, include from interbotix_perception_modules.armtag import InterbotixArmTagInterface in your Python script.

  • arm_tag_frame - ROS parameter that specifies the name of the link (usually in the URDF) on the arm where the AR tag is located
  • ref_frame - ROS parameter that specifies the ‘parent’ frame of the static transform you’d like to publish using the help of the AprilTag
  • arm_base_frame - ROS parameter that specifies the ‘child’ frame of the static transform you’d like to publish using the help of the AprilTag

• pointcloud - small API that can be used to tune various filter parameters used in the Perception Pipeline. To import, include from interbotix_perception_modules.pointcloud import InterbotixPointCloudInterface in your Python script.

  • filter_params - ROS parameter that specifies the filepath to a YAML file where all the filter settings are stored

Below is a list and short description of each helper node. Over time, this list will grow to include others.

• armtag_tuner_gui - presents a PyQt GUI to the user with a ‘snap’ button that can be used to find an AprilTag and publish the desired static transform; it also shows the resulting transform to the user

• picture_snapper - small node that presents a ‘snap_picture’ ROS Server; it saves the latest picture obtained from a camera to the specified file location; it’s important for this node to run on the same computer that the apriltag_ros_single_image_server_node node is running on since the latter node needs to be able to access the image file.

• pointcloud_tuner_gui - presents a PyQt GUI to the user with display boxes and slider bars to allow a user to tune filter parameters used in the perception pipeline; it also loads/saves these settings to a YAML file

• perception_pipeline - implements the Perception Pipeline using the PointCloud Library

Usage

To use any module, simply include the appropriate launch file in the launch directory to your master launch file. Then make sure to include the appropriate import statement in your Python script as described above.

GUI Options

There are two GUIs in this package. The first GUI - the Armtag Tuner GUI can be used to snap an AprilTag pose and publish a static transform using it. The second GUI - the PointCloud Tuner GUI can be used to adjust filter parameters used in the Perception Pipeline. See descriptions about each part of these GUIs below. Note that the order in which the the bullet points appear is the order that a user should operate the GUI.

• Num Samples - this adjusts from a value of 1 to 10 in integer increments; it tells the backend how many samples it should take of the AprilTag before averaging them together to get a more accurate pose
• Snap Pose - by pressing this button, the AprilTag algorithm is run Num Samples times; the description below the button tells you what the transform represents; the resulting average of these transform is then sent over to the static_trans_pub node from the interbotix_tf_tools package to be eventually included in the /tf_static tree
• Displays - the six non-editable fields below the transform description let the user know what the actual transform is

The next GUI shows the filters implemented in the Perception Pipeline in the order that they are used. The main objective of the pipeline is to input a raw pointcloud and output a processed pointcloud which only shows the points that represent objects (i.e. clusters).

• CropBox Filter Settings - these six editable displays and slider bars allow a user to crop out any points in the pointcloud outside a ‘cube’ specified by their values; the resulting cube should only include the surface that the objects are on in addition to the objects themselves.

• Voxel Grid Setting - the leaf size specifies the height, width, and length of a cube used to downsample the points output from the cropbox filter. All points within the space of that cube are averaged into just one point; this significantly reduces the number of point that the remaining filters have to operate on - resulting in faster and more efficient processing. However, the larger the cube, the coarser the pointcloud becomes…

• Segmentation Filter Settings - these are parameters used to segment out the points representing the tabletop or ground surface; Max Iterations is the maximum number of iterations the sample consensus method should run (not sure, but I believe the higher the value, the more accurate the planar model representing the points is); the default value is 50 and that seems to work just fine; Dist Thresh is the max distance perpendicular from the calculated planar model in which a point should be considered part of the plane; the higher the value, the better the plane points will be segmented out at the cost of losing some of the points near the bottom of each object; the lower the value, the more the points that make up the bottom of each object are preserved at the cost of not having every planar point removed…

• Radius Outlier Removal Settings - these are parameters to reduce the number of ‘noisy’ points that have still made it through the filters above; Radius Search is the radial distance around any given point in the pointcloud where neighboring points should be searched. If a point has at least ‘X’ Min Neighbors in the search radius, the point is kept; otherwise, it is thrown out.

• Cluster Settings - these are parameters used to determine what’s considered an acceptable cluster. Cluster Tol is a thresholding parameter that says any point within a specified distance from another point should be treated as a single cluster; Cluster Min Size specifies that any cluster must be made up of at least ‘Y’ many points to be considered a cluster; similarly, Cluster Max Size specifies that any cluster with more than ‘Z’ many points should no longer be considered a cluster.

• Load Configs - this allows a user to specify a file from which to load a set of desired filter parameters.

• Save configs - this allows a user to specify a file to which to save the current settings.

• Reset Configs - this allows a user to reset the current settings to be what they were when the file was originally loaded into the GUI.

Launch File Arguments

Short descriptions for each launch file’s arguments are below…

apriltag.launch

| Argument | Description | Default Value | | ——– | ———– | :———–: | | tag_family | family to which the AprilTag belongs | tagStandard41h12 | | standalone_tags | individual AprilTags the algorithm should be looking for | refer to tags.yaml | | camera_frame | the camera frame in which the AprilTag will be detected | camera_color_optical_frame | | apriltag_ns | name-space where the AprilTag related nodes and parameters are located | apriltag | | camera_color_topic | the absolute ROS topic name to subscribe to color images | camera/color/image_raw | | camera_info_topic | the absolute ROS topic name to subscribe to the camera color info | camera/color/camera_info |

armtag.launch

Besides for the arguments listed below, the arguments above are in this launch file as well…

pc_filter.launch

| Argument | Description | Default Value | | ——– | ———– | :———–: |

File truncated at 100 lines see the full file