I would like to introduce a set of ROS tools that uses depth data for multiple purposes related to the autonomous navigation of mobile robots. The open-source code has been written in C++ and published on GitHub as a ROS package depth_nav_tools. Already it’s available also as a ROS2 package.
What is the ROS?
The Robot Operating System (ROS) is the most popular, open-source robotics framework used as a base for different robotics algorithms. The major advantage of such a framework is that it provides mechanisms for communication between components (called nodes in ROS) but also mechanisms for launching and software debugging.
Depth-based ROS tools
The tools use mostly depth data, which means data, especially from RGB-D sensors like Microsoft Kinect or Intel RealSense. A few tools (or rather ROS nodes) are placed in the depth_nav_tools package.
- There is a
laserscan_kinectused for conversion of RGB-D data to a format of a laser scanner. - Another tools is a
depth_sesor_posethat estimates the RGB-D sensor pose based on the ground plane detection. - Finally, a node
cliff_detectoris able to detect stairs or holes on the ground, using RGB-D data.
3D data conversion with the laserscan_kinect
The laserscan_kinect node converts the depth image to a lidar format (LaserScan). It finds the smallest value of distance in each column of the depth image and converts it to polar coordinates. The package provides the following features:
- removing a ground plane from the output data,
- a sensor tilt compensation.
However, the sensor pose (height and tilt angle) must be known for correct data processing. Parameters should be calculated in a frame of the ground plane.
The below picture shows a comparison between a laser scan based on the converted depth image from a Microsoft Kinect (blue points) and a laser scan from a lidar Hokuyo URG-04LX-UG01 (black points).
During the tuning process, additional debug images can be generated and checked. It contains lines that represent the lower and upper bounds of the detection area. Also, the closest points in each image column are visible.
A sensor height and angle estimation with the depth_sensor_pose
The depth_sensor_pose node detects the ground plane on the depth image and estimates the height and the tilt angle of the depth sensor relative to the ground. The procedure of ground plane detection is based on the RANSAC algorithm and ranges of acceptable parameters. It could be really helpful in the sensor position calibration procedure but also it could be run online to estimate the sensors’ pose during the robot movement.
Obstacles detection with the cliff_detector
The cliff_detector node detects negative obstacles like cliffs or downstairs. It uses a known sensor pose (for example from depth_sensor_pose) to determine obstacles that are located below the ground plane.
The idea behind this detector is to check for each pixel if the distance is above the threshold which means the point is below the ground plane. Then, pixels are grouped, and based on simple voting, hypotheses about obstacles are confirmed or rejected. I have shared the source code of this solution on GitHub: cliff_detector.
Tests
A ReMeDi robot was used to detect obstacles that are below the ground plane. It was equipped with the first version of the Microsoft Kinect sensor.
All tools running together
It is possible to run all tools together, especially in configuration with depth_sensor_pose which estimates the tilt angle and height of the sensor. Then such values are passed to laserscan_kinect and cliff_detector. In this configuration, the detectors are resistant to small changes in the sensor orientation or distance from the ground. It can be useful for example in the case of not very stable robots.
Additional documentation
- More details can be found in github repository README
- or in the paper M. Drwięga, J. Jakubiak. A set of depth sensor processing ROS tools for wheeled mobile robot navigation. JAMRIS, 2017.