Stereo vision

The rc_cube is based on stereo vision using the SGM (Semi-Global Matching) method. In stereo vision, 3D information about a scene can be extracted by comparing two images taken from different viewpoints. The main idea behind using a camera pair for measuring depth is the fact that object points appear at different positions in the two camera images depending on their distance from the camera pair. Very distant object points appear at approximately the same position in both images, whereas very close object points occupy different positions in the left and right camera image. The object points’ displacement in the two images is called disparity. The larger the disparity, the closer the object is to the camera. The principle is illustrated in Fig. 6.

_images/stereo_sketch_v2.svg

Fig. 6 Sketch of the stereo-vision principle: The more distant object (black) exhibits a smaller disparity \(d_2\) than that of the close object (gray), \(d_1\).

Stereo vision is a form of passive sensing, meaning that it emits neither light nor other signals to measure distances, but uses only light that the environment emits or reflects. Thus, the Roboception products utilizing this sensing principle can work indoors and outdoors and multiple devices can work together without interferences.

To compute the 3D information, the stereo matching algorithm must be able to find corresponding object points in the left and right camera images. For this, the algorithm requires texture, meaning changes in image intensity values due to patterns or the objects’ surface structure, in the images. Stereo matching is not possible for completely untextured regions, such as a flat white wall without any visible surface structure. The SGM stereo matching method used provides the best trade-off between runtime and accuracy, even for fine structures.

The following software components are required to compute 3D information:

  • Stereo camera: This component is responsible for capturing synchronized stereo image pairs and transforming them into images approaching those taken by an ideal stereo camera (rectification).
  • Stereo matching: This component computes disparities for the rectified stereo camera pair using SGM.