NASA - National Aeronautics and Space Administration
+ Visit


Unit B head

The first vision function added to Robonaut is a stereo vision tracker. This tracker uses a stream of stereo image pairs from Robonaut's head to distinguish foreground objects from the background. Robonaut tracks the closest foreground object by panning its head to keep the selected object centered in its field of view as the object moves; for example, it can track a person walking around the laboratory.

Each image is filtered using a sign of Laplacian-of-Gaussian filter, a bandpass filter that emphasizes the edges in the image while smoothing out small scale detail. Binary correlation is then used to find matching patches between the left and right images of a stereo pair. 3-D world constraints are used to limit the volume over which this matching is allowed to take place.

An initial target is acquired on the basis of a sparse set of stereo matches obtained by searching in a stereo pair over a limited depth range centered at nominal 3-D location. If more than one target is found, the closest one is used. When the next stereo pair is available, Robonaut searches over the same depth range now centered on the location of the target found in the previous stereo pair. This simple scheme causes Robonaut to switch to a new target if a person walks in front of and close to the current target. However, if someone walks in front of the current target but at some distance from it, he will be ignored since Robonaut starts its search from the previous location of the target. Likewise, if the current target ducks down out of the field of view, Robonaut will switch its tracking to the closest foreground object. If necessary, the search range will be expanded until a foreground object is found.

Robonaut vision

These behaviors can be observed in this video of Robonaut tracking a person walking around the laboratory. Note that the target stays centered in the larger picture. This is the Robonaut's eye view after filtering the image with the LoG. The smaller picture-in-picture shows the world view from a fixed camera. Robonaut's head on the left side of this image can be seen panning to follow the person that it is tracking.