By analyzing how image sequences digitized by a camera evolve over time, it is possible to estimate both the motion of moving obstacles and the three dimensional (3D) structure of the world, and to begin to interpret that 3D data. This work integrates a real-time motion segmentation system (ASSET-2) and a 3D reconstruction/interpretation system (ALTRUISM).
The motion segmentation system is used to pre-segment the image sequence into the (probably moving) background and independently moving objects. ASSET-2 tracks all of the moving objects and then subtracts from the original list of 2D features those features which correspond to these objects, leaving a list of features which have been produced by the static part of the scene, i.e. the rest of the world.
Here is an example image, with the complete (original) list of 2D features superimposed on it:
The next picture shows the estimate of the object's outline made by ASSET-2. The number is the object number of the Landrover, as labelled internally by ASSET-2. The square near the centre of the Landrover is the tracked 2D object centre. The line coming out of this square is the overall image motion of the object, scaled by a factor of two:
Now the final list of 2D features output by ASSET-2 is shown:
This enables the structure-from-motion (SFM) system DROID (which, like many SFM algorithms, cannot cope with more than one set of motion parameters in the field of view) to run successfully on the background part of the image sequence. Finally, the output of the SFM system is interpreted by ALTRUISM to give warning of non-ground-plane obstacles. In these example outputs of the 3D processing and segmentation, successfully tracked 3D points are split into ``road'' and ``off-road'' groups. The moving objects have been ignored and 3D interpretation is successful.
A paper on this work has been published in ICPR96; for the original report, either download the PostScript (integration.ps.gz - 600 K / integration.ps - 2 Megs) or view the html.