A summary of previous work on segmentation of the image flow field is now given. Smith [26] gives a more complete review.
Spoerri and Ullman [31] and Waxman and Duncan [37] segment the image flow field into independently moving objects using local flow discontinuities, giving fairly inaccurate boundaries. Meygret and Thonnat [21], Thompson [32], Thompson and Pong [33], McLauchlan et al. [19] and Nelson [23] segment on the basis of simple flow clustering or inconsistency with the background flow. Adiv [1], Debrunner and Ahuja [11], Diehl [12], Burt et al. [10], Bergen et al. [4] and Torr and Murray [34] segment using analytic image transformations; fits are found within the flow field of analytic functions with a number of parameters. Murray and Buxton [22], Bouthemy and Lalande [8] and Black [5] use statistical regularization to perform globally optimal segmentation, utilizing various locally defined cost functionals. Verri et al. [35] and Rognone et al. [24] perform segmentation on the basis of global motion clustering. Previous work on segmentation has usually involved making restrictive assumptions about the world, the motion of the camera etc. ASSET-2 does not depend on this sort of assumption; it is shown later that good results can be obtained without doing so.
Very little work has been undertaken which involves the temporal integration, or tracking, of motion segmentation results. Irani et al. [16] use the segmentation methods of Burt, Bergen et al. with temporal integration of the segmentation results to improve performance. However, this ``temporal integration'' does not involve any sort of shape tracking or modelling, so that information about scene events is not readily available. Meyer and Bouthemy [20] use the approach of Bouthemy and François [7] to achieve segmentation, and then track moving objects' outlines over time. The objects are matched over time using a polygonal representation. The motion estimation and segmentation stages are quite separate from the shape tracking stage, leading to a rather unintegrated approach. Results are only presented for a static camera.
The research described in this paper covers this area as well as the segmentation, that is, a temporally coherent list of segmented objects is maintained as time proceeds, and objects move about in the image.