1 Introduction

Traditional military ground-level Automatic Target Recognition (ATR) systems analyze IR images for the signatures of potential targets. Although such systems have proven quite successful in wide-spread use, they fail in certain predictable scenarios, notably when the targets are colder than expected, or when the background is hotter than expected (see figure  1 ). One approach to this problem is to develop more sophisticated target recognition algorithms for IR images (Schachter [ 17 ] contains a review of several methods). It is our belief, however, that the gains possible through this line of research are limited due to problems inherent to the data. A more promising approach, we believe, is to collect additional data using non-IR sensors, and to look for target signatures there. The issues with this approach are cost and independence (in the sense that ATR on the additional data should succeed in scenarios where the IR-based system fails).

   

Figure 1: Visible light (left) and IR (middle) images of targets. In the IR image, not all targets are visible, and some parts of the background are as bright as the targets. The results (right) from applying the DARPA-UGV Demo-C IR ATR system show one missed target and one false positive.

This paper presents an alternative approach to ATR that uses color imagery. There are several advantages to using color (described later) which enable our system to be used either in stand-alone mode, or with systems based on other sensors. We should emphasize that we are suggesting supplementing - not replacing - IR-based ATR systems. IR systems work well in many scenarios and are already in wide-spread use; color-based systems (or any other method based on visible spectrum data), on the other hand, cannot ordinarily be used at night. However, at least one of the scenarios in which IR systems fail (i.e., due to background heat) is an typically daytime scenario, when color-based systems should be most reliable.

Color-based target recognition is inherently difficult, due to (i) the camouflage on targets, and (ii) variation in the apparent color of objects under outdoor imaging conditions. Camouflage, is, of course, the standard counter-measure against detection in visible light, and it forces any color-based ATR system to make very fine distinctions in order to separate target from background. However, the color of background vegetation continually changes, so it is difficult, if not impossible, for camouflage color to perfectly match the background; furthermore, mismatches in color between target and background are made even more common by the multiple colors used.

The apparent color of a given target (or object) varies under outdoor conditions due to a number of factors, namely the color of the incident daylight, surface reflectance properties of the target, illumination geometry (i.e., the position and orientation of the target surface w.r.t. the illuminant) and viewing geometry (the position and orientation of the camera w.r.t. the target surface). The color of daylight changes significantly due to the sun-angle and weather conditions, and the position and orientation of the target are also expected to vary. Consequently, the apparent color of a target varies under realistic conditions. Previous methods in computational color recognition, such as color constancy algorithms [ 18 , 7 , 6 ], have dealt with varying color in highly constrained environments, and are generally not applicable to outdoor imagery.

It will be shown that as imaging conditions vary, the apparent color of objects forms characteristic types of clusters in color (RGB) space, depending on the surface properties. The method presented here uses multivariate decision trees (MDT's) for recursive, non-parametric function approximation to estimate the clusters in RGB, based on training samples of targets. Given samples of a target under different lighting conditions, MDT's construct a piece-wise linear approximation of the boundary of the region in color space. After the training phase is complete, every image pixel can be classified as target or background according to whether or not it lies within the learned boundary. The result is a binary region-of-interest image that marks all the pixels that lie within the region in color space occupied by the object's representation; the target pixels in the binary images are then grouped to produce bounding rectangles around the targets. The RGB representation of color makes it possible to use a lookup table for real-time classification on standard hardware.

This method has been implemented in a system for ATR of camouflaged military vehicles in real-time, and has been tested in a DARPA-sanctioned study [ 19 ] on the Ft. Carson data set [ 1 ] and at the DARPA UGV Demo-C [ 11 ]. In each test, over 90% of the targets were detected (compared to a 45% detection rate by the IR-based system). A combination of color and IR systems resulted in detection of nearly 100% of the targets.