Existing FRSs are generally imagebased, video-based, or 2D- or 3D-based, although these are broad classifications. Image-based systems use stationary face images, whereas video-based systems use videos for temporal or multiple-instance information. 2D-based systems use typical 2D imaging or image-analysis techniques, and 3D-based systems use 3D imaging or information about face shape, such as depth and curvature. In all these categories, the systems operate under either constrained sensing with cooperative subjects, such as scanning a driver’s license or passport photo, or unconstrained sensing with uncooperative subjects, as in video surveillance.1

Face recognition tasks

As Figure 1 shows, most FRSs perform seven main tasks. Figure 1a shows the enrollment stage, which starts with face acquisition, during which the FRS acquires an image of an individual’s face. Face detection and face normalization involve localizing the acquired face and normalizing its appearance. Finally, in feature extraction, the FRS obtains a feature set to be used as a face template, which it stores in the database along with an identifier. In Figure 1b, which shows the recognition stage, the FRS repeats the feature acquisition, detection, normalization, and extraction steps, but this time rather than storing the feature set, it performs matching, in which it compares it against the stored templates and then attempts to make a decision about whether or not the new feature set is a match to one of the templates.

Representative algorithms

Aside from their classification category, FRSs differ according to the face recognition methods they use, which fall roughly into four types.1,2 Table 1 lists some examples along with the year they first appeared in the literature.2 (Details are available at viswww.cs.umass.edu/lfw/results.html and www.face-rec.org.)