Recently deep learning has become dominant in face recognition and many other artificial intelligence areas. We raise a question: Can deep learning truly solve the face recognition problem? If not, what is the challenge for deep learning methods in face recognition? We think that the face image quality issue might be one of the challenges for deep learning, especially in unconstrained face recognition. To investigate the problem, we partition face images into different qualities, and evaluate the recognition performance, using the state-of-the-art deep networks. Some interesting results are obtained, and our studies can show directions to promote the deep learning methods towards high-accuracy and practical use in solving the hard problem of unconstrained face recognition.

INTRODUCTION

Deep learning (DL) [1], [2] has recently become dominant in a wide variety of biometrics problems and many other artificial intelligence (AI) areas. One of the greatest successes of DL has been in face recognition (FR) where the accuracies have been improved greatly over the traditional methods [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]. We raise a question: Can deep learning really solve the face recognition problem? Or, can we say that, given the great success of DL, the face recognition problem has been solved or almost solved? To answer this question, and get a better understanding of the performance of DL methods in FR, we perform an empirical study with designed experiments accordingly. In face recognition, it is well-known that the hard problem is unconstrained face matching, in which we believe that the face image quality variations are probably the biggest issue that makes the problem hard. Based on this view, our conjecture is that the face image quality issue may still be a grand challenge even for the recently developed DL methods. In FR with traditional features, it is well-known that the face image quality has a big influence on recognition accuracy; In DL features, however, a large dataset with face images of different quality for each subject, is used to train the deep models. Will the quality still be an issue? In our empirical study, we design the face recognition experiments with matching across different face image qualities, which is seldom done in an explicit way in previous face recognition approaches. In practice, however, one can meet the cross-quality face matching problem frequently. For example, in the FBI’s interstate photo system (IPS), millions This was partly supported by a NSF-CITeR grant and a WV HEPC grant. of mugshot photos could be matched to face images collected from social media web sites where the wild photos may have a wide variety of qualities. Since there is no existing face database, which is purposely assembled with annotated face image qualities, we partition some recent, public face databases into different face image qualities, using an automated face image quality assessment method. After the quality partition, the face images from the same subject are divided into different qualities, such as low, middle, and high. Then we can perform face recognition experiments across quality changes. For the deep learning techniques, we select some representatives of the state-of-the-art. To avoid any bias in training and parameter tuning, we adopted the already-trained face models that have reported very high accuracies in the popular face database LFW (labeled faces in the wild) [16].