We are interested in the reproducibility of face recognition systems. By reproducibility we mean: is the scientific community, and are the researchers from different sides, capable of reproducing the last published results by a big company, that has at its disposal huge computational power and huge proprietary databases? With the constant advancements in GPU computation power and availability of open-source software, the reproducibility of published results should not be a problem. But, if architectures of the systems are private and databases are proprietary, the reproducibility of published results can not be easily attained. To tackle this problem, we focus on training and evaluation of face recognition systems on publicly available data and software. We are also interested in comparing the best Deep Neural Net (DNN) based results with a baseline ”classical” system. This paper exploits the OpenFace open-source system to generate a deep convolutional neural network model using publicly available datasets. We study the impact of the size of the datasets, their quality and compare the performance to a classical face recognition approach. Our focus is to have a fully reproducible model. To this end, we used publicly available datasets (FRGC, MS-celeb-1M, MOBIO, LFW), as well publicly available software (OpenFace) to train our model in order to do face recognition. Our best trained model achieves 97.52% accuracy on the Labelled in the Wild dataset (LFW) dataset which is lower than Google’s best reported results of 99.96% but slightly better than FaceBook’s reported result of 97.35%. We also evaluated our best model on the challenging video dataset MOBIO and report competitive results with the best reported results on this database.

INTRODUCTION

In the last years, mainly due to the advances of deep learning, more concretely convolutional networks, the quality of image recognition and object detection has been progressing at a dramatic pace. With the advent of GPU computation and big datasets, neural networks saw a huge resurgence. This results in huge improvements in image recognition and consequently face recognition. Many works [1]–[۵] report near perfect biometric performance. But in most cases, all systems are either proprietary or trained on private datasets. This raises the problem of the difficulty of reproducing published results [6]. In this paper we try to reach the best reported results on the Labeled Faces in the Wild (LFW) [7] database, by using the open-source OpenFace [8] software. This software is based on the Google’s FaceNet architecture [5] that achieves the best results on LFW, but is fully proprietary. CMU has already worked in this direction, but their published results of 92.92% are far from the 99.96% that Google got on LFW. We have chosen to exploit the publicly available MS-celeb1M [9] dataset. We evaluate the performance of our newly trained system on the (LFW), as well as the MOBIO [10] dataset (a very challenging audio-visual dataset). The rest of the paper is organized as follows: Section II summarizes the latest achievements of Convolutional Neural Nets and DNN based face recognition. Section III explains our approach to try to reach best published and reproducible results. Our experimental results are given in Section IV, followed by conclusions and perspectives.