Abstract

Developmental prosopagnosia (DP) is a condition that indicates the inability to recognize individuals by their faces from birth, without any history of brain damage. The assessment of face recognition ability and diagnosis of DP involve the use of face tests such as the Cambridge Face Memory Test (CFMT) and the Cambridge Face Perception Test, along with self-reported measures like the 20-Item Prosopagnosia Index (PI20). Face recognition accuracy is affected by anxiety. However, previous studies on the relationship between face recognition ability and anxiety have not used the PI20 measure. This study aimed to investigate the relationship between self-reported measures of face recognition ability and anxiety tendencies among healthy young individuals for DP diagnosis and its implications. We used a face recognition test, involving the PI20, CFMT, Visual Perception Test for Agnosia–Famous Face Test (VPTA–FFT), and State-Trait Anxiety Inventory (STAI). We assessed the performance of 116 Japanese young adults (75 females, median age of 20.7 years, with a standard deviation of 1.2). Subsequently, we conducted a statistical analysis to examine the relationship between the outcomes of the face recognition tests and STAI scores using Pearson correlation analysis and single correlation coefficients. The results showed a positive correlation between state anxiety and PI20 (r = 0.308, p = 0.007), and a weak positive correlation was also observed between trait anxiety and PI20 (r = 0.268, p = 0.04). In contrast, there was no correlation between CFMT and VPTA–FFT with respect to STAI. The results of the hierarchical multiple regression analysis also suggested that the correlation between the performance on the PI20 (self-report) and objective measures of face recognition performance (the CFMT and the VPTA–FFT) are driven by differences in anxiety. This study is the first to explore the relationship between face recognition abilities and anxiety using the PI20 self-report measure. There are implications for future research on the diagnosis of DP and the relationship between anxiety and face recognition.

Introduction

Face recognition ability enables individuals to differentiate and identify faces, playing a crucial role in interpersonal recognition and everyday communication. Clinical and functional imaging studies have highlighted that brain lesions, particularly in regions such as the fusiform gyrus, can lead to face recognition impairment. In recent years, there has been a growing interest in the field of face recognition research regarding developmental prosopagnosia (DP). DP refers to a condition in which individuals experience lifelong impairments in face recognition despite possessing normal visual and intellectual capabilities since birth. Clinically, there is no indication of overt central nervous system disorders, and neuroimaging studies also reveal the absence of organic brain lesions (Bate & Tree, 2017; Behrmann & Avidan, 2005; Cook & Biotti, 2016; Duchaine & Nakayama, 2006a). Epidemiological studies in Europe and Asia have reported a prevalence rate of approximately 1.9 %–2.9 % of the population (Bowles et al., 2009; Kennerknecht et al., 2006; Kennerknecht et al., 2008). People with DP encounter challenges and psychosocial difficulties in their interpersonal interactions due to impaired face recognition (Yardley et al., 2008). Early diagnosis and support are crucial, especially before adulthood and entering the workforce.

Currently, diagnosing DP presents challenges due to the absence of established strict objective criteria or biological markers (Barton & Corrow, 2016; Susilo & Duchaine, 2013). Previous studies have revealed ambiguous criteria for diagnosing DP and differentiating it from conditions other than DP in face recognition tests (Bate & Tree, 2017; Dalrymple & Palermo, 2016). Dalrymple and Palermo (2016) provided guidelines that define DP as objectively poor performance on face memory tests and a subjective feeling of repeated face recognition failures in daily life. The authors emphasized the importance of using questionnaires and interviews to assess face recognition difficulties in daily life and highlighted the limitations of relying solely on questionnaires for DP diagnosis, as individuals with DP may lack accurate insight into their own face recognition abilities.

Conclusions

The present study is the first to investigate the relationship between self-reported face recognition abilities and anxiety in young, healthy Japanese individuals using the self-perception face recognition test PI20. The findings indicated that unlike the objective face recognition tests CFMT and VPTA–FFT that showed no association with anxiety, the self-perception test scores may be influenced by state anxiety. Self-reporting, at least in its current form, provides limited insights into predicting face recognition performance in young individuals and diagnosing DP. Therefore, self-reporting is insufficient to evaluate face recognition abilities in young individuals, and DP cannot be diagnosed solely based on self-perception tests. It is necessary to combine self-perceived indicators of face recognition impairment with other perception-based face recognition function tests and consider the presence and extent of anxiety tendencies. Our findings have significant psychological implications for understanding face recognition impairments, DP diagnosis, and clinical practices. In the future, expanding research to investigate the interaction between facial and expression recognition abilities and anxiety is expected to enhance the accurate diagnosis and support for young individuals with facial recognition impairments.

Abstract

Face recognition systems that do not implement liveness detection are susceptible to face spoofing attacks. This vulnerability implies that an attacker could disguise themselves as another individual and the system would falsely take the attendance of that other individual. To prevent these attacks, a liveness detection step can be implemented before recognizing subjects. Face recognition-based attendance system devices are typically installed at the entrance to an event or space, so having a portable device that can be easily relocated is practical and efficient. Hence, face recognition systems should be lightweight enough to be able to run on portable devices with limited computational power. Implementing liveness detection will increase the system’s processing time. Therefore, this study aims to develop a lightweight liveness detection method that can be run on a Raspberry Pi. To achieve this, several pre-trained models were evaluated and MobileNetV2 was chosen based on the results. The MobileNetV2 model was then trained using transfer learning method. The proposed attendance system achieved an average processing time below 0.6 s and 96 % accuracy for live subjects, 79 % accuracy for level A spoof attacks, 83.7 % accuracy for level B spoof attacks, and 70 % accuracy for level C spoof attacks.

Introduction

There are two types of attendance systems, manual attendance systems, and automated attendance systems [1]. With manual attendance system, the attendance of each individual present in an event is recorded manually by someone in charge. In an automated attendance system, the tedious process of manually recording everyone’s attendance is replaced with an automated system. One way to achieve this is by using an authentication method to verify and record individuals’ attendance. Fingerprint, palm veins, face, and iris recognition are often used due to their false rejection rate and higher acceptance rate [2]. Face recognition is preferred over other biometric authentication methods because of its inherent benefits such as non-intrusive interaction and accessibility [1].

Biometric authentication methods are robust and convenient because it verifies the identity of the subject by using their physiological and/or behavioral characteristics and not the subject’s knowledge or possession. Password-based authentication is based on knowledge, which implies that if someone knows the knowledge, they can authenticate as someone else. Ownership-based authentication is based on item possession, which means that someone who owns the object can authenticate as someone else. While password-based authentication and ownership-based authentication have their own security concerns, biometric authentication such as face recognition also has its own security concerns.

Face recognition, one of the biometric identification techniques, uses an analysis of an individual’s distinctive facial features to confirm their identity. In most cases, this involves capturing an image or a video of the person’s face, after which algorithms are used to extract and evaluate particular facial features, such the distance between the eyes, the curves of the nose, and the jawline. To see if there is a match, these features are then put in comparison with a database of recognized faces. Face recognition systems are vulnerable to face spoofing attacks, where the attacker would try to gain illegitimate access by presenting a fake face of an authorized individual [2].

Conclusions

Face recognition systems, especially those that do not implement liveness detection, are vulnerable to face spoofing attacks where attackers would try to gain access by presenting a fake face of another individual. On face recognition-based attendance systems, liveness detection is used to prevent individual attendance that is not really present to be taken. Attendance systems typically run on a portable device as it is more practical and efficient because it could be relocated to any appropriate location as needed. Having a limitation of computing power on portable devices, implementing liveness detection will be a challenge as the liveness detection method that will be implemented must be lightweight so the attendance system could still be run on portable devices. By adding a liveness detection step, the processing time of each face presented to the system will be impacted.

In this study, several pre-trained CNN models were used to perform experiments. The pre-trained models used were pre-trained models trained for face recognition and object recognition. Transfer learning with several variations of new layers was performed on MobileNetV2, FaceNet, and MobileFaceNet. From the results of the experiment, variation C obtained the best result. While MobileFaceNet has a lower average processing time, MobileNetV2 was chosen because the average processing time difference is small, has better accuracy on CelebA-Spoof dataset which has a larger range of variations and has decent accuracy on NUAA dataset.

1- Introduction

2- Related works

3- Robust coding based supervised sparse representation

4- Sparsity and robustness of our model

5- Experiment results

6- Conclusions

References

Abstract

Sparse representation based classification (SRC) has become a popular methodology in face recognition in recent years. One widely used manner is to enforce minimum -norm on coding coefficient vector, which is considered as an unsupervised sparsity constraint and usually requires high computational cost. On the other hand, supervised sparsity representation based method (SSR) realizes sparse representation classification with higher efficiency by multiple phases of representing a probe. Nevertheless, since previous SSR methods only deal with Gaussian noise, they cannot satisfy empirical face recognition application which faces wide variations. In this paper, we propose a robust supervised sparse representation (RSSR) model, which uses two-phase of robust representation to compute a sparse coding vector. Huber loss is employed as the fidelity term in the linear representation, which improves the competitiveness of correct class in the first phase. Then training samples with weak competitiveness are removed by supervised way. In the second phase, the competitiveness of correct class is further boosted by Huber loss. We compare the RSSR with other state-of-the-art methods under different conditions, including illumination variations, gesture changes, expressions, corruptions, and occlusions. Comprehensive experiments on four open databases demonstrate the robustness of RSSR and competitive performance is obtained in dealing with face images with occlusion or not.

introduction

Using biometric identification technology for verifying identity is gaining more importance, and different techniques have been developed such as Palmprint recognition[1][2] and face recognition(FR)[3][4]. FR[5][6][7]has been extensively studied for its broad application prospects in recent years, such as authentication and payment system. The primary task of FR consists of feature extraction and classification[8][9][10]. For many classifiers, feature extraction that tends to discover discriminative feature is very important, which has great influence on recognition rate. Since there is rich redundancy in a face image, low dimensional features are extracted to concisely represent the samples in training set[11][12][13][14]. So that using these features can alleviate the computational cost and improve the recognition performance of classifiers. For empirical applications of FR, various changes including lighting, expression, pose, and occlusion can be seen in a probe and which could not included in training set, which leads to the consequence that the computed feature becomes inefficient. Image segmentation can be used to help the effective representation of an image by retaining the informative parts of images[15][16].

1- Introduction

2- Related works

3- Method

4- Experiments

5- Conclusions

References

Abstract

Unconstrained face recognition still remains a challenging task due to various factors such as pose, expression, illumination, partial occlusion, etc. In particular, the most significant appearance variations are stemmed from poses which leads to severe performance degeneration. In this paper, we propose a novel Deformable Face Net (DFN) to handle the pose variations for face recognition. The deformable convolution module attempts to simultaneously learn face recognition oriented alignment and identity-preserving feature extraction. The displacement consistency loss (DCL) is proposed as a regularization term to enforce the learnt displacement fields for aligning faces to be locally consistent both in the orientation and amplitude since faces possess strong structure. Moreover, the identity consistency loss (ICL) and the pose-triplet loss (PTL) are designed to minimize the intra-class feature variation caused by different poses and maximize the inter-class feature distance under the same poses. The proposed DFN can effectively handle pose invariant face recognition (PIFR). Extensive experiments show that the proposed DFN outperforms the state-of-the-art methods, especially on the datasets with large poses.

INTRODUCTION

Face recognition, as a fundamental problem in computer vision, has received more and more attentions in recent years. Equipped with powerful convolutional neural networks (CNNs), the accuracy has a rapid boost that face recognition under controlled settings (i.e., near-frontal poses, neutral expressions, normal illuminations, etc.) seems to be solved. However, under the uncontrolled environment, a number of factors (e.g., pose, illumination, resolution, occlusion, and expression) significantly affect the performance of face recognition system. Among these factors, self-occlusion from out-plane poses brings about large appearance variations. The misalignment problem heavily hurts the face recognition system. In this paper, we further push the frontier of this research area by simultaneously considering face recognition oriented alignment and identitypreserving feature extraction under deep neural networks, which aims at tackling the pose-invariant face recognition (PIFR) problem.

1- Introduction

2- Related work

3- The proposed method

4- Experiments

5- Conclusions

References

Abstract

Face images in real Closed-Circuit Television (CCTV) are usually with low resolution, which remarkably deteriorates the performance of existing face recognition algorithms and hinders the application of face recognition. The main technical focus of this issue, matching between high-resolution (HR) and low-resolution (LR) face images has attracted significant attention. In order to better address this problem, we propose a Classifier Shared Deep Network with Multi-Hierarchy Loss (CS-MHL-Net) for low-resolution face recognition (LRFR) in this paper. Firstly, considering that contrastive loss and its variants are not conducive to the convergence of network and the reduction of discrepancy, a shared classifier between HR and LR face images is proposed to further narrow the domain gap between HR and LR by sharing the corresponding weights which can be seen as the class center. Secondly, to fully exploit intermediate features and loss constraints, we embed multi-hierarchy loss into intermediate layers, with the target of reducing the distances between HR and LR intermediate features after max pooling and avoiding the decreasing of accuracy caused by over-utilization of intermediate features. Experimental results on LFW and SCface demonstrate the effectiveness and superiority of the proposed method.

Introduction

Convolutional neural networks (CNNs) have achieved great success in many fields such as object classification [1, 2], scene understanding [3, 4], and action recognition [5]. Most importantly, CNNs have greatly improved the perfor mance of face recognition [6, 7, 8, 9] in recent years, which laid the foundation for face recognition in real applications. Current accuracy of the-state-ofthe-art face recognition algorithms has achieved more than 99% on the LFW database [10]. However, in reality, the qualities of images captured by surveillance videos are severely affected by different image resolutions. The recognition accuracy dropped severely when identifying extremely low-resolution images. In this paper, we will focus on improving the performance of low-resolution face recognition (LRFR) which has made progress and many more [11, 12, 13, 14, 15, 16, 17, 18]. This paper focuses on the matching problem between low-resolution (LR) face images and high-resolution (HR) face images. How to make the network extracting discriminative features of LR face images and narrowing the domain gap between HR and LR are the main directions to improve the performance of LRFR. There are many traditional works [14, 19, 20, 13, 21, 22, 23, 24, 18, 15, 25, 26] making contributions to the improvement of LRFR. Some of these works [14, 13, 21, 19] focus on transforming the LR images to HR images and promoting the recognition accuracy through the reconstructed LR images. The other works [19, 20, 22, 23, 24, 18, 15, 25, 26] pay more attention to the process of extracting the LR features and narrow the distances between LR features and HR features.

1- Introduction

2 Related studies

3- Materials and methods

4- Experimental results and discussion

5- Conclusion

References

Abstract

In recent years, the Internet-of-Things (IoT) technology is being used in many application areas such as healthcare, video surveillance, transportation etc. The massive adoption and growth of IoT in these areas are generating a massive amount of data. For example, IoT devices such as cameras are generating a huge amount of images when used in hospital surveillance scenarios. Here, face recognition is an important element that can be used for securing hospital facilities, emotion detection and sentiment analysis of patients, detecting patient fraud, and hospital traffic pattern analysis. Automatic and intelligent face recognition systems have high accuracy in a controlled environment; however, they have low accuracy in an uncontrolled environment. Also, the systems need to operate in real-time in many applications such as smart healthcare. This paper suggests a tree-based deep model for automatic face recognition in a cloud environment. The proposed deep model is computationally less expensive without compromising the accuracy. In the model, an input volume is split into several volumes, where a tree is constructed for each volume. A tree is defined by its branching factor and height. Each branch is represented by a residual function, which is constituted by a convolutional layer, a batch normalization, and a non-linear function. The proposed model is evaluated in various publicly available databases. A comparison of performance is also done with state-of-the-art deep models for face recognition. The results of the experiments demonstrate that the proposed model achieved accuracies of 98.65%, 99.19%, 95.84% on FEI, ORL, and LFW databases, respectively.

Introduction

The introduction of many Internet of Things (IoT) and smart body sensors has increased the volume of data significantly in recent years. The nature of the data is heterogeneous and sparse in most of the cases. The processing of Big Data is a matter of concern for realtime applications [1–3]. Consider a scenario where a person is to be recognized in an airport where there are many sensor cameras. These cameras capture images of many objects including humans in their focus areas, and these are capturing images continuously. This huge amount of image data should be processed in a meaningful way in a cloud environment so that a specified person can easily be recognized. Face recognition is one of the oldest yet a dynamic topics of research. It is necessary for security and biometric applications. Early face recognition systems relied on manual features and traditional classifiers. Some hand-crafted features include local binary pattern (LBP), Weber local descriptors (WLD), principal component analysis (PCA), and histogram of oriented gradients (HOG). Traditional classifiers include support vector machines (SVM), linear discriminant analysis (LDA), and some minimum distant-based classifiers. These features and classifiers work well in a controlled environment, where faces are mostly frontal and with a neutral expression, and having less variation of illumination. However, in many applications such as those related to surveillance, face images may be occluded, not frontal, and having low resolution and high variation of illumination. For these applications, the traditional face recognition systems may not work properly.

1- Introduction

2- Related work

3- The proposed method

4- Discussion

5- Experiments

6- Conclusions

References

Abstract

Increasing inter-class variance and shrinking intra-class distance are two main concerns and efforts in face recognition. In this paper, we propose a new loss function termed inter-class angular margin (IAM) loss aiming to enlarge the inter-class variance. Instead of restricting the inter-class margin to be a constant in existing methods, our IAM loss adaptively penalizes smaller inter-class angles more heavily and successfully makes the angular margin between classes larger, which can significantly enhance the discrimination of facial features. The IAM loss can be readily introduced as a regularization term for the widely-used Softmax loss and its recent variants to further improve their performances. We also analyze and verify the appropriate range of the regularization hyper-parameter from the perspective of backpropagation. For illustrative purposes, our model is trained on CASIA-WebFace and tested on the LFW, CFP, YTF and MegaFace datasets; the experimental results show that the IAM loss is quite effective to improve state-of-the-art algorithms.

Introduction

Convolutional neural networks (CNNs) are widely used for face recognition [1–15], in which recent researches have been focused on increasing the inter-class variance and reducing the intra-class distance. A typical pipeline of using a network for training WebFace can be found in Fig. 1, in which the network is trained with the loss function in the last layer, and the representation in the penultimate layer is used as the feature of human faces. Hence the recent efforts and achievements in increasing the interclass variance and reducing the intra-class distance can be summarized into two categories. First, to optimize the Euclidean distance between facial features, mainly through regularization. For example, the Triplet loss [6] makes the intra-class Euclidean distance of features shorter than the interclass distance. Wen et al. [16] reduce the intra-class Euclidean distance by adding an extra penalty. The Marginal loss of [17] and our past work [18] limit both intra-class and inter-class Euclidean distances to improve recognition accuracy. The Range loss [19] overcomes the problem of long-tailed data by equalizing intra-class Euclidean distance and increasing inter-class Euclidean distance. Except for the Triplet loss, all above methods add a regularization term on the basis of the Softmax loss, which is generally adjusted via a regularization hyper-parameter.

1- Introduction

2- From softmax loss to minimum margin loss

3- Experiments

4- Conclusion

References

Abstract

Face recognition has achieved great success owing to the fast development of deep neural networks in the past few years. Different loss functions can be used in a deep neural network resulting in different performance. Most recently some loss functions have been proposed, which have advanced the state of the art. However, they cannot solve the problem of margin bias which is present in class imbalanced datasets, having the so-called long-tailed distributions. In this paper, we propose to solve the margin bias problem by setting a minimum margin for all pairs of classes. We present a new loss function, Minimum Margin Loss (MML), which is aimed at enlarging the margin of those overclose class centre pairs so as to enhance the discriminative ability of the deep features. MML, together with Softmax Loss and Centre Loss, supervises the training process to balance the margins of all classes irrespective of their class distributions. We implemented MML in Inception-ResNet-v1 and conducted extensive experiments on seven face recognition benchmark datasets, MegaFace, FaceScrub, LFW, SLLFW, YTF, IJB-B and IJB-C. Experimental results show that the proposed MML loss function has led to new state of the art in face recognition, reducing the negative effect of margin bias.

Introduction

In the past ten years, deep neural network (DNN) based methods have achieved great progress in various computer vision tasks, including face recognition [1], person re-identification [2], object detection [3] and action recognition [4]. The progress on face recognition is particularly remarkable due largely to two important factors – larger face datasets and better loss functions. The quantity and quality of the face datasets used for training directly influence the performance of a DNN model in face recognition. Currently, there are a few large-scale face datasets that are publicly available, for example, MS-Celeb-1M [5], VGGFace2 [6], MegaFace [7] and CASIA WebFace [8]. As shown in Table 1, CASIA WebFace consists of 0.5M face images; VGGFace2 contains totally 3M face images but only from 9K identities; MS-Celeb-1M and MegaFace both contain more images and more identities, thus should have greater potential for training a better DNN model. However, both MS-Celeb-1M and MegaFace have the problem of long-tailed distribution [9], which means a minority of people owns a majority of face images and a large number of people have very limited face images. Using datasets with long-tailed distribution, the trained model tends to overfit the classes with rich samples thus weakening the generalisation ability on the longtailed portion [9]. Specifically, the classes with rich samples tend to have a relatively large margin between their class centres; conversely, the classes with limited samples tend to have a relatively small margin between their class centres as they only occupy a small region in space and are thus easy to be compressed. This margin bias problem is due to long-tailed class distribution, which leads to performance drop on face recognition [9].

1- Introduction

2- Previous works

3-  Proposed method

4- Experimental evaluation

5- Discussion and conclusion

References

Abstract

We propose a novel coupled mappings method for low resolution face recognition using deep convolutional neural networks (DCNNs). The proposed architecture consists of two branches of DCNNs to map the high and low resolution face images into a common space with nonlinear transformations. The branch corresponding to transformation of high resolution images consists of 14 layers and the other branch which maps the low resolution face images to the common space includes a 5-layer super-resolution network connected to a 14-layer network. The distance between the features of corresponding high and low resolution images are backpropagated to train the networks. Our proposed method is evaluated on FERET, LFW, and MBGC datasets and compared with state-of-the-art competing methods. Our extensive experimental evaluations show that the proposed method significantly improves the recognition performance especially for very low resolution probe face images (5% improvement in recognition accuracy). Furthermore, it can reconstruct a high resolution image from its corresponding low resolution probe image which is comparable with the state-of-the-art super-resolution methods in terms of visual quality.

Introduction

In the past few decades, face recognition has shown promising performance in numerous applications and under challenging conditions such as occlusion (Jia & Martinez, 2009), variation in pose, illumination, and expression (Martínez, 2002). While many face recognition systems have been developed for recognizing high quality face images in controlled conditions (Zhao, Chellappa, Phillips, & Rosenfeld, 2003), there are a few studies focused on face recognition in real world applications such as surveillance systems with low resolution faces (Pnevmatikakis & Polymenakos, 2007). One important challenge in these applications is that high resolution (HR) probe images may not be available due to the large distance of the camera from the subject. Here, we focus on addressing the problem of recognizing low resolution probe face images when a gallery of high quality images is available. There are three standard approaches to address this problem. (1) down sampling the gallery images to the resolution of the probe images and then performing the recognition. However, this approach is suboptimal because the additional discriminating information available in the high resolution gallery images is lost.

1- Introduction

2- Related work

3- Proposed RGB-D face recognition approach

4- Experimental results

5- Conclusion

References

Abstract

In this article we present a novel RGB-D learned local representations for face recognition based on facial patch description and matching. The major contribution of the proposed approach is an efficient learning and combination of data-driven descriptors to characterize local patches extracted around image reference points. We explored the complementarity between both of deep learning and statistical image features as data-driven descriptors. In addition, we proposed an efficient high-level fusion scheme based on a sparse representation algorithm to leverage the complementarity between image and depth modalities and also the used data-driven features. Our approach was extensively evaluated on four well-known benchmarks to prove its robustness against known challenges in the case of face recognition. The obtained experimental results are competitive with the state-of-the-art methods while providing a scalable and adaptive RGB-D face recognition method.

Introduction

Face recognition for an automated person identification has received great attention over the years as it offers the most user-friendly and non-invasive modality. Face recognition based on standard two dimensional (2-D) images was extensively studied but it still suffers from problems related to imaging conditions and face pose variations. Thanks to the progress in three-dimensional (3-D) technology, recent research has shifted from 2-D to 3-D (Abbad et al., 2018). Indeed, 3-D face representation ensures a reliable surface shape description and adds geometric shape information to the face characterization. Most recently, some researchers proposed to use image and depth data captured from cost-effective RGB-D sensors like MS Kinect or Intel RealSense instead of bulky and expensive 3-D scanners. In addition to color images, RGB-D sensors provide depth maps describing the scene 3-D shape by active vision or an alternative technology. Driven by the emergence of this type of sensors and the latest advances in deep learning techniques, RGB-D face recognition is now becoming at the heart of several recent research studies. Indeed, it is nowadays crystal clear that data-driven feature extraction, using Convolutional Neural Networks (CNNs) for example, outperforms traditional hand-crafted features for many computer vision tasks like object detection (Szegedy et al., 2013), image clas sification (Krizhevsky et al., 2012), etc. When it comes to the RGB-D face recognition, the observed challenges basically deal with face pose variations, partial occlusions, imaging conditions, and discriminant feature extraction.

1- Introduction

2- Related work

3- Deep eigen-filters and DEFNet for feature representation

4- Experiments and results

5- Analysis on the strategy of proposed deep eigen-filters approach

6- Conclusion

References

Abstract

Training deep convolutional neural networks (CNNs) often requires high computational cost and a large number of learnable parameters. To overcome this limitation, one solution is computing predefined convolution kernels from training data. In this paper, we propose a novel three-stage approach for filter learning alternatively. It learns filters in multiple structures including standard filters, channel-wise filters and point-wise filters which are inspired from variations of CNNs’ convolution operations. By analyzing the linear combination between learned filters and original convolution kernels in pre-trained CNNs, the reconstruction error is minimized to determine the most representative filters from the filter bank. These filters are used to build a network followed by HOG-based feature extraction for feature representation. The proposed approach shows competitive performance on color face recognition compared with other deep CNNs-based methods. Besides, it provides a perspective of interpreting CNNs by introducing the concepts of advanced convolutional layers to unsupervised filter learning.

Introduction

With the development of deep learning in recent years, deep neural networks, especially deep convolutional neural networks (CNNs) have achieved state-of-the-art performance in many image-based applications [1], e.g., image classification [2, 3], face recognition [4, 5], fine-grained image categorization [6, 7] and depth estimation [8, 9]. Compared with traditional visual recognition methods, CNNs have the advantage of learning both low-level and high-level feature representations automatically instead of designing hand-crafted feature descriptors [10, 11]. Due to these powerful features, CNNs have revolutionized the computer vision community and become one of the most popular tools in many visual recognition tasks [7, 12, 13]. Generally, CNNs are made up of three types of layers, i.e. convolutional layers, pooling layers, and fullyconnected layers. The features are extracted by stacking many convolutional layers on top of each other, and backpropagation starts from the loss function and goes back to the input in order to learn the weights and biases contained in the layers. However, how this kind of mechanism works on images remains an open question and yet needs to be explored. Besides, learning powerful feature representations requires a large amount of labeled training data otherwise the performance may deteriorate [14, 15], whereas training data in practical applications are often not readily available. To solve these problems, some researchers propose learning convolutional layers alternatively independent of training data. In [16], ScatNet was proposed by using wavelet transforms to represent convolutional filters. These predefined wavelet transforms are cascaded with nonlinear and pooling operations to build a multilayer convolutional network. Therefore, no learning is needed in computing image representation. Different from ScatNet, researchers in [17] introduced a structured receptive field network that combines the flexible learning property of CNNs and the fixed basis filters.

1- Introduction

2- The proposed MSPRFL

3- Experiments and discussions

4- Conclusions

References

In practical video surveillance, the quality of facial regions of interest is usually affected by the large distances between the objects and surveillance cameras, which undoubtedly degrade the recognition performance. Existing methods usually consider the holistic representations, while neglecting the complementary information from different patch scales. To tackle this problem, this paper proposes a multi-scale patch based representation feature learning (MSPRFL) scheme for low-resolution face recognition problem. Specifically, the proposed MSPRFL approach first exploits multi-level information to learn more accurate resolution-robust representation features of each patch with the help of a training dataset. Then, we exploit these learned resolution-robust representation features to reduce the resolution discrepancy by integrating the recognition results from all patches. Finally, by considering the complementary discriminative ability from different patch scales, we try to fuse the multi-scale outputs by learning scale weights via an ensemble optimization model. We further verify the efficiency of the proposed MSPRFL on low-resolution face recognition by the comparison experiments on several commonly used face datasets.

Introduction

Face image recognition, as one of the most commonly used biometrics technologies, has become the research hotspot of the pattern recognition community in past decades [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]. Generally, most of the current methods perform well on the cases that the acquired region of interest (ROI) has high image resolution and contains enough discriminative information for recognition tasks. However, in real-world robotics and video surveillance applications, the observed faces generally have low-resolution (LR) together with pose and illumination variations, while the referenced faces are always enrolled with high resolution (HR). The main challenge is to match an LR probe face with limited details against HR gallery faces. We name this kind of problem as low-resolution face recognition (LRFR). An alternative solution is down-sampling the HR galleries and then matching in the same resolution space. In this way, the resolution discrepancy is reduced at the expense of losing the discriminative facial details in the gallery. Generally, there are two typical categories to address the LRFR problem. One is superresolution approaches, which first synthesize the target HR faces from the observed LR image, and then utilize traditional face recognition approaches in the common resolution domain. The other is resolution-robust feature extraction methods, which directly extract discriminative features from respective domains, thus obtain better performance than superresolution methods.

1- Introduction

2- Related work

3- Preliminary

4- Clustering based domain adaptation

5- Experiments

6- Conclusion

References

Abstract

Despite great progress in face recognition tasks achieved by deep convolution neural networks (CNNs), these models often face challenges in real world tasks where training images gathered from Internet are different from test images because of different lighting condition, pose and image quality. These factors increase domain discrepancy between training (source domain) and testing (target domain) database and make the learnt models degenerate in application. Meanwhile, due to lack of labeled target data, directly fine-tuning the pre-learnt models becomes intractable and impractical. In this paper, we propose a new clustering-based domain adaptation method designed for face recognition task in which the source and target domain do not share any classes. Our method effectively learns the discriminative target feature by aligning the feature domain globally, and, at the meantime, distinguishing the target clusters locally. Specifically, it first learns a more reliable representation for clustering by minimizing global domain discrepancy to reduce domain gaps, and then applies simplified spectral clustering method to generate pseudo-labels in the domain-invariant feature space, and finally learns discriminative target representation. Comprehensive experiments on widely-used GBU, IJB-A/B/C and RFW databases clearly demonstrate the effectiveness of our newly proposed approach. State-of-the-art performance of GBU data set is achieved by only unsupervised adaptation from the target training data.

Introduction

Benefiting from convolutional neural networks (CNNs) [1, 2, 3, 4, 5], deep face recognition (FR) has been the most efficient biometric technique for identity authentication and has been widely used in enormous areas such as military, finance, public security as well as our daily life. However, deep networks which perform perfectly on benchmark datasets may fail badly on real world applications. This is because the set of real world images is infinitely large and so it is hard for any dataset, no matter how big, to be representative of the complexity of the real world. One persuasive evidence is presented by P.J. Phillips’ study [6] which conducted a cross benchmark assessment of VGG model [7] for face recognition. The VGG model, trained on over 2.6 million face images of celebrities from the Web, is a typical FR systems and achieves 98.95% on LFW [8] and 97.30% on YTF [9]. However, It only obtains 26%, 52% and 85% on Ugly, Bad and Good partition of GBU database, even if all of images in GBU are nominally frontal. The main reason is a different distribution between training data (source domain) and testing data (target domain), referred to as domain or covariate shift. Visual examples of this domain shift are shown in Fig. 1. Each dataset in Fig. 1 displays a unique “signature” and thus one can easily distinguish them only by these signatures, which proves the existence of significant discrepancies. The images in CASIA-WebFace [10] are collected from Internet under unconstrained environment and most of the figures are celebrities and public taken in ambient lighting; The GBU [11] contains still frontal facial images and is taken outdoors or indoors in atriums and hallways with digital camera; IJB-A [12] covers large pose variations and contains many blurry video frames. Sometimes, the images of GBU and IJB-A datasets may be closer to the ones in real life which are taken with digital camera under different shooting environments and contain larger variations.