Abstract

     The detection of the contact area formed between a human finger and a counter surface is of great interest because it is the key parameter for various interaction parameters. Adhesional friction forces and the thermal contact conductance critically depend on the contact area, further influencing the tactile sensation of stickiness and warmth. The contact area is also of concern regarding safety issues. Injuries caused by objects slipping out of our hands might be prevented by optimizing the contact area and the concomitant grip through appropriate surface structures and material choice. Until now the contact area is mainly studied on smooth and transparent materials. The contact area is recorded optically and rule-based image processing methods can be used for detection. These methods might be insufficient for rough surfaces where the contact area is optically unclear due to light scattering. In this paper we demonstrate the successful analysis of such optically unclear contact area images via convolutional neural networks to identify the fingerprint ridges in contact with structured surfaces. The proposed method relies on the generation of synthetic contact images that provide the pixelwise ground truth for the efficient training of a segmentation pipeline based on convolutional neural networks.

Introduction

     The contact area formed between a human finger and a counter surface is the key parameter for a variety of interaction processes. The contact area is the main factor for adhesional friction forces, which are dominant for many finger/counter surface systems (Derler and Gerhardt (2012)). The frictional forces determine to a large extent how sticky the surface is felt (van Kuilenburg, Masen, and van der Heide (2015)). In addition, an understanding of the contact area and its evolution with time could possibly minimize the danger of severe injuries imposed by objects that slip a person’s hand (Dzidek et al. (2017)). Furthermore, the contact area affects the heat flux determining the sensation of warmth when touching a material (van Kuilenburg, Masen, and van der Heide (2015)). So far, the contact area has been mainly studied on flat transparent counter surfaces using optical methods. The majority of the literature relies on light scattering (Gruber, Winkler, and Resch (2015)) and Frustrated Total Internal Reflection (FTIR) to produce a high-contrast image of the contact areas. Different optical set-ups have been employed to take advantage of the FTIR to study the dynamics of a fingertip during the tactile exploration of smooth and transparent counter surfaces (André et al. (2011); Bochereau et al. (2017); Delhaye et al.

Conclusion

     In this contribution we present a new approach that enables the analysis of the contact area formed between a human finger and a rough surface in sliding contact. We propose an experimental testing site that allows the continuous acquisition of corresponding images, showing not only the apparent contact area with the outlines of the finger but also the apparent finger pad ridges contact area. While on smooth surfaces a segmentation of the finger pad ridges is possible with rule-based image processing techniques, this is not the case for rough surfaces. For this reason, we investigated the segmentation of finger pad ridges via convolutional neural networks (CNNs). Since pixel-wise labeling of the finger pad ridges is difficult and extremely time-consuming we propose the use of synthetic contact images that were generated by fusing images of rough surfaces with contact images on smooth transparent surfaces. Since for the latter a ground truth based on rule-based image processing is available, this is also the case for the generated synthetic images. We further proposed a segmentation pipeline consisting of two independently trained CNNs. While the first network was trained to detect the gross contact area, the second network was trained to segment the finger pad ridges within the gross contact area. We further demonstrate that the pipeline can be trained using only 10–۲۰ training images without showing notable overfitting on the validation images.