Open-set activity recognition remains as a challenging problem because of complex activity diversity. In previous works, extensive efforts have been paid to construct a negative set or set an optimal threshold for the target set. In this paper, a model based on Generative Adversarial Network (GAN), called ‘OpenGAN’ is proposed to address the open-set recognition without manual intervention during the training process. The generator produces fake target samples, which serve as an automatic negative set, and the discriminator is redesigned to output multiple categories together with an ‘unknown’ class. We evaluate the effectiveness of the proposed method on measured micro-Doppler radar dataset and the MOtion CAPture (MOCAP) database from Carnegie Mellon University (CMU). The comparison results with several state-of-the-art methods indicate that OpenGAN provides a promising open-set solution to human activity recognition even under the circumstance with few known classes. Ablation studies are also performed, and it is shown that the proposed architecture outperforms other variants and is robust on both datasets.

Introduction

In the last decades, human activity recognition has aroused general concern in numerous fields [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12] due to the increasing demand for entertainment, medical monitoring, security, emergency rescue and other ap5 plications. Human activity recognition has relied on visual data for decades and develops rapidly with the help of computer vision. Most available studies focus on analyzing human activities based on image sequences captured by optical sensors. Confronting severe interference caused by the weather, object shelter and light condition, optical sensors are unsuitable for recording activities in 10 the severe environment. Compared with optical sensors, radar has some unique properties. It is invariable to the environment and is capable of detecting objects through the wall. What’s more, radar could operate long-distance detections regardless of shelters. Radar-based perception has broad application prospects such as survival search, enemy status perception, and terrorist detection. 15 Human activity recognition using radar data aims at automatically recognizing human motions from radar spectrograms. When radar echoes are modulated by human activities, there will be micro-Doppler signatures motivated by micromovements. Micro-Doppler frequency varies with the velocity of a moving target so that each movement has its unique micro-Doppler signatures, which can be 20 used for activity recognition. Conventional Doppler-based activity recognition requires well-designed features from micro-Doppler spectrogram to support an effective training process of classifiers [13, 14]. Substantial efforts have been dedicated to developing discriminating visual feature descriptors [15], and deep learning is employed due to its superior ability of automatically learning from 25 data. Deep learning has achieved significant success in the field of image classification, object detection and semantic segmentation [16, 17, 18, 19, 20, 21, 22].