۱- Introduction

۲- Principles of kernel methods and set-membership techniques

۳- Proposed centroid-based set-membership kernel normalized least-mean-square algorithm

۴- Proposed nonlinear regression-based SM-KNLMS algorithm

۵- Analysis

۶- Simulations

۷- Conclusions

References

Abstract

In the last decade, a considerable research effort has been devoted to developing adaptive algorithms based on kernel functions. One of the main features of these algorithms is that they form a family of universal approximation techniques, solving problems with nonlinearities elegantly. In this paper, we present data-selective adaptive kernel normalized least-mean square (KNLMS) algorithms that can increase their learning rate and reduce their computational complexity. In fact, these methods deal with kernel expansions, creating a growing structure also known as the dictionary, whose size depends on the number of observations and their innovation. The algorithms described herein use an adaptive step-size to accelerate the learning and can offer an excellent tradeoff between convergence speed and steady state, which allows them to solve nonlinear filtering and estimation problems with a large number of parameters without requiring a large computational cost. The data-selective update scheme also limits the number of operations performed and the size of the dictionary created by the kernel expansion, saving computational resources and dealing with one of the major problems of kernel adaptive algorithms. A statistical analysis is carried out along with a computational complexity analysis of the proposed algorithms. Simulations show that the proposed KNLMS algorithms outperform existing algorithms in examples of nonlinear system identification and prediction of a time series originating from a nonlinear difference equation.

Introduction

Adaptive filtering algorithms have been the focus of a great deal of research in the past decades and the machine learning community has embraced and further advanced the study of these methods. In fact, adaptive algorithms are often considered with linear structures, which limits their performance and does not draw attention to nonlinear problems that can be solved in various applications. In order to deal with nonlinear problems a family of nonlinear adaptive algorithms based on kernels has been developed. In particular, a kernel is a function that compares the similarity between two inputs and can be used for filtering, estimation and classification tasks. Kernel adaptive filtering (KAF) algorithms have been tested in many different scenarios and applications [1], [2], [3], [4], [5], showing very good results. One of the main advantages of KAF algorithms is that they are universal approximators [1], which gives them the ability to address complex and nonlinear problems. However, their computational complexity is much higher than their linear counterparts [1]. One of the first KAF algorithms to appear, which is widely adopted in the KAF family because of its simplicity, is the kernel least-mean square (KLMS) algorithm proposed in [6] and later extended in [7]. The KLMS algorithm has been inspired by the least-mean square (LMS) algorithm and, thanks to its good performance, led many researchers to work in the development of kernel versions of conventional adaptive algorithms. For instance, a kernel version of the NLMS algorithm has been proposed in [5] using a nonlinear regression approach for time series prediction. In [8], [9], the affine projection algorithm (APA) has been used as the basis of the derivation of kernel affine projection (KAP) algorithms. Adaptive projection algorithms using kernel techniques have been reported in [10], [11]. The recursive least squares algorithm (RLS) has been extended in [12], where the kernel recursive least squares (KRLS) has been described. Later, the authors of [13] proposed an extension of the KRLS algorithm and the use of multiple kernels has been studied in [14] and [15].