Published papers

This paper elucidates the grouping effect of the sparse stable outlier-robust regression (S-SORR) estimator which exploits the minimax concave (MC) penalty and the Tikhonov regularization simultaneously together with the MC loss. The main theoretical result is the following: where μ 1 > 0$ is the regularization parameter, and ai and aj are the unit vectors with their associated coefficients hat xi and hat xj. Remarkably, the bound is independent of possible outliers which may be contained in the observation vector y, whereas the bound for the popular elastic net estimator increases in proportion to the norm of y which is largely affected by outliers. Numerical examples show that S-SORR extracts the group structure correctly under huge outliers.

The contributions of this paper are twofold. The first is to give theoretical motivation for whitening in the recently proposed adaptive filtering algorithm named the Krylov-proportionate normalized least-mean-square (KPNLMS) algorithm. The second is to present the details of whitening in KPNLMS (In the original work of KPNLMS, the whitening procedure is mentioned but is not described in detail). An interesting connection among the transform-domain adaptive filter (TDAF), proportionate normalized cast-mean-square (PNLMS), and KPNLMS algorithms is also provided. Numerical examples demonstrate that KPNLMS drastically outperforms TDAF especially in noisy situations.

The adaptive parallel subgradient projection (PSP) algorithm was proposed in 2002 as a set-theoretic adaptive filtering algorithm providing fast and stable convergence, robustness against noise, and low computational complexity by using weighted parallel projections onto multiple time-varying closed half-spaces. In this paper, we present a novel weighting technique named pairwise optimal weight realization (POWER) for further acceleration of the adaptive PSP algorithm. A simple closed-form formula is derived to compute the projection onto the intersection of two closed half-spaces defined by a triplet of vectors. Using the formula inductively, the proposed weighting technique realizes a good direction of update. The resulting weights turn out to be pairwise optimal in a certain sense. The proposed algorithm has the inherently parallel structure composed of q primitive functions, hence its total computational complexity O(qrN) is reduced to O(rN) with q concurrent processors (r: a constant positive integer). Numerical examples demonstrate that the proposed technique for r = 1 yields significantly faster convergence than not only adaptive PSP with uniform weights, affine projection algorithm, and fast Newton transversal filters but also the regularized recursive least squares algorithm.

In this paper, we propose an efficient design of proportionality factors in the recently established algorithm named Krylov-proportionate normalized least mean-square (KPNLMS), which is an extention of the PNLMS algorithm to nonsparse (or dispersive) unknown systems by means of a Krylov subspace. The designing task takes a form of minimizing the number of iterations that is needed for an upper bound of the system mismatch to reach a specified target value. The minimization is performed under several constraints related to numerical stability, computational requirements, and nonnegativity, and its closed-form solution is derived. Numerical examples demonstrate that the proposed design significantly reduces the number of iterations needed to achieve target values of system mismatch especially when a low level of system mismatch is required.

It is well-known that the Capon beamformer is sensitive to array steering vector errors and may result into a worse performance than classical data-independent beamformers. This paper follows a different path from the well-established Diagonal Loading techniques and designs a robust Capon beamformer by a recent extension of the Adaptive Projected Subgradient Method. The proposed method arks a computational complexity of O(N-2), where N is the number of array elements. The simulation results show that the proposed beamformer achieves excellent performance especially in cases where the Diagonal Loading techniques face difficulties, i.e. in cases where the Interference to Noise Ratio (INR) is moderately larger than SNR.

We propose a robust approach to recovering the jointly-sparse signals in the presence of outliers. We formulate the recovering task as a minimization problem involving three terms: (i) the minimax concave (MC) loss function, (ii) the MC penalty function, and (iii) the squared Frobenius norm. The MC-based loss and penalty functions enhance robustness and group sparsity, respectively, while the squared Frobenius norm induces the convexity. The problem is solved, via reformulation, by the primal-dual splitting method, for which the convergence condition is derived. Numerical examples show that the proposed approach enjoys remarkable outlier robustness.

In this paper, we propose a novel reduced-rank adaptive filtering algorithm exploiting the Krylov subspace associated with estimates of certain statistics of input and output signals. We point out that, when the estimated statistics are erroneous (e. g., due to sudden changes of environments), the existing Krylov-subspace-based reduced-rank methods compute the point that minimizes a "wrong" mean-square error (MSE) in the subspace. The proposed algorithm exploits the set-theoretic adaptive filtering framework for tracking efficiently the optimal point in the sense of minimizing the "true" MSE in the subspace. Therefore, compared with the existing methods, the proposed algorithm is more suited to adaptive filtering applications. A convergence analysis of the algorithm is performed by extending the adaptive projected subgradient method (APSM). Numerical examples demonstrate that the proposed algorithm enjoys better tracking performance than the existing methods for system identification problems.

In this article, we propose a fast and efficient algorithm named the adaptive parallel Krylov-metric projection algorithm. The proposed algorithm is derived from the variable-metric adaptive projected subgradient method, which has recently been presented as a unified analytic tool for various adaptive filtering algorithms. The proposed algorithm features parallel projection-in a variable-metric sense-onto multiple closed convex sets containing the optimal filter with high probability. The metric is designed based on (i) sparsification by means of a certain data-dependent Krylov subspace and (ii) maximal use of the obtained sparse structure for fast convergence. The numerical examples show the advantages of the proposed algorithm over the existing ones in stationary/nonstationary environments. Copyright (C) 2011 John Wiley & Sons, Ltd.

In this paper, we propose a novel reduced-rank adaptive filtering algorithm based on set-theoretic adaptive fltering. We discuss the orthonormatity of the transformation (rank-reduction) matrix. We present, under the assumption that the transformation matrix has an orthonormal structure, an interpretation of the proposed algorithm in the original (full-size) vector space. The interpretation suggests that the use of an orthonormal transformation matrix leads to performance depending solely on the subspace spanned by the column vectors of the matrix but not on the matrix itself. This is verified by simulations, and the numerical examples demonstrate the efficacy of the proposed algorithm.

The goal of this paper is to establish a novel signal processing paradigm that enables us to find a point meeting time-variable specifications in dual domain (e.g., time and frequency domains) simultaneously. For this purpose, we define a new problem which we call adaptive split feasibility problem (ASFP). In the ASFP formulation, we have (i) a priori knowledge based convex constraints in the Euclidean spaces R-N and R-M and (ii) data-dependent convex sets in R-N and R-M; the latter are obtained in a sequential fashion. Roughly speaking, the problem is to find a common point of all the sets defined on R-N such that its image under a given linear transformation is a common point of all the sets defined on R-M, if such a point exists. We prove that the adaptive projected subgradient method (APSM) deals with the ASFP by employing (i) a projected gradient operator with respect to (w.r.t.) a 'fixed' proximity function reflecting the convex constraints and (ii) a subgradient projection w.r.t. 'time-varying' objective functions reflecting the data-dependent sets. The resulting algorithm requires no unwanted operations such as matrix inversion, therefore it is suitable for real-time implementation. A convergence analysis is presented and verified by numerical examples.

We propose a novel formulation for stable sparse recovery from measurements contaminated by outliers and severe noise. The proposed formulation evaluates noise and outliers with a quadratic function and the minimax concave function, respectively, to reflect their statistical properties (Gaussianity and sparsity). This makes a significant difference from the conventional robust methods, which typically evaluate noise and outliers with a single loss function, leading to stability of the estimate. While the proposed formulation involves a nonconvex penalty to reduce estimation biases of sparse estimates, overall convexity of the whole cost is guaranteed under a certain condition by adding the Tikhonov regularization term. The problem is solved via a reformulation by the forward-backward primal-dual splitting algorithm, for which convergence conditions are derived. The remarkable outlier-robustness of the proposed method is demonstrated by simulations under highly noisy environments.

This paper presents a novel framework for sparse robust signal recovery integrating the sparse recovery using the minimax concave (MC) penalty and robust regression called sparse outlier-robust regression (SORR) using the MC loss. While the proposed approach is highly robust against huge outliers, the sparseness of estimates can be controlled by taking into consideration a tradeoff between sparseness and robustness. To accommodate the prior information about additive Gaussian noise and outliers, an auxiliary vector to model the noise is introduced. The remarkable robustness and stability come from the use of the MC loss and the squared \ell {2} penalty of the noise vector, respectively. In addition, the simultaneous use of the MC and squared \ell {2} penalties of the coefficient vector leads to a certain remarkable grouping effect. The necessary and sufficient conditions for convexity of the smooth part of the cost are derived under a certain nonempty-interior assumption via the product space formulation using the linearly-involved Moreau-enhanced-over-subspace (LiMES) framework. The efficacy of the proposed method is demonstrated by simulations in its application to speech denoising under highly noisy environments as well as to toy problems.

We propose a novel sparsity-aware adaptive filtering algorithm based on iterative use of weighted soft-thresholding. The weights are determined based on a rough local approximation of the l(p) norm (0 < p < 1). The proposed algorithm operates the weighted soft-thresholding for enhancing the sparsity, following estimation error managements with the affine projection. The proposed weighting technique alleviates an extra bias of no benefit caused by shrinking dominant coefficients. The numerical examples demonstrate that the proposed weighting technique outperforms the existing one when the situation changes under the fixed parameter settings.

We propose an efficient regression method which is highly robust against outliers and stable even in the severely noisy situations. The robustness here comes from the adoption of the minimax concave loss, while the stability comes from separate treatments of the outlier and noise by an introduction of an auxiliary vector modeling the Gaussian noise. We present a necessary and sufficient condition for convexity of the smooth part of the entire cost under a certain assumption, where a general model is used with its potential use for other applications envisioned. We show that the proposed formulation can be solved via reformulation by the forward-backward-based primal-dual method under the convexity condition. The numerical examples show the remarkable robustness of the proposed estimator under highly noisy situations.

An active-period-aware supervised nonnegative matrix factorization (NMF) approach for music transcription is proposed. Supervised NMF relies on a set of known spectrograms associated with all musical instruments that may possibly be involved with given music data; this is supported by the availability of large database of a variety of musical instruments. It is free from the source-number determination problem and this is a significant advantage over the unsupervised NMF approaches. The proposed approach is composed of three steps. Step 1: Apply the existing supervised NMF algorithm. Step 2: Estimate the 'active' periods (during which musical sounds are present) based on the outcomes of Step 1. Step 3: Optimize a refined cost function reflecting the estimate of active periods. The awareness of active periods leads to avoidance of the so-called octave-errors which is a central issue of the existing supervised NMF method. Simulation results show the efficacy of the proposed approach.(1)

In this paper, we present two novel approaches to the issue of exploiting multiple metrics jointly for efficient adaptive filtering. The key is the introduction of product-space formulation for taking into account multiple metrics in a single Hilbert space. The first approach is based on the Pierra's idea of reformulating the problem of finding a common point of multiple closed convex sets as a problem of finding a common point of two closed convex sets in a product space. The second approach is a slight modification of the first one along the idea of constraint-embedding. An interesting relation between our approaches and the improved proportionate normalized least mean square (IPNLMS) algorithm is provided. The monotone approximation properties of the two algorithms are also presented. A numerical example suggests the efficacy of the presented multimetric strategy.

The autocorrelation matrix of the kernelized input vector is well approximated by the squared Gram matrix (scaled down by the dictionary size). This holds true under the condition that the input covariance matrix in the feature space is approximated by its sample estimate based on the dictionary elements, leading to a couple of fundamental insights into online learning with kernels. First, the eigenvalue spread of the autocorrelation matrix relevant to the hyperplane projection along affine subspace algorithm is approximately a square root of that for the kernel normalized least mean square algorithm. This clarifies the mechanism behind fast convergence due to the use of a Hilbertian metric. Second, for efficient function estimation, the dictionary needs to be constructed in general by taking into account the distribution of the input vector, so as to satisfy the condition. The theoretical results are justified by computer experiments.

This paper presents a mathematically rigorous analysis of linearly constrained adaptive filtering algorithms based on the adaptive projected subgradient method. We provide the novel concept of constraint-embedding functions that enables to analyze certain classes of linearly constrained adaptive algorithms in a unified manner. Trajectories of the linearly constrained adaptive filters always lie in the affine constraint set, a translation of a closed subspace. Based on this fact, we translate all the points on the constraint set to its underlying subspace which we regard as a Hilbert space thereby making the analysis feasible. Derivations of the linearly constrained adaptive filtering algorithms are finally presented in connection with the analysis.

We present a rigorous deterministic analysis of the variable-metric adaptive filtering algorithms (including the transform-domain, LMS/Newton, and proportionate adaptive filters) by using the framework of variable-metric adaptive projected subgradient method (Yukawa et al. 2007). Under small metric-fluctuations, we present the useful properties of (i) monotone approximation - with respect to a certain constant metric - indicating the stability of the algorithm and (ii) convergence to an asymptotically optimal point. Numerical examples show the advantage of the variable-metric adaptive filtering algorithms and suggest the validity of the analysis.

In this paper, we propose a novel stochastic gradient algorithm for efficient adaptive filtering. The basic idea is to sparsity the initial error vector and maximize the benefits from the sparsification under computational constraints. To this end, we formulate the task of algorithm-design as a constrained optimization problem and derive its (non-trivial) closed-form solution. The computational constraints, are formed by focusing on the fact that the energy of the sparsified error vector concentrates at the first few components. The numerical examples demonstrate that the proposed algorithm achieves the convergence as fast as the computationally expensive method based on the optimization without the computational constraints.