Published papers

This paper introduces a non-parametric learning framework to combat outliers in online, multi-output, and nonlinear regression tasks. A hierarchical-optimization problem underpins the learning task: Search in a reproducing kernel Hilbert space (RKHS) for a function that minimizes a sample average ℓp-norm (1 ≤ p ≤ 2) error loss defined on data contaminated by noise and outliers, under affine constraints defined as the set of minimizers of a quadratic loss on a finite number of faithful data devoid of noise and outliers (side information). To surmount the computational obstacles inflicted by the choice of loss and the potentially infinite dimensional RKHS, approximations of the ℓp-norm loss, as well as a novel twist of the criterion of approximate linear dependency are devised to keep the computational-complexity footprint of the proposed algorithm bounded over time. Numerical tests on datasets showcase the robust behavior of the advocated framework against different types of outliers, under a low computational load, while satisfying at the same time the affine constraints, in contrast to the state-of-the-art methods which are constraint agnostic.

We propose a novel stochastic-optimization framework based on the regularized dual averaging (RDA) method. The proposed approach differs from the previous studies of RDA in three major aspects. First, the squared-distance loss function to a 'random' closed convex set is employed for stability. Second, a sparsity-promoting metric (used implicitly by a certain proportionate-type adaptive filtering algorithm) and a quadratically-weighted \ell -1 regularizer are used simultaneously. Third, the step size and regularization parameters are both constant due to the smoothness of the loss function. These three differences yield an excellent sparsity-seeking property, high estimation accuracy, and insensitivity to the choice of the regularization parameter. Numerical examples show the remarkable advantages of the proposed method over the existing methods (including AdaGrad and the adaptive proximal forward-backward splitting method) in applications to regression and classification with real/synthetic data.

The relaxed zero-forcing (RZF) beamformer is a quadratically-and-linearly constrained minimum variance beamformer. The central question addressed in this paper is whether RZF performs better than the widely-used minimum variance distortionless response and zero-forcing beamformers under temporally-correlated interference. First, RZF is rederived by imposing an ellipsoidal constraint that bounds the amount of interference leakage for mitigating the intrinsic gap between the output variance and the mean squared error (MSE) which stems from the temporal correlations. Second, an analysis of RZF is presented for the single-interference case, showing how the MSE is affected by the spatio-temporal correlations between the desired and interfering sources as well as by the signal and noise powers. Third, numerical studies are presented for the multiple-interference case, showing the remarkable advantages of RZF in its basic performance as well as in its application to brain activity reconstruction from EEG data. The analytical and experimental results clarify that the RZF beamformer gives near-optimal performance in some situations.

We propose a robust approach to recovering jointly sparse signals in the presence of outliers. The robust recovery task is cast as a convex optimization problem involving a minimax concave loss function (which is weakly convex) and a strongly convex regularizer (which ensures the overall convexity). The use of the nonconvex loss makes the problem difficult to solve directly by the convex optimization methods even with the well-established firm shrinkage. We circumvent this difficulty by reformulating the problem via the Moreau decomposition so that the objective function becomes a sum of convex functions that can be minimized by the primal-dual splitting method. The parameter designs/ranges for the present specific case are derived to ensure the convergence. We demonstrate the remarkable robustness of the proposed approach against outliers by extensive simulations to the application of multi-lead electrocardiogram as well as synthetic data.

Spectral clustering is an empirically successful approach to separating a dataset into some groups with possibly complex shapes based on pairwise affinity. Identifying the number of clusters automatically is still an open issue, although many heuristics have been proposed. In this paper, imposing sparsity on the eigenvectors of graph Laplacian is proposed to attain reasonable approximations of the so-called cluster-indicator-vectors, from which the clusters as well as the cluster number are identified. The proposed algorithm enjoys low computational complexity as it only computes a relevant subset of eigenvectors. It also enjoys better clustering quality than the existing methods, as shown by simulations using nine real datasets.

We previously proposed a joint learning scheme of Gaussian parameters (scales and centers) and coefficients for online nonlinear estimation. The instantaneous squared error cost in terms of the Gaussian scales, however, tends to have shallow slopes when the initial guess is far from optimal, causing extremely slow convergence. In this paper, we propose steepening the cost function by adding a squared distance function from the instantaneously-optimal scale. Numerical examples show that the use of the steepened cost ameliorates the convergence behaviors of the scale parameters in inappropriate initial-scale settings.