Published papers

This paper presents an efficient robust method to learn sparse graphs from contaminated data. Specifically, the convex-analytic approach using the minimax concave penalty is formulated using the so-called gamma-lasso which exploits the gamma-cross entropy. We devise a weighting technique which designs the data weights based on the l(1) distance in addition to the Mahalanobis distance for avoiding possible failures of outlier rejection due to the combinatorial graph Laplacian structure. Numerical examples show that the proposed method significantly outperforms gamma-lasso and tlasso as well as the existing non-robust graph learning methods in contaminated situations.

We study the problem of robust online multiuser detection in the context of non-orthogonal multiple access. The optimal multiuser detector (in terms of the uncoded bit error rate) is the nonlinear maximum a posteriori (MAP) filter. Learning good nonlinear functions of this type (e.g., with deep neural networks) typically requires a large number of training samples and complicated signal processing, which poses a fundamental problem in dynamic wireless environments. Furthermore, compared with linear approaches, nonlinear filters are generally less robust against changes in the environment. To overcome these problems, we first show that the optimal MAP filter belongs to reproducing kernel Hilbert spaces (RKHSs) associated with Gaussian kernels whose widths satisfy a condition related to the standard deviation of the receiver noise. Second, we show how to approximate the optimal MAP filter with a computationally simple signal processing algorithm using a relatively small number of training samples. Third, to make the filter robust against changes in the wireless environment, we design a partially linear filter in the sum of an RKHS containing the MAP filter and an RKHS of a linear kernel. Finally, based on this partially linear design, we propose a multiuser detection framework that, in contrast to some state-of-the-art approaches, has the following desired features: (i) it has low complexity; (ii) it can work with small sample sets; (iii) it shows better robustness than a purely nonlinear receiver; and (iv) it does not require user parameter estimation (e.g., channels), which is prone to errors and may not be possible in certain scenarios.

A color-tone similarity index (CSIM) between two color images is presented and another index, picture similarity index (PSIM), is also given for a comprehensive similarity comparison between color images. CSIM is defined by a statistical analysis of cumulative histograms in a hue-oriented color space. It characterizes the color distributions, while the existing structural similarity index reflects the spatial structure involved with grayscale images. The behaviors of CSIM are checked by the comparisons of color code chips. Experimental results are given. The proposed indexes combined with SSIM are hopeful to provide a tool for color image quality analysis (IQA).

In stereophonic acoustic echo cancellation (SAEC) problem, fast and accurate tracking of echo path is strongly required for stable echo cancellation. In this paper, we propose a class of efficient fast SAEC schemes with linear computational complexity (with respect to filter length). The proposed schemes are based on pairwise optimal weight realization (POWER) technique, thus realizing a "best" strategy (in the sense of pairwise and worst-case optimization) to use multiple-state information obtained by preprocessing. Numerical examples demonstrate that the proposed schemes significantly improve the convergence behavior compared with conventional methods in terms of system mismatch as well as echo return loss enhancement (ERLE).

This paper presents an efficient robust method to learn sparse graphs from contaminated data. Specifically, the convex-analytic approach using the minimax concave penalty is formulated using the so-called ?-lasso which exploits the ?-cross entropy. We devise a weighting technique which designs the data weights based on the l1 distance in addition to the Mahalanobis distance for avoiding possible failures of outlier rejection due to the combinatorial graph Laplacian structure. Numerical examples show that the proposed method significantly outperforms ?-lasso and t-lasso as well as the existing non-robust graph learning methods in contaminated situations.

The Lebesgue spectrum filter (LSF), a finite impulse response (FIR) filter whose coefficients decay exponentially with a negative factor r:=root 3-2, is shown to be effective preprocessing for spreading code in asynchronous code-division multiple-access (CDMA) systems. The LSF has only been studied independently from the well-known minimum mean-square error (MMSE) filter, an optimal FIR filter in the mean-square error sense. In this paper, we propose an efficient structure, employing the LSF at the transmitter and the MMSE filter at the receiver, for asynchronous CDMA systems. We employ a spreading code preprocessed by the LSF (referred to as LSF-code), and the LSF-code supplies a "best" initial estimate (among the ones obtained without any a priori information) to an adaptive algorithm for the MMSE filter, leading to significant reduction of iterations in adaptation. This is verified by computer simulations. Also we investigate the link between the LSF and the MMSE filter by examining their autocorrelation properties. Copyright (C) 2008 Masahiro Yukawa et al.

This paper addresses the problem of estimating the angular power spectrum (APS) of massive multiple input multiple output wireless channels. Estimating the APS is useful, for instance, for simplifying the downlink channel estimation problem in frequency division duplex systems. We propose an efficient online algorithm that estimates the APS from the channel spatial covariance matrix. The proposed algorithm approximates the APS as a sum of Gaussian functions and leverages the framework of multikernel adaptive filtering.

This work proposes to apply directional lapped orthogonal transforms to image restoration. A DirLOT is an orthonormal transform of which basis is allowed to be anisotropic with the symmetric, real-valued and compact-support property. In this work, DirLOTs are used to generate symmetric orthonormal discrete wavelet transforms and then a redundant dictionary as a union of unitary transforms. The multiple directional property is suitable for representing natural images which contain diagonal edges and textures. The performances of deblurring, super-resolution and inpainting are evaluated for several images with the iterative-shrinkage/thresholding algorithm. It is verified that the proposed dictionary yields comparable or superior restoration performance to the non-subsampled Haar transform.

This paper proposes a novel adaptive filtering scheme named the Krylov-proportionate normalized least-mean-square (KPNLMS) algorithm. KPNLMS exploits the benefits (i.e., fast convergence for sparse unknown systems) of the proportionate NLMS algorithm, but its applications are not limited to sparse unknown systems. A set of orthonormal basis vectors is generated from a certain Krylov sequence. It is proven that the unknown system is sparse with respect to the basis vectors in case of fairly uncorrelated input data. Different adaptation gain is allocated to a coefficient of each basis vector, and the gain is roughly proportional to the absolute value of the corresponding coefficient of the current estimate. KPNLMS enjoys i) fast convergence, ii) linear complexity per iteration, and iii) no use of any a priori information. Numerical examples demonstrate significant advantages of the proposed scheme over the reduced-rank method based on the multistage Wiener filter (MWF) and the transform-domain adaptive filter (TDAF) both in noisy and silent situations.

This paper proposes a fast converging adaptive filtering algorithm named Krylov-proportionate normalized least mean-square (KPNLMS) by extending the proportionate normalized least mean square (PNLMS) algorithm. PNLMS is known to exhibit faster convergence than the standard NLMS algorithm for sparse unknown systems. The proposed algorithm attains similar effects for non-sparse unknown systems by constructing, based on the multistage Wiener filter (MWF) representation, an orthonormal basis with which the unknown system has a sparse structure. The proposed algorithm can be analyzed by the adaptive parallel variable-metric projection framework. Numerical studies for non-sparse unknown systems are presented, comparing KPNLMS and the MWF-based reduced-rank method.

The l(p)-constrained least squares, which is denoted by (P-c), for 0 < p < 1 is addressed. A maximal continuous curve of its critical solutions beta(c) for different bounds c forms a critical path which can be constructed with the variational method. The path is a piecewise smooth single-valued function of c, containing non-optimal points such as saddle points and local maxima in general as well as global minima. The path of global minima may coincide with a critical path but may jump from a critical path to another one. The breakpoints of the greedy path (a critical path constructed with a certain greedy selection criterion) coincide with the step-by-step solutions generated by the orthogonal matching pursuit (OMP). A critical point of (P-c) is also a critical point of the l(p)-penalized least squares (Q(lambda)) which reformulates (P-c) with the Lagrangian multiplier. The greedy path is a multi-valued function of lambda and is formed by a collection of multiple critical paths of (Q(lambda)).

This paper presents a convex-analytic framework to learn sparse graphs from data. While our problem formulation is inspired by an extension of the graphical lasso using the so-called combinatorial graph Laplacian framework, a key difference is the use of a nonconvex alternative to the `1 norm to attain graphs with better interpretability. Specifically, we use the weakly-convex minimax concave penalty (the difference between the `1 norm and the Huber function) which is known to yield sparse solutions with lower estimation bias than `1 for regression problems. In our framework, the graph Laplacian is replaced in the optimization by a linear transform of the vector corresponding to its upper triangular part. Via a reformulation relying on Moreau’s decomposition, we show that overall convexity is guaranteed by introducing a quadratic function to our cost function. The problem can be solved efficiently by the primal-dual splitting method, of which the admissible conditions for provable convergence are presented. Numerical examples show that the proposed method significantly outperforms the existing graph learning methods with reasonable computation time.

We present an efficient mathematical framework to derive promising methods that enjoy 'enhanced' desirable properties. The popular minimax concave penalty for sparse modeling subtracts, from the ℓ 1 norm, its Moreau envelope, inducing nearly unbiased estimates and thus yielding considerable performance enhancements. To extend it to underdetermined linear systems, we propose the projective minimax concave penalty, which leads to 'enhanced' sparseness over the input subspace. We also present a promising regression method which has an 'enhanced' robustness and substantial stability by distinguishing outlier and noise explicitly. The proposed framework, named the linearly-involved Moreau-enhanced-over-subspace (LiMES) model, encompasses those two specific examples as well as two others: stable principal component pursuit and robust classification. The LiMES function involved in the model is an 'additively nonseparable' weakly convex function, while the 'inner' objective function to define the Moreau envelope is 'separable'. This mixed nature of separability and nonseparability allows an application of the LiMES model to the underdetermined case with an efficient algorithmic implementation. Two linear/affine operators play key roles in the model: one corresponds to the projection mentioned above and the other takes care of robust regression/classification. A necessary and sufficient condition for convexity of the smooth part of the objective function is studied. Numerical examples show the efficacy of LiMES in applications to sparse modeling and robust regression.

This paper addresses the minimal subset selection of antennas achieving designated channel capacity. This is one of the most natural approaches to alleviating the power consumption in MIMO systems, while it is a mathematically challenging nonlinearly-constrained sparse optimization (l(0)-norm minimization) problem. We present an efficient algorithmic solution, to this highly combinatorial problem, using convex and differentiable relaxations of the l(0)-norm. The proposed algorithm is based on the hybrid steepest descent method for the subgradient projection operator together with the soft-thresholding technique, minimizing the Moreau envelope of the l(1)-norm subject to the capacity constraint. The simulation results show that the proposed algorithm realizes a near optimal solution to the original nonlinearly-constrained sparse optimization problem.

The first aim of this paper is to present a useful toolbox of quasi-nonexpansive mappings for convex optimization from the viewpoint of using their fixed point sets as constraints. Many convex optimization problems have been solved through elegant translations into fixed point problems. The underlying principle is to operate a certain quasi-nonexpansive mapping T iteratively and generate a convergent sequence to its fixed point. However, such a mapping often has infinitely many fixed points, meaning that a selection from the fixed point set Fix(T) should be of great importance. Nevertheless, most fixed point methods can only return an "unspecified" point from the fixed point set, which requires many iterations. Therefore, based on common sense, it seems unrealistic to wish for an "optimal" one from the fixed point set. Fortunately, considering the collection of quasi-nonexpansive mappings as a toolbox, we can accomplish this challenging mission simply by the hybrid steepest descent method, provided that the cost function is smooth and its derivative is Lipschitz continuous. A question arises: how can we deal with "nonsmooth" cost functions?The second aim is to propose a nontrivial integration of the ideas of the hybrid steepest descent method and the Moreau-Yosida regularization, yielding a useful approach to the challenging problem of nonsmooth convex optimization over Fix(T). The key is the use of smoothing of the original nonsmooth cost function by its Moreau-Yosida regularization whose derivative is always Lipschitz continuous. The field of application of hybrid steepest descent method can be extended to the minimization of the ideal smooth approximation over Fix(T). We present the mathematical ideas of the proposed approach together with its application to a combinatorial optimization problem: the minimal antenna-subset selection problem under a highly nonlinear capacity-constraint for efficient multiple input multiple output (MIMO) communication systems.

We propose multi-domain adaptive filtering based on the idea of feasibility splitting - dealing with feasibility in individual domains. The proposed approach provides a useful and mathematically rigorous framework to incorporate multiple pieces of information (expressed in different domains) efficiently. Indeed, it processes such multiple pieces of information by means of the metric projection in each individual domain; this is a significant advantage over existing single-domain approaches. Also we provide a reasonable strategy to treat the case where the available prior information is inconsistent. A convergence analysis and numerical examples are presented to support the proposed method.

We propose the multi-domain adaptive learning that enables us to find a point meeting possibly time-varying specifications simultaneously in multiple domains. e.g. space, time, frequency, etc. The novel concept is based on the idea of feasibility splitting - dealing with feasibility in each individual domain. We show that the adaptive projected subgradient method (Yamada, 2003) realizes the multi-domain adaptive learning by employing (i) a projected gradient operator with respect to a 'fixed' proximity function reflecting the time-invariant specifications and (ii) a subgradient projection with respect to 'time-varying' objective functions reflecting the time-varying specifications. The resulting algorithm is Suitable for real-time implementation, because it requires no more than metric projections onto closed convex sets each of which accommodates the specification in each domain. A convergence analysis and numerical examples are presented.

This paper exemplifies that the use of multiple kernels leads to efficient adaptive filtering for nonlinear systems. Two types of multikernel adaptive filtering algorithms are proposed. One is a simple generalization of the kernel normalized least mean square (KNLMS) algorithm [2], adopting a coherence criterion for dictionary designing. The other is derived by applying the adaptive proximal forward-backward splitting method to a certain squared distance function plus a weighted block norm penalty, encouraging the sparsity of an adaptive filter at the block level for efficiency. The proposed multikernel approach enjoys a higher degree of freedom than those approaches which design a kernel as a convex combination of multiple kernels. Numerical examples show that the proposed approach achieves significant gains particularly for nonstationary data as well as insensitivity to the choice of some design-parameters.

In this paper, we propose a novel approach using multiple kernels to nonlinear adaptive filtering problems. We present two types of multi-kernel adaptive filtering algorithms, both of which are based on the kernel normalized least mean square (KNLMS) algorithm (Richard et al., 2009). One is a simple generalization of KNLMS, adopting the coherence criterion for dictionary selection. The other is derived by applying the adaptive proximal forward-backward splitting method to a certain squared distance function penalized by a weighted block l(1) norm. The latter algorithm operates the weighted block soft-thresholding which encourages the sparsity of dictionary at the block level. Numerical examples demonstrate the efficacy of the proposed approach.

In this paper, a novel adaptive channel-equalization scheme using multiple kernels is proposed. The proposed scheme equalizes distorted signals by means of a linear combination of multiple Gaussian functions having different means and variances. The combination coefficients are computed by the multi-kernel normalized least squares with block soft-thresholding (MKNLMS-BT) algorithm. To deal with channel nonstationarity, an efficient technique to control the block soft-thresholding operator itself adaptively is presented, based on the block l(p)-norm (0 < p < 1). The proposed scheme enjoys low computational complexity and global optimality (unlike the Volterra filtering and neural network approaches) as well as high estimation accuracy. Simulation results show the efficacy of the proposed scheme both in stationary and nonstationary environments.