Published papers

This paper describes the model proposed and submitted by our RIJP team to SemEval 2020 Task1: Unsupervised Lexical Semantic Change Detection. In the model, words are represented by Gaussian distributions. For Subtask 1, the model achieved average scores of 0.51 and 0.70 in the evaluation and post-evaluation processes, respectively. The higher score in the post-evaluation process than that in the evaluation process was achieved owing to appropriate parameter tuning. The results indicate that the proposed Gaussian-based embedding model is able to express semantic shifts while having a low computational complexity.

We propose a convex-analytic approach to supervised nonnegative matrix factorization (NMF), using the Moreau envelope, a smooth approximation, of the β-divergence as a loss function. The supervised NMF problem is cast as minimization of the loss function penalized by four terms: (i) a time-continuity enhancing regularizer, (ii) the indicator function enforcing the nonnegativity, (iii) a basis-vector selector (a block ℓ1 norm), and (iv) a sparsity-promoting regularizer. We derive a closed-form expression of the proximity operator of the sum of the three non-differentiable penalty terms (ii)–(iv). The optimization problem can thus be solved numerically by the proximal forward–backward splitting method, which requires no auxiliary variable and is therefore free from extra errors. The source number is automatically attained as an outcome of optimization. The simulation results show the efficacy of the proposed method in an application to polyphonic music transcription.

In this paper, we analyze the properties of a fixed point of a certain mapping that is implicitly used in each of the regularized dual averaging (RDA) and projection-based RDA (PDA) algorithms. It turns out that, if the loss function has a nonexpansive (1-Lipschltz) gradient such as in the case of a half squared-distance function, RDA converges to a minimizer of the penalized loss function under a restrictive condition. Meanwhile, the fixed point for PDA gives a minimizer of the 'unpenalized' loss function. Some simulation studies are also presented to support the theoretical findings.

This paper presents a graph learning framework to produce sparse and accurate graphs from network data. While our formulation is in- spired by the graphical lasso, a key difference is the use of a noncon- vex alternative of the ℓ1 norm as well as a quadratic term to ensure overall convexity. Specifically, the weakly-convex minimax concave penalty (MCP) is used, which is given by subtracting the Huber func- tion from the ℓ1 norm, inducing a less-biased sparse solution than ℓ1. In our framework, the graph Laplacian is represented by a lin- ear transform of the vector corresponding to its upper triangular part. Via a reformulation relying on the Moreau decomposition, the prob- lem can be solved by the primal-dual splitting method. An admis- sible choice of parameters for provable convergence is presented. Numerical examples show that the proposed method significantly outperforms its ℓ1-based counterpart for sparse grid graphs.

We propose a hybrid dictionary approach for distributed kernel-based adaptive learning of a nonlinear function by a network of nodes. The hybrid dictionary incorporates a local part to improve learning of high frequency components in the function within the local domain of each node and a global part to provide a consensus estimate of the function over the whole region of interest. We apply our scheme to the reconstruction of a spatial distribution by a network of mobile nodes. Performance evaluations show that high frequency components are reconstructed accurately by our hybrid dictionary approach while common schemes are not able to recover them completely.

We propose a novel automatic shrinkage tuning technique for the adaptive proximal forward-backward splitting (APFBS) algorithm. The shrinkage tuning aims to choose an appropriate value of the shrinkage parameter and achieve minimal system mismatch as possible. The system mismatch is approximated based on time-averaged second-order statistics. Numerical examples show that the proposed method achieves performance fairly close to that with a manually chosen shrinkage parameter for colored input signals at some signal to noise ratio (SNR).

We present a convexly-constrained beamformer design for brain activity reconstruction from non-invasive electroencephalography (EEG) signals. An intrinsic gap between the output variance and the mean squared errors is highlighted that occurs due to the presence of interfering activities correlated with the desired activity. The key idea of the proposed beamformer is reducing this gap without amplifying the noise by imposing a quadratic constraint that bounds the total power of interference leakage together with the distortionless constraint. The proposed beamformer can be implemented efficiently by the multi-domain adaptive filtering algorithm. Numerical examples show the clear advantages of the proposed beamformer over the minimum-variance distortionless response (MVDR) and nulling beamformers.

Motivated by the success of reinforcement learning (RL) for discrete-time tasks such as AlphaGo and Atari games, there has been a recent surge of interest in using RL for continuous-time control of physical systems (cf. many challenging tasks in OpenAI Gym and DeepMind Control Suite). Since discretization of time is susceptible to error, it is methodologically more desirable to handle the system dynamics directly in continuous time. However, very few techniques exist for continuous-time RL and they lack flexibility in value function approximation. In this paper, we propose a novel framework for model-based continuous-time value function approximation in reproducing kernel Hilbert spaces. The resulting framework is so flexible that it can accommodate any kind of kernel-based approach, such as Gaussian processes and kernel adaptive filters, and it allows us to handle uncertainties and nonstationarity without prior knowledge about the environment or what basis functions to employ. We demonstrate the validity of the presented framework through experiments.

This work proposes convolutional-sparse-coded dynamic mode decomposition (CSC-DMD) by unifying extended dynamic mode decomposition (EDMD) and convolutional sparse coding. EDMD is a data-driven method of analysis used to describe a nonlinear dynamical system with a linear time-evolution equation. Compared with existing EDMD methods, CSC-DMD has the advantage of reflecting the spatial structure of a target. As an example, the proposed method is applied to river bed shape estimation from the water surface observation. This estimation problem is reduced to sparsityaware signal restoration with a hard constraint given by the CSC-DMD prediction, where the algorithm is derived by the primal-dual splitting method. A time series set of water surface and bed shape measured through an experimental river setup is used to train and test the system. From the result, the efficacy of the proposed method is verified.

Non-orthogonal multiple access (NOMA) has emerged as a promising radio access technique for enabling the performance enhancements promised by the fifth-generation (5G) networks in terms of connectivity, latency, and spectrum efficiency. In the NOMA uplink, detection based on successive interference cancellation (SIC) with device clustering has been suggested. If the receivers are equipped with multiple antennas, SIC can be combined with minimum mean-squared error (MMSE) beamforming. However, there exists a tradeoff between the NOMA cluster size and the incurred SIC error. Larger clusters lead to larger errors but they are desirable from the spectrum efficiency and connectivity point of view. To enable the deployment of large clusters, we propose a novel online learning detection method for the NOMA uplink. We design an online adaptive filter in the sum space of linear and Gaussian reproducing kernel Hilbert spaces (RKHSs). Such a sum space design is robust against variations of a dynamic wireless network that can deteriorate the performance of a purely nonlinear adaptive filter. We demonstrate by simulations that the proposed method outperforms (symbol level) MMSE-SIC based detection for large cluster sizes.

We propose an adaptive scheme for distributed learning of nonlinear functions by a network of nodes. The proposed algorithm consists of a local adaptation stage utilizing multiple kernels with projections onto hyperslabs and a diffusion stage to achieve consensus on the estimates over the whole network. Multiple kernels are incorporated to enhance the approximation of functions with several high- A nd low-frequency components common in practical scenarios. We provide a thorough convergence analysis of the proposed scheme based on the metric of the Cartesian product of multiple reproducing kernel Hilbert spaces. To this end, we introduce a modified consensus matrix considering this specific metric and prove its equivalence to the ordinary consensus matrix. Besides, the use of hyperslabs enables a significant reduction of the computational demand with only a minor loss in the performance. Numerical evaluations with synthetic and real data are conducted showing the efficacy of the proposed algorithm compared to the state-of-the-art schemes.

We study the use of weakly-convex minmax concave (MC) regularizes in distributed sparse optimization. The global cost function is the squared error penalized by the MC regularizer. While it is convex as long as the whole system is overdetermined and the regularization parameter is sufficiently small, the local cost of each node is usually nonconvex as the system from local measurements are underdetermined in practical applications. The Moreau decomposition is applied to the MC regularizer so that the total cost takes the form of a smooth function plus the rescaled ℓ1 norm. We propose two solvers: the first applies the proximal gradient exact first-order algorithm (PG-EXTRA) directly to our cost, while the second is based on convex relaxation of the local costs to ensure convergence. Numerical examples show that the proposed approaches attain significant gains compared to the ℓ1 -based PG-EXTRA.

We present an exact analytic expression of the contributions of the kernel principal components to the relevant information in a nonlinear regression problem. A related study has been presented by Braun, Buhmann, and Müller in 2008, where an upper bound of the contributions was given for a general supervised learning problem but with 'uncentered' kernel PCAs. Our analysis clarifies that the relevant information of a kernel regression under explicit centering operation is contained in a finite number of leading kernel principal components, as in the 'uncentered' kernel-Pca case, if the kernel matches the underlying nonlinear function so that the eigenvalues of the centered kernel matrix decay quickly. We compare the regression performances of the least-square-based methods with the centered and uncentered kernel PCAs by simulations.

We propose a novel online algorithm for efficient nonlinear estimation. Target nonlinear functions are approximated with 'unfixed' Gaussians of which the parameters are regarded as (a part of) variables. The Gaussian parameters (scales and centers), as well as the coefficients, are updated to suppress the instantaneous squared errors regularized by the ℓ1 norm of the coefficients to enhance the model efficiency. Another point for enhancing the model efficiency is the multiscale screening method, which is a hierarchical dictionary growing scheme to initialize Gaussian scales with multiple choices. To reduce the computational complexity, a certain selection strategy is presented for growing the dictionary and updating the Gaussian parameters. Computer experiments show that the proposed algorithm enjoys high adaptation-capability and produces efficient estimates.

This paper proposes an extension of multichannel non-negative matrix factorization (MNMF) that simultaneously solves source separation and dereverberation. While MNMF was originally formulated under an underdetermined problem setting where sources can outnumber microphones, a determined counterpart of MNMF, which we call the determined MNMF (DMNMF), has recently been proposed with notable success. This approach is particularly notable in that the optimization process can be more than 30 times faster than the underdetermined version owing to the fact that it involves no matrix inversion computations. One drawback as regards all methods based on instantaneous mixture models, including MNMF, is that they are weak against long reverberation. To overcome this drawback, this paper proposes an extension of DMNMF using a frequency-domain convolutive mixture model. The optimization process of the proposed method consists of iteratively updating (i) the spectral parameters of each source using the majorization-minimization algorithm, (ii) the separation matrix using iterative projection, and (iii) the dereverberation filters using multichannel linear prediction. Experimental results showed that the proposed method yielded higher separation performance and dereverberation performance than the baseline method under highly reverberant environments.

Multikernel adaptive filtering is an attractive nonlinear approach to online estimation/tracking tasks. Despite its potential advantages over its single-kernel counterpart, a use of inappropriately weighted kernels may result in a negligible performance gain. In this paper, we propose an efficient recursive kernel weighting technique for multikernel adaptive filtering to activate all the kernels. The proposed weights equalize the convergence rates of all the corresponding partial coefficient errors. The proposed weights are implemented via a certain metric design based on the weighting matrix. Numerical examples show, for synthetic and multiple real datasets, that the proposed technique exhibits a better performance than the manually-tuned kernel weights, and that it significantly outperforms the online multiple kernel regression algorithm.

We propose a novel problem formulation for sparsity-aware adaptive filtering based on the nonconvex minimax concave (MC) penalty, aiming to obtain a sparse solution with small estimation bias. We present two algorithms: the first algorithm uses a single firm-shrinkage operation, while the second one uses double soft-shrinkage operations. The twin soft-shrinkage operations compensate each other, promoting sparsity while avoiding a serious increase of biases. The whole cost function is convex in certain parameter settings, while the instantaneous cost function is always nonconvex. Numerical examples show the superiority compared to the existing sparsity-aware adaptive filtering algorithms in system mismatch and sparseness of the solution.

We propose an efficient adaptive update method for the kernel parameters: the kernel coefficients, scales and centers. The mirror descent and the steepest descent method for squared error cost function are employed to update the kernel scales and centers, respectively. Although the problem considered in this paper is nonconvex, we reduce the possibility of falling into local minima by using a novel multiple initialization scheme to grow the dictionary without great increases of the dictionary size. Through computer experiments, we show that the proposed algorithm enjoys a high adaptation-capability while maintaining a small dictionary size, without detailed tuning of the initial kernel parameters.

We present an online method for multiscale data classification, using the multikernel adaptive filtering framework. The target application is Twitter sentiment analysis, which is a notoriously challenging task of natural language processing. This is because (i) each tweet is typically short, and (ii) domain-specific expressions tend to be used. The efficacy of the proposed multiscale online method is studied with dataset of Twitter. Simulation results show that the proposed approach achieves a higher F1 score than the other online-classification methods, and also outperforms the nonlinear support vector machine.

We propose a novel online learning paradigm for nonlinear-function estimation tasks based on the iterative projections in the $L^2$ space with probability measure reflecting the stochastic property of input signals. The proposed learning algorithm exploits the reproducing kernel of the so-called dictionary subspace, based on the fact that any finite-dimensional space of functions has a reproducing kernel characterized by the Gram matrix. The $L^2$ -space geometry provides the best decorrelation property in principle. The proposed learning paradigm is significantly different from the conventional kernel-based learning paradigm in two senses: first, the whole space is not a reproducing kernel Hilbert space; and second, the minimum mean squared error estimator gives the best approximation of the desired nonlinear function in the dictionary subspace. It preserves efficiency in computing the inner product as well as in updating the Gram matrix when the dictionary grows. Monotone approximation, asymptotic optimality, and convergence of the proposed algorithm are analyzed based on the variable-metric version of adaptive projected subgradient method. Numerical examples show the efficacy of the proposed algorithm for real data over a variety of methods including the extended Kalman filter and many batch machine-learning methods such as the multilayer perceptron.