論文

Building generalist robots that can perform well in complex environments requires high generalization ability. Imitation learning and reinforcement learning are two promising approaches, but they have a tradeoff between data collection cost and data richness. To overcome this tradeoff, we focus on 'play data' collected by humans teleoperating a robot out of curiosity. Specifically, we propose a framework for real-world robot control and data augmentation based on world-model learning from play data. We demonstrate that our framework can be used to train robots to perform goal-conditioned object-manipulation tasks in the real world. Furthermore, we show that imagination in the world model can be used to create novel combinations of actions that were not included in the original play data. These findings suggest that further learning from the augmented data may enable robots to acquire a higher generalization ability.

Despite significant advances in robotics and deep learning, the ability of robots to perceive and act remain far below that of humans. To bridge this gap, we utilize active inference, a framework based on the free-energy principle that accounts for various human brain functions. Despite the utility of active inference, most previous studies have focused on simulated or real-world environments with low-dimensional observations. Therefore, direct applications in real-world robotics with high-dimensional observation and action spaces remain a challenge. To address this issue, we combine the active inference framework with contrastive learning and develop an extension by introducing learning from demonstration. This framework comprises a world model, an action model, and an expected free-energy model. The world model ensures an adequate perception of the environment by minimizing contrastive variational free energy. Meanwhile, the expected free-energy model estimates the contrastive expected free energy, while the action model generates adaptive actions by learning from demonstrations, with the aim of minimizing the estimates provided by the expected free-energy model. We show the effectiveness and robustness of the proposed framework through image-based robotic reaching tasks in both learned and unlearned real-world environments. As a result, while the baselines achieved a success rate of up to approximately 71%, the proposed framework achieved a success rate of around 96%. Moreover, the proposed framework realized performance improvements by leveraging estimated expected free energy for action selection. These results highlight the utility of active inference with contrastive learning and learning from demonstration for real-world robot control.

Robots in uncertain realworld environments must perform both goaldirected and exploratory actions. However, most deep learning-based control methods neglect exploration and struggle under uncertainty. To address this, we adopt deep active inference, a framework that accounts for human goal-directed and exploratory actions. Yet, conventional deep active inference approaches face challenges due to limited environmental representation capacity and high computational cost in action selection. We propose a novel deep active inference framework that consists of a world model, an action model, and an abstract world model. The world model encodes environmental dynamics into hidden state representations at slow and fast timescales. The action model compresses action sequences into abstract actions using vector quantization, and the abstract world model predicts future slow states conditioned on the abstract action, enabling low-cost action selection. We evaluate the framework on object-manipulation tasks with a real-world robot. Results show that it achieves high success rates across diverse manipulation tasks and switches between goal-directed and exploratory actions in uncertain settings, while making action selection computationally tractable. These findings highlight the importance of modeling multiple timescale dynamics and abstracting actions and state transitions.

これまで時系列パターンの記憶学習に，現状態と文脈情報から次状態を予測することが可能な再帰結合神経回路（recurrent neural network: RNN）モデルが広く用いられてきた．しかし，決定論モデルであるという特性から教示時系列に潜む確率的な構造を獲得することができないという問題がある．また，複数パターン間でノイズの程度（分散）が異なると，小さな分散のパターンが大きな分散のパターンに埋もれてしまい，全パターンを安定的に学習することが出来ないという問題もある．本研究では，従来の学習方法である二乗誤差最小化に基づく点予測の枠組みを拡張し，尤度最大化に基づき確率分布の予測学習が可能なStochastic Continuous Time RNN（S-CTRNN）を用いることで，前述の問題が解決可能であることを示す．

Intelligent robots are being developed with the expectation that they will perform various tasks in diverse environments. Such robots need to autonomously engage in both exploratory behavior to reduce environmental uncertainty and goal-directed behavior to achieve their preferred observations (or goals). In this study, we focus on active inference, which provides a unified scheme for these distinct behavioral modes. Policy selection in active inference is based on minimizing expected free energy (EFE), which consists of one term representing epistemic value and another representing extrinsic value. Specifically, we investigate the influence of preference precision, which controls the balance between these two terms, on policy selection by a physical robot receiving high-dimensional and uncertain observations. We developed a deep active inference framework comprising a world model and a policy suggester. The world model predicts future hidden states and observations based on candidate policies from the policy suggester. The EFE for each policy is approximated using the predicted future hidden states and observations as well as the preferred observation. We implemented our proposed framework in a robot, requiring it to select a policy that minimizes EFE and then generate actions accordingly. The experimental results showed that the robot implementing the proposed framework selected exploratory or goal-directed behavior depending on the level of preference precision. These findings suggest that adjusting preference precision plays a crucial role in the autonomous selection of exploratory or goal-directed behavior in real-world situations with potential uncertainty.

For assisting humans in their daily lives, robots need to perform long-horizon tasks, such as tidying up a room or preparing a meal. One effective strategy for handling a long-horizon task is to break it down into short-horizon subgoals, that the robot can execute sequentially. In this paper, we propose extending a predictive learning model using deep neural networks (DNN) with a Subgoal Proposal Module (SPM), with the goal of making such tasks realizable. We evaluate our proposed model in a case-study of a long-horizon task, consisting of cutting and arranging a pizza. This task requires the robot to consider: (1) the order of the subtasks, (2) multiple subtask selection, (3) coordination of dual-arm, and (4) variations within a subtask. The results confirm that the model is able to generalize motion generation to unseen tools and objects arrangement combinations. Furthermore, it significantly reduces the prediction error of the generated motions compared to without the proposed SPM. Finally, we validate the generated motions on the dual-arm robot Nextage Open. See our accompanying video here: https://youtu.be/3hYS2knRm50

This article introduces the System 0/1/2/3 framework as an extension of dual-process theory, employing a quad-process model of cognition. Expanding upon System 1 (fast, intuitive thinking) and System 2 (slow, deliberative thinking), we incorporate System 0, which represents precognitive embodied processes, and System 3, which encompasses collective intelligence and symbol emergence. We contextualize this model within Bergson's philosophy by adopting multiscale time theory to unify the diverse temporal dynamics of cognition. System 0 emphasizes morphological computation and passive dynamics, illustrating how physical embodiment enables adaptive behavior without explicit neural processing. Systems 1 and 2 are explained from a constructive perspective, incorporating neurodynamical and artificial intelligence (AI) viewpoints. In System 3, we introduce collective predictive coding to explain how societal-level adaptation and symbol emergence operate over extended timescales. This comprehensive framework ranges from rapid embodied reactions to slow-evolving collective intelligence, offering a unified perspective on cognition across multiple timescales, levels of abstraction, and forms of human intelligence. The System 0/1/2/3 model provides a novel theoretical foundation for understanding the interplay between adaptive and cognitive processes, thereby opening new avenues for research in cognitive science, AI, robotics, and collective intelligence.

Latent psychiatric and cognitive characteristics of individuals may be present in multiple data sources, such as subjective self-report measures and objective behavioral data. Therefore, integrative learning of multimodal data from individuals has the potential to contribute to understanding these characteristics. In this study, we propose a novel generative model and evaluate it with synthetically generated multimodal data. In multimodal data, each modality is considered to have low-dimensional latent representations, but these representations are not always fully shared with another modality. Therefore, we assume that latent representations of multimodal data consist of shared representations across all the modalities and private representations unique to each modality. Under this assumption, we develop a deep generative model that can learn to extract these different latent representations from both non-time-series and time-series data in an end-to-end manner. To evaluate our proposed model, we conducted simulation experiments with a synthetic multimodal dataset containing shared and private information. Experimental results on the inference of shared and private latent representations, as well as on self-modal and cross-modal reconstructions, demonstrate the potential of our proposed model to better understand latent psychiatric and cognitive characteristics in multimodal data.

This paper proposes a novel stable learning theory for recurrent neural networks (RNNs), so-called variational adaptive noise and dropout (VAND). As stabilizing factors for RNNs, noise and dropout on the internal state of RNNs have been separately confirmed in previous studies. We reinterpret the optimization problem of RNNs as variational inference, showing that noise and dropout can be derived simultaneously by transforming the explicit regularization term arising in the optimization problem into implicit regularization. Their scale and ratio can also be adjusted appropriately to optimize the main objective of RNNs, respectively. In an imitation learning scenario with a mobile manipulator, only VAND is able to imitate sequential and periodic behaviors as instructed.

This paper introduces an imitative model that enables a robot to acquire viewpoints of the self and others from its own sensory-motor experiences. This is important for recognizing and imitating actions generated from various directions. Existing methods require coordinate transformations input by human designers or complex learning modules to acquire a viewpoint. In the proposed model, several neurons dedicated to generated actions and viewpoints of the self and others are added to a dynamic nueral network model reffered as continuous time recurrent neural network (CTRNN). The training data are labeled with types of actions and viewpoints, and are linked to each internal state. We implemented this model in a robot and trained the model to perform actions of object manipulation. Representations of behavior and viewpoint were formed in the internal states of the CTRNN. In addition, we analyzed the initial values of the internal states that represent the viewpoint information. We confirmed the distinction of the observational perspective of other's actions self-organized in the space of the initial values. Combining the initial values of the internal states that describe the behavior and the viewpoint, the system can generate unlearned data.

Human communicative behavior is both dynamic and bidirectional. This study aims to analyze such behavior by conducting imitative interactions between human subjects and a humanoid robot that has a neuro-dynamical system. For this purpose, we take a robot-centered approach in which the change in robot performance according to difference in human partner is analyzed, rather than adopting the typical human-centered approach. A small humanoid robot equipped with a neuro-dynamical system learns imitative arm movement patterns and interacts with humans after the learning process. We analyze the interactive phenomena by different methods, including principal component analysis and use of a recurrence plot. Through this analysis, we demonstrate that different classes of interactions can be observed in the contextual dynamics of the neuro-dynamical system.

In recent years there has been increased interest in studies that explore integrative learning of language and other modalities by using neural network models. However, for practical application to human-robot interaction, the acquired semantic structure between language and meaning has to be available immediately and repeatably whenever necessary, just as in everyday communication. As a solution to this problem, this study proposes a method in which a recurrent neural network self-organizes cyclic attractors that reflect semantic structure and represent interaction flows in its internal dynamics. To evaluate this method we design a simple task in which a human verbally directs a robot, which responds appropriately. Training the network with training data that represent the interaction series, the cyclic attractors that reflect the semantic structure is self-organized. The network first receives a verbal direction, and its internal state moves according to the first half of the cyclic attractors with branch structures corresponding to semantics. After that, the internal state reaches a potential to generate appropriate behavior. Finally, the internal state moves to the second half and converges on the initial point of the cycle while generating the appropriate behavior. By self-organizing such an internal structure in its forward dynamics, the model achieves immediate and repeatable response to linguistic directions. Furthermore, the network self-organizes a fixed-point attractor, and so able to wait for directions. It can thus repeat the interaction flexibly without explicit turn-taking signs.

This paper investigates a possible neurodynamic mechanism that enables autonomous switching between two basic behavioral modes, namely a "proactive mode" and a "reactive mode." In the proactive mode, actions are generated as intended, whereas in the reactive mode actions are generated in response to the sensory state.We conducted neurorobotics experiments to investigate how these two modes can develop and how a robot can learn to switch autonomously between the two modes as necessary by utilizing our recently developed dynamic neural network model. Tasks designed for the robot included switching between proactive imitation of other's predictable movements using acquired memories and reactive following of other's unpredictable movements through iterative learning of alternating predictable and unpredictable movement patterns. The experimental results revealed that this "meta-learning" capability can lead to self-organization of adequate contextual dynamical structures that can perform autonomous switching between the different behavioral modes. © Springer-Verlag 2013.

Active inference offers a unified framework in which agents can exhibit both goal-directed and epistemic behaviors. However, implementing policy search in high-dimensional continuous action spaces presents challenges in terms of scalability and stability. Our previously proposed model, T-GLean, addressed this issue by enabling efficient goal-directed planning through low-dimensional latent space search, further reduced by conditioning on prior habituated behavior. However, the lack of an epistemic term in minimizing expected free energy limited the agent’s ability to engage in information-seeking behavior that can be critical for attaining preferred outcomes. In this study, we present EFE-GLean, an extended version of T-GLean that overcomes this limitation by integrating epistemic value into the planning process. EFE-GLean generates goal-directed policies by inferring low-dimensional future posterior trajectories while maximizing expected information gain. Simulation experiments using an extended T-maze task—implemented in both discrete and continuous domains—demonstrate that the agent can successfully achieve its goals by exploiting hidden environmental information. Furthermore, we show that the agent is capable of adapting to abrupt environmental changes by dynamically revising plans through simultaneous minimization of past variational free energy and future expected free energy. Finally, analytical evaluations detail the underlying mechanisms and computational properties of the model.

This paper explores the application of active inference and the free energy principle (FEP) to enable active vision in physical robots, using pixel-level RGB camera observations. By adapting existing methodologies previously limited to simulated environments, we introduce architectural improvements, including spatial softmax, to address the challenges of real-world application. Our model demonstrates proficiency in both exploratory and goal-directed behaviors within complex environments, achieving a dynamic understanding of visual scenes from pixel data. Our findings further demonstrate the potential of active inference and the FEP for tackling active vision in real-world robotics, and in bridging the gap between artificial and biological systems, offering a robust framework for developing more adaptive and aware robotic agents.

Collaborative robots are expected to generate actions by inferring a task goal from an environmental situation while performing dynamic optimization of the inferred goal based on a human partner's behavior. The objective of this study was to develop a computational framework that enables collaborative robots to learn this inference ability. For this, we propose a deep-learning based framework consisting of (1) a goal-inference network, (2) a goal-recognition network, and (3) an action- generation network. The goal-inference network is jointly trained with the goal-recognition network to infer a latent goal only from the initial image of a task space. The action-generation network generates a prediction about the visuomotor state from its current state and the inferred latent goal. During action-generation, the inferred latent goal is dynamically optimized to minimize visual prediction errors in order to adapt to changes in the human partner's goal. To evaluate this framework, we conducted an experiment on a collaborative object arrangement task. In the task, a human was allowed to change its goal during collaboration and a robot was required to adapt to such a situational change by inferring the human's goal. The experimental results demonstrate that the robot using the developed framework realized a successful collaboration.