Published papers

This paper focuses on a multimodal language understanding method for ``Carry and Place'' tasks with domestic service robots. We address the case of ambiguous instructions, that is when the target area is not specified. For instance ``Put away the milk and cereal.'' is a natural instruction where there is ambiguity on the target area, considering daily life environments. Conventionally, this instruction can be disambiguated from a dialogue system, but at the cost of time and cumbersomeness. Instead, we propose a multimodal approach, where the instructions are disambiguated from the robot state and environment context. We develop MultiModal Classifier Generative Adversarial Network (MMC-GAN) to predict the likelihood of the different target areas considering the robot physical limitation and the target clutter. Our approach, MMC-GAN, significantly improves accuracy compared to baseline methods using instructions only or simple deep neural networks.

In this paper we investigate the evolutionary development of embodied agents that evolve not only control mechanisms but also the sensitivity and temporal resolution of their sensors. In our system, agents can adapt their sensory parameters to their morphology and task environments, while no explicit cost for sensors is used. We have carried out two kinds of experiments in which agents perform phototactic and pursuit/evading behaviors. The experimental results indicate that the sensors and controller co-evolve in an agent through interaction with the environments.