Building a Truly Caring Doraemon Through Japan’s World-Leading HAI Technology “I want to build Doraemon”

This is something that many people have asked me, but to be honest, I don’t actually know. It was something I had already begun thinking about before entering elementary school, so I don’t clearly remember when this dream started. Of course, I had many other interests as a child. In grade school I played the piano, and through my piano teacher, even worked as a child actor. I’ve always loved making things since kindergarten, and I would have fun crafting things out of toilet paper rolls. Then when I was in elementary school, I started attending robotics classes and became interested in electronics. Looking back, this love of making things was certainly one of my starting points. However, if someone asked me “Why Doraemon?” I wouldn’t be able to give a definitive answer.  Now that I’m older, I can give people some plausible explanation, but no matter what it was, it would always feel untrue.

When I was a child and I said that I wanted to build Doraemon, the adults around me would laugh. Some would tell me, “Good luck!” but even as a child, I knew they didn’t truly mean it.  Eventually I became reluctant to talk about it and began hiding my dream. At one point, I even tried to forget that I liked Doraemon. So rather than saying I held onto the dream, it might be more accurate to say I simply couldn’t forget it. I finally began openly saying “I want to build Doraemon” when I started doing actual research in my fourth year of university.

Up until junior high school, I disliked having to study prescribed subjects according to a fixed timetable. I thought that at a technical high school, I would be able to immerse myself in hands-on activities like building robots, so I enrolled there, where I studied extremely hard. However, the level of the students around me was very high, and I couldn’t keep up with the coursework. At first, I was by far the weakest student in the class. Around that time, my grandfather, whom I loved dearly, passed away suddenly. I hadn’t seen him for a while because I didn’t want him to see me in my state at the time, as a poorly performing student. His death was a tremendous shock. But I told myself that I needed to turn my grades around and make my grandfather proud. This motivated me to take my studies even more seriously than before. Fortunately, my friends encouraged me when I was feeling down and began studying together with me. We would divide up the work and cooperate in preparing for regular exams, and gradually I started to enjoy my studies in high school. As that happened, my grades improved rapidly, and in the end I was able to graduate at the top of my class in the field of information systems. During my time there, I also competed in national contests related to programming and information processing. It was demanding, but I feel that I had a very fulfilling and enjoyable high school life.

To be honest, I missed the internal recommendation for the Tokyo Institute of Technology (present-day Institute of Science Tokyo). Since I hadn’t prepared for general entrance examinations at all, I applied through recommendation/AO admissions. I chose Keio almost intuitively. I believe it was the right decision. Had I not entered Keio, I might not be pursuing the research enabling me to build Doraemon today.

Until high school, my life and the lives of those around me focused entirely on technology and research. But at Keio, the “world” changed completely. Students at the Faculty of Science and Technology immersed themselves in their studies, but they also enjoyed their student life, delving into sports, music, and entertainment. It was a real culture shock. I found that there are many ways of seeing the world and people value different things. That was the first thing I learned when I entered college. Until then, I thought the research to build Doraemon would have be done all on my lonesome, one slow but sure step at a time. But meeting people with diverse perspectives made me do a 180 and realize collaboration could open new possibilities.

My first goal was simply to become a “normal” university student like my peers. To that end, during the three years before selecting a laboratory, I tried to limit my exposure to science and technology outside of the classes I was required to take, and instead threw myself into student clubs and part-time jobs in order to learn about the wider world. I was especially involved in a volunteer group for children, since I enjoy working with children. In fact, even today, alongside my research, I run a group where kids can build Doraemon.

Right. By then, I had finally begun to understand the world around me and had made many friends, so I decided to return to the field of science and technology and seriously start working toward my goal of “building Doraemon.” From the second year of my master’s program, I studied under Professor Michita Imai, who specializes in intelligent systems such as interactive AI, and he supervised me through the completion of my doctoral program. Back when I was in the first year of my master's program, I was also greatly inspired by talks I heard at an on-campus AI symposium, including those by Professor Hiroshi Ishiguro from Osaka University and Professor Hiroshi Yamakawa, who was then director of the Dwango Artificial Intelligence Laboratory. I also participated in the “Whole Brain Architecture Study Group,” led by Professor Yamakawa and Professor Yutaka Matsuo from the University of Tokyo, where I had opportunities to interact with others working in AI research. Later, together with other young participants from that community, I took the initiative in founding the “Whole Brain Architecture Future Leaders” group, an independent organization established separately from the aforementioned study group. The Whole Brain Architecture Future Leaders group included not only AI researchers but also specialists in fields such as neuroscience, psychology, and design.

Yes. In my view, knowledge of AI alone is not enough to “build Doraemon.” In my work with neuroscience and cognitive science, I study the hippocampus, the part of the brain responsible for human memory. Another major research topic is the design of robots that people can feel affection for and connect with.

My own personal view that guides my research is that what many people perceive as Doraemon-like intelligence could be closely tied to the presence of a "heart." Generative AI such as ChatGPT is based on deep learning, which absorbs patterns and rules from vast amounts of collected data. However, with conventional deep learning techniques alone, it is still difficult to create a caring heart that truly feels like Doraemon to everyone. What is needed is a new approach. That comes in the form of Human-Agent Interaction (HAI), which incorporates knowledge beyond AI, including insights from neuroscience and cognitive science. This is a field that studies systems in which humans and robots or AI collaborate. In a sense, it resembles the relationship between Nobita and Doraemon: a system in which a robot can be perceived as having a heart and as a companion to humans. Moreover, by involving human support, the technical hurdles for building such robots can be lowered. In fact, Japan has produced a considerable body of research in the field of HAI and is in a leading position globally.

Yes, that marks exactly 30 years since I began this journey as a researcher in my fourth year of university. However, this is just a matter of completing the project, rather than perfecting Doraemon, per se.

If I were to say “completion,” it would imply a single, definitive version of Doraemon that everyone in the world agrees on.  But I believe each person has their own image of what Doraemon should be. Of course, I will do my utmost to aim for completion. But our goal is not simply to create a finished product. I also hope that everyone involved in the project, both the experts who have contributed their knowledge and the members of the public who have supported us, will find happiness through the process of creating Doraemon together. Indeed, that is precisely the kind of project that befits the realization of a Doraemon loved by everyone.

Over the past ten years, the “building Doraemon” project has progressed faster than expected. We began by defining Doraemon, then identified the necessary fields of expertise and technologies, as I mentioned earlier. Along the way, we were also fortunate to gain the support of dedicated collaborators, research institutions, and companies willing to join us in this endeavor. During this period, generative AI has also seen explosive development. In the early stages, I intentionally avoided using natural language-based approaches in my research, because I felt they might obscure the essence of intelligence. However, after seeing the emergence of large language models, I came to realize that they capture something fundamental about intelligence, and I began incorporating them into my research. Now, by integrating the natural language capabilities of generative AI, I am exploring how to create a Doraemon that can connect with people through both “heart” and “language.” Looking ahead, new technologies that we have never seen before may emerge in the next 10 or 20 years. If these new technologies prove useful for bringing about a real Doraemon, I intend to actively incorporate them as well.

Keio University offers a comprehensive education that brings together students with diverse values.  I came to Keio after attending a technical high school where I focused almost exclusively on technology and research. Being at Keio completely transformed my life, and it ultimately enabled me to turn my “building Doraemon” research into a reality. Along the way, I met many people who shared the same dream, even though our positions and directions differed, and I am truly grateful for those encounters. I hope that all of you will also treasure the connections you make during your school years as Keio students. That’s the general advice I would offer to younger students. In addition, I believe each of you has your own dreams, goals, and aspirations. Some of you may have felt discouraged because others dismissed your dreams, just as people around me did not initially take my goal of “building Doraemon” seriously. But ultimately, you are the one who thinks most deeply about your own future. I hope you will nurture the feelings of “this is what I love” and “this is what I want to challenge myself with” as you go through your student life. If you do, I believe you’ll naturally expand your chances of meeting others who share the same passions and perspectives.