[Special Feature: Science, Technology, and Social Issues] Hirotaka Osawa: From Artificial Agent Research to Narrative Application Research — The Expansion of SF Imagination

2024/08/05

I have been researching the field of Human-Agent Interaction (HAI) for many years. On the other hand, I also conduct research on the application of narratives. I would like to explain the relationship between these two.

HAI is an interdisciplinary field that has only been established for a little over 20 years, so I have given many explanations about what kind of field it is. HAI is a discipline that deals with the interaction between humans and agents, but that alone is not a sufficient explanation. The "agents" referred to here are primarily what are called social agents—artificial objects that behave as if they have intentions, much like humans or animals.

HAI is a field that spans various domains, but in a nutshell, it can be described as "a discipline that adjusts people's imagination toward social agents and explores useful applications for humans." HAI research did not emerge out of nowhere; there were many disciplines dealing with similar technologies, such as social robots and virtual agents. However, HAI was born as a field that treats these agents not merely as media, but as agents themselves that can socially influence people in a complex manner. Therefore, researchers from multiple fields—not only artificial intelligence and robotics, but also human-computer interaction, cognitive science, simulation, game theory, artificial life, video game studies, ethics, social sciences, and philosophy—have focused on this field. Research ranges from the design and implementation of the appearance, communication, and interaction of robots, virtual agents, voice dialogue, and remote dialogue systems, to the estimation of others' intentions and the creation of cooperative behavior in AI systems, as well as the examination of the social applications of agent systems and their legal, ethical, and social issues. From this perspective, I myself have conducted research on anthropomorphizing home appliances and research on artificial intelligence that cooperates with and deceives humans in games like Werewolf.

In HAI research, the emphasis is placed more on how an agent is treated by humans when it comes into contact with them, rather than the information processing process of the agent itself. For example, when designing the emotions of a pet robot, the focus is on how the pet robot's emotions are perceived by humans, rather than how the pet robot actually "feels." To put it bluntly, HAI research could be described as research into "deceiving artificial objects as if they have intentions." However, this is essential in a sense, and HAI research is also research into designing the "fiction within the user."

An interesting example of implementation from an HAI perspective is Primo Puel (released in 1999), a doll-type stuffed toy developed by Bandai. This doll was a simple stuffed toy where sensors attached to various parts responded to how a human touched it by uttering certain words; from the perspective of AI technology, it could hardly be said to use superior technology. However, Primo Puel was a "robot" hit primarily among middle-aged and older women, with over one million units sold and utilized in various locations, including nursing care settings.

This Primo Puel has a backstory that serves as a setting before it is purchased by the user. In it, Primo Puel is described as an alien from a small star who exists to heal people's hearts, and its various character traits were also part of its appeal. While the story of Primo Puel might not be called a "masterpiece," it is noteworthy that, regardless of the narrative's quality, Primo Puel emphasized creating a relationship with the purchaser.

In artificial intelligence research, narrative research has a long history of accumulation, from narrative analysis to narrative generation. Traditionally, such narrative research and HAI research were not treated within the same framework. This is because HAI is primarily a field for designing dynamic computer devices, where the "fiction" the user believes in is secondary and the range of control was narrow. On the other hand, narratives are basically a static medium, a framework where users receive chronological fiction embedded within it through text or video. However, in interactive media such as video games, the designs and objectives of both approach each other. For example, the design of non-player characters (NPCs) in video games and the research of virtual agents appearing on screens use similar technologies and are sometimes presented at similar academic conferences.

In particular, recent HAI research does not just refine the technology of a single agent, but rather takes an approach of improving human society as a result by intervening with single or multiple agents in a society where humans exist. For example, in "Learning by Teaching" proposed by Professor Fumihide Tanaka of the University of Tsukuba, an approach is taken where the agent itself becomes something to be taught by a child, and as a result, the child learns. Also, in the "Weak Robots" proposed by Professor Michio Okada of Toyohashi University of Technology, the presence of a robot that can do nothing results in prompting people's actions. Such research can be said to be HAI research that expands human society, and at the same time, it is close to narrative research in that it stimulates people's imagination and prompts human action as a result. Narratives are a challenge that requires dealing with multiple social entities over time, and in that sense, the work of designing the imagination toward agents can be said to be included as a part of this. In almost all narratives, there is a narrator, and in many narratives, multiple actors and societies exist.

From the above points, I believe there are great benefits to encouraging exchange between HAI research and narrative research in their respective developments.

I have shown the intersection between HAI research and narrative research, but the positioning of narratives in humanity itself is also transforming with the development of information technology. In particular, artificial intelligence technology centered on generative AI and literary fields related to science fiction (SF) are prompting this development.

In human history, the power to handle fiction based on imagination is thought to have been a major factor in the development of human culture. Human imagination has contributed in various aspects, such as social cohesion, the transmission of culture, the formation of individual and group identities, and even problem-solving and the promotion of creativity. Literary scholar Brian Boyd discusses how narratives became an essential element for human cognitive development, showing that narratives have played an important role in transmitting information, sharing social rules and values, and fostering a sense of solidarity within a group. Since the modern era, this imagination for handling fiction in humanity has been organized in the form of an industrial structure centered on narratives. In particular, science fiction (SF), which deals with science and technology, was born as a genre to explore the state of science, technology, and society in such times.

SF has brought various influences to social design, not staying within a single literary genre, while repeating longing for science and harsh self-criticism. Several related influences can also be seen in the humanities. SF has been used as a tool for science communication and, like futurology, has become food for connecting multiple disciplines and discussing the future. What was once called science propaganda is now a major educational support provided by the imagination of SF in fields such as STEAM (Science, Technology, Engineering, Art, and Mathematics) education. In management science, attempts at future prediction using scenarios have been conducted in fields such as foresight. Furthermore, in recent years, attempts such as "SF Prototyping" have emerged, where the process of creating a work—rather than the work itself—is shared between authors and experts to gain new visions for corporate management and education. Also, art fields like critical design and speculative art have appeared, inheriting the critical attitude and spirit of SF and making problem-posing their primary focus.

SF prototyping is particularly a trend in the application of new narratives. This is an expansion of the "SF that foresees the future" framework that was previously expected of SF, into a much wider range. SF prototyping does not necessarily predict the future. This is the same as how SF novels are not simply future-prediction novels. However, SF can connect current knowledge, technology, people's values, and multiple visions for the future to provide people with possible possibilities in the form of a story. Then, people can appreciate that story, examine it critically in some cases, and by getting involved in its creation process in other cases, they can ultimately have a deeper perspective on the future and build a vision. In this way, SF contributes to the creation of a vision for the future, and as a result, some SF works receive the honor (whether it is welcome or not) of having "foreseen the future." SF prototyping can be said to be the intentional creation of such attempts within a company.

One of the areas where SF is most effective is the introduction of a simulational perspective in an era where humans are expanded by science and technology and human values are shifting. The state of humanity in such a future is treated as the theme of the "Post-human." Due to science and technology that emerged from the late 20th century, especially information technology, the narratives created by imagination and the state of being human are currently being greatly shaken. With internet technology and social media, we have entered an era where anyone can create content themselves, and an environment has been established where narratives themselves are created not only by financial feedback but also by the evaluation of others. In a sense, this is the restoration of narrative as communication. Also, with generative AI technology that appeared in the 2010s, while receiving great backlash from some creators whose works are used for learning, it is expected that the state of creative skills will become more democratized. On the other hand, due to structural changes in the publishing industry caused by population decline and changes in the content environment, authors themselves are bringing narrative methodologies into the sites where science and technology are born as a way for narratives to exist. In the society of the post-humanity era, where human society is fundamentally strengthened by information technology and the state of our own intelligence is expanded, I believe that research on human imagination needs to be captured in a more cross-border manner than ever before.

In such an era as described above, we established a center named the "Science Fiction Research and Development / Implementation Center" at Keio University in January 2024. It is the first university research center in Japan to bear the name SF. Similar frameworks exist in multiple locations in the United States. For example, the University of Kansas has a center that researches SF as literature, and in recent years, as centers for innovation emergence using SF, there are the Center for Science and the Imagination at Arizona State University and the Arthur C. Clarke Center for Human Imagination at the University of California, San Diego. Also, in China, SF writer and researcher Wu Yan has established the Research Center for Science and Human Imagination at Southern University of Science and Technology. However, until now, there was no center specializing in SF in Japan. Sakyo Komatsu, a writer known for novels such as "Japan Sinks," once focused on the possibilities of SF under the name "SF-ology" and worked hard to establish an academic field. Also, writer Hideaki Sena has discussed the relationship between SF and science and technology, primarily in collaboration with robotics researchers.

Inheriting these trends, the SF Center will carry out research on imagination centered on SF along six fields: literature, cognitive science, artificial intelligence, management science, science communication, and art. Although I am an engineering researcher, the center is a field where not only science and technology but also the humanities and social sciences contribute greatly. If the changes brought about by the evolution of AI and communication technology are not captured from a human-centered perspective rather than just a technical aspect, it will lead to technological acceleration that leaves behind value for humans. In particular, examining the influence of narratives on the developmental process of children who will lead the future society can be said to be an urgent task. Also, management science in particular expands theoretical frameworks and analytical tools to deeply understand the impact of new technologies and innovations on society and culture. The transformation of creative activities by AI technology has a major impact on social values, cultural identity, and communication methods, and in order to understand and give meaning to these changes, cooperation with researchers of future design methods is essential. In our country, which has sent out a unique narrative culture comparable to the US and Europe with rich narratives as a hidden resource, while fundamental structural changes including changes in the publishing industry are predicted due to the declining birthrate and aging population, a major contribution will be born by organizing the influence of SF, which has played a role as the pivot of narrative culture including not only novels but also manga, animation, videos, video games, and derivative works, and by transmitting new ways of imagination to the world. Furthermore, in the current situation where uncertain and catastrophic situations occur, such as the COVID-19 pandemic, Russia's invasion of Ukraine, and large-scale language model technology aiming for artificial general intelligence, it becomes important to develop the ability to examine possibilities in an uncertain future, rather than predicting the future itself. In this regard, science communication and speculative art play a major role.

Furthermore, the development of AI technology, including generative AI, brings about a major difference from the traditional ecosystem regarding narratives. Traditionally, narratives have been recognized as creative expressions of human thoughts or emotions, with rights including moral rights acknowledged, and have received legal and ethical protection. In contrast, when the creative process is automated as in generative AI, intervention is performed regarding the creativity of the work. Because of this, from the perspectives of literature and aesthetics, it is thought that there will be more things to newly examine. Another point is the diversion of narrative imagination itself. In the narrative environment since the modern era, a privileged role called the author existed, which publishing assisted, and a form where many people receive the narrative was established as an industry. However, in the modern society of the evaluation economy, where network technology and social network platforms have developed and feedback to authors is given not only through money but also through praise for self-expression, the boundary between author and reader has become blurred. This era is in a sense the restoration of narrative as communication, but it is happening that the imagination of the writer who weaves the narrative is directly involved in the construction of a future vision as part of a kind of simulation ability and insight into the future. Such an environment, coupled with the imagination expansion environment by AI, results in further expanding the role in society of the narratives created by human imagination. Current generative AI is causing problems in various frameworks, such as the attribution of creative works, the ethics of creative activities by AI, and the redefinition of the relationship between humans and AI. I want the center to provide ethical frameworks and philosophical insights to handle these problems and serve as a bridge to deepen discussions on how society should respond to these new challenges.