Designing Sensory and Behavioral Information to Overcome Physical Constraints

Maki Sugimoto (Professor, Department of Information and Computer Science)

Have you ever wished there were two of you during your busy days? Information technology provides the foundation for sharing sensations and actions beyond physical constraints, holding the power to turn such imaginings into reality. In our research field of human augmentation, we explore whether it is possible to extend beyond our physical limitations while maintaining our human sense of agency by utilizing technologies such as virtual reality, robotics, and machine learning.

The essence of this research goes beyond simply expanding physical abilities or perceptual ranges. It is also deeply concerned with how strongly and flexibly we can form our "sense of agency." The "sense of agency" refers to the feeling of being an active participant in one's environment, acting with intention. With the advancement of human augmentation technologies, it becomes crucial to maintain this sense of agency, no matter how our bodies or consciousness are extended.

For example, in a virtual reality (VR) space, one can simultaneously operate multiple hands or bodies, enabling experiences impossible in the real world. However, whether these experiences can be naturally accepted and recognized as one's own actions depends on the precision of visual, auditory, and tactile feedback, as well as the design of the interaction. In our research, we are verifying that by providing sensory feedback from multiple bodies simultaneously in response to predictions based on the efference copy of neural activity associated with active motor commands to the body, it is possible to have a certain sense of agency and sense of body ownership over the extended bodies.

Furthermore, in the field of robotics, the development of remote-controlled robots based on information transmission is advancing. This makes it possible to operate a robot as if it were one's own avatar, even in physically distant locations. Even when operating multiple robots simultaneously based on such technology, it is important to design behavioral information that can convey our intentions corresponding to multiple different remote environments. The ability to intuitively control the movements of multiple bodies according to one's own intentions, making them feel as if they are part of one's own body, is key to forming a "sense of agency."

Thus, human augmentation technology does not merely enhance functions; it creates our sense of agency in new ways and plays a crucial role in overcoming physical constraints. When we wish "if only there were two of me," the essence of this desire is not simply to increase our workload but to be actively involved in multiple situations simultaneously. By designing sensory and behavioral information to support this sense of agency, we can discover a new self and are expected to be able to proactively engage with new possibilities. How human augmentation technology will develop in the future remains unknown, but it is essential to maximize its potential while deeply understanding the relationship between technology and the human sense of agency.