論文

In this paper, we propose a method for utilizing musical artifacts and physiological data as a means for creating a new form of live music experience that is rooted in the physiology of the performers and audience members. By utilizing physiological data (namely Electrodermal Activity (EDA) and Heart Rate Variability (HRV)) and applying this data to musical artifacts including a robotic koto (a traditional 13-string Japanese instrument fitted with solenoids and linear actuators), a Eurorack synthesizer, and Max/MSP software, we aim to develop a new form of semi-improvisational and significantly indeterminate performance practice. It has since evolved into a multi-modal methodology which honors improvisational performance practices and utilizes physiological data which offers both performers and audiences an ever-changing and intimate experience. In our first exploratory phase, we focused on the development of a means for controlling a bespoke robotic koto in conjunction with a Eurorack synthesizer system and Max/MSP software for controlling the incoming data. We integrated a reliance on physiological data to infuse a more directly human elements into this artifact system. This allows a significant portion of the decision-making to be directly controlled by the incoming physiological data in real-time, thereby affording a sense of performativity within this non-living system. Our aim is to continue the development of this method to strike a novel balance between intentionality and impromptu performative results.

Tactile sensation is essential for recognizing mechanical interactions between the human body and its environment; it is generally uniquely experienced by individuals. Haptic sharing technologies enable to share of other person's (or robot's) tactile sensations, and it is expected to apply for skill learning, collaborative work, and robot operation. While wearable tactile sensors are useful for tactile measurements, they are affected by differences in physical characteristics between individuals. Therefore, to accurately convey the tactile information felt by others and utilize it for work, it is necessary to remove individual differences due to differences in individual physical characteristics. In this study, based on the frequency based intensity of skin vibration of an individual, the vibration detected by the acceleration sensor affixed to the fingertip was modulated to remove individual differences. To confirm the effectiveness of the proposed method, we investigated both physical and perceptual similarities for skin vibrations collected from three persons with three different fine textures. The results showed that vibrations individually modulated with the proposed method was similar in the intensity and power spectrum density than those using the uniform modulation function, and was perceived similar when presenting those with a vibrator.

Mixed reality environments are the next step in remote collaboration, yet there has been little attention towards avatar design for the en-joyability and efficiency of diverse users, particularly for the lesbian, gay, bisexual, transgender and queer (LGBTQ+) community. We believe that it is important to adapt the design of virtual avatars users collaborate with depending on not only the users gender but also their sexuality. In this study, we focus on examining the relationship between the sexual orientation of male users with the gender expression of virtual avatars, while they perform a collaborative task in mixed reality. We analyzed the perception of straight cisgender men and gay cisgender men in relation to the gender expression of the above mentioned agents and find differences that might affect design decisions for creating user-base specific avatars. We found that Female avatars tend to have a more positive perception when it comes to gay men. And for mixed reality, gay men also tend to have both negative and positive experiences more intensely than straight men.

Journaling is a well-known evidence-based strategy for practicing better self-regulation and self-awareness in daily life. It is also an effective way for reducing the effects of negative emotions like anxiety and depression. We explore the combination of virtual reality (VR) and a body-tracking mirror as a cognitive activity in the form of VR journaling. VR-based journaling aims to build cognitive reappraisal, self-reflection, and autonomic emotion regulation in a sustainable way. The user performs in front of the mirror, using body-tracking to help them keep a daily journal. Considering that looking into the mirror is already a daily habit for most people, it does not require the users to form a new habit when trying to do journaling. Furthermore, it activates creativity by encouraging the user to use their body language as a story-telling tool.

During the grieving process, physical objects often serve as catalysts for remembering and honouring the relationship with departed loved ones. Leveraging a participatory design approach, we created Be.side, a fully customisable multi-modal artefact that incorporates scent, sound, and heartbeat stimulation and acts as a touch-point between the deceased and the bereaved. We conducted a four-week study with three participants to understand how the artefact, continuously attuned to each participant, helped to continue bonds with the deceased. Our results show that Be.side's bespoke elements helped participants to evoke memories of the deceased. Participants created personalised rituals for remembrance. They sustained bonds by not only interacting with Be.side but also participating in the research. Finally, highlighting that remembrance can both provide comfort and deepen sadness, we discuss future design considerations.

Our goal has been to create an environment in which anyone can take advantage of tactile technology. Content that incorporates tactile sensations in addition to visual and auditory sensations will activate people's interaction with it. On the other hand, both sensing and display are complicated to create high-quality content, and ease of use is lost. In this paper, we propose a platform that can easily develop multi-channel vibrotactile contents by utilizing existing workflows. By preparing a 4-channel fingertip-mounted vibrotactile device as a display, a high-resolution experience with less load during the experience is realized. We demonstrate sensor-based pre-recorded or real-time multi-channel tactile contents and show the concept of near-future tactile technology including use cases.

Generative Artificial Intelligence (AI) has been a fast-growing technology, well known for generating high-quality design drawings and images in seconds with a simple text input. However, users often feel uncertain about whether generative art should be considered created by AI or by themselves. Losing the sense of ownership of the outcome might impact the learning process and confidence of novice designers and design learners who seek to benefit from using Generative Design Tools. In this context, we propose OwnDiffusion, a design pipeline that utilizes Generative AI to assist in the physical prototype ideation process for novice product designers and industrial design learners while preserving their sense of ownership. The pipeline incorporates a prompt weight assessing tool, allowing designers to fine-tune the AI’s input based on their sense of ownership. We envision this method as a solution for AI-assisted design, enabling designers to maintain confidence in their creativity and ownership of a design.

We explore a "Parallel Interaction", where a user controls several bodies concurrently. To demonstrate this type of interaction we developed "Parallel Ping-Pong". The user plays ping-pong by controlling 2 robot arms with a Virtual Reality (VR) controller, and while looking from the viewpoint of one of the 2 robot arms with a Head Mounted Display (HMD). The HMD view switches between the 2 robot arms' viewpoints according to the ping-pong ball position and direction. To keep the sense of agency over the robot arms (hindered by the system latency), the robot arms calculate the balls' paths and position themselves to hit the balls back; all while integrating the controller motion. Throughout this demonstration, we investigate a real-life design of Parallel Interaction with multiple bodies by a single user.

We propose Parallel Ping-Ping, a system that realizes "Parallel Embodiment", the experience of a single user controlling multiple bodies simultaneously. The user plays ping-pong by controlling 2 robot arms using a Virtual Reality (VR) controller, while looking at 2 tables through a Head Mounted Display (HMD). The computer constantly calculates the ball trajectory and the motion for the robot arm to hit the balls back automatically through color cameras. Based on this calculation, the system integrates the motion of the user's controller, maintaining the sense of agency of the robot arms even though single user plays ping-pong with two opponents. In addition, the user's view through the HMD is automatically switched to the appropriate table according to the calculated position and direction, so that the user can perceive both tables' situation smoothly. We exhibited a three-day Parallel Ping-Pong demonstration and surveyed 142 participants about their perception while controlling multiple bodies. In this paper, we introduce the Parallel Ping-Pong system framework and the insights from the survey of demonstration. We also discuss the design implication for Parallel Embodiment based on these results.

Robotic avatars and telepresence technology enable people with disabilities to engage in physical work. Despite the recent popularity of the metaverse, few studies have explored the use of virtual avatars and environments by people with disabilities. In this study, seven disabled participants working in a cafe where remote customer service is provided via robotic avatars, were engaged in the development and use of personalized virtual avatars displayed on a large screen in-situ in combination with existing physical robots, creating a hybrid cyber-physical space. We conducted longitudinal semi-structured interviews to investigate the psychological changes experienced by the participants. The results revealed that mass-produced robotic avatars allowed participants to not disclose their disability if they did not want to, but also backgrounded their identities; by contrast, customized virtual avatars shaped without physical constraints, highlighted their personalities. The combined use of robotic and virtual avatars complemented each other and can support pilots in redefining their identity.

Phantom Undulations is a mixed-media work in which an artist's physiological data is being used remotely to manipulate the sounds and visuals of an abstract artifact in a gallery setting. This work relies heavily on the concept of showing the artist's presence or liveness in an abstract and remote manner through changes in the harmony, rhythm, and timbre of a loosely structured soundscape as well as the physical appearance of the artifact. We propose a method of utilizing real time physiological sensing data through a custom built sensing wristband and accompanying software. This system reads the physiological data of the artist and sends it to the Internet, where it can be received by the artifact anywhere on Earth. In addition to the artist's physiological data, we also offer a way for the audience to incorporate their own data into the work via several sensing wristbands which will accompany the artifact. Through this collaborative process, we wish to invite the audience to join the artist in manipulating the sonic and visual characteristics of this artifact and create a contrapuntally fluid and responsive musical experience.

"Phantom Walls"is a novel technique that establishes continuous spatial perception without vision. By creating an auditory environment where obstacles emit sounds, users can perceive and navigate around visually imperceptible "Phantom"obstacles by listening to the generated "soundscapes". This method allows individuals to perceive and avoid these obstacles while walking without vision.

Horror is often associated with a negative connotation. Yet, in today's popular culture, the benefits of thrills have become an engaging topic. Why are people drawn to the supposedly repulsive contents of horror? It is an inherent psychological trait we all possess called morbid curiosity. Can we better investigate this behavior towards horror to confront our fears and help us build confidence? In Project Corvus, we explore the ways in which virtual reality (VR) and psychological programs such as exposure therapy can be combined to inform the psycho-educational opportunities in this horror-themed direction. More specifically, to leverage people's morbid curiosity towards horror for the development of self-efficacy. Through short repetition of horror virtual experiences, the goal is to design an assistive program that can help the participants develop their skills in controlling their emotions even when facing real crises in the future.

The human arm has seven DoF. If the human arm were more redundant, it would improve workability. However, intuitive manipulation of redundant arms is difficult due to the complexity of the input system or the limitations of the human arm. In this study, we constructed a system in which multiple operators integrate their bodies to intuitively operate a redundant multi-DoF robotic arm. The robotic arm was assembled by connecting two arms, and the roles of each section -distal arm and proximal arm- were assigned to different operators. First, vibrotactile feedback methods to the users based on the operation movement were investigated to compensate for the recognition of the partner's movement. Three types of feedback methods - the actions of both oneself and one's partner, the actions of the partner, and none - were compared in order to improve the body awareness. It was shown that feedback of the movement from the partner who operates the proximal robotic arm significantly enhanced the sense of body ownership. Then, the proposed co-operation of the redundant robotic arm was evaluated in the workability. Two types of manipulation methods - a single-operator system that switched the target robotic arm and a multi-DoF body-integration system used by two people - were compared through an object-manipulation task involving obstacle avoidance. It was shown that the two-person co-operation through body integration system to be faster than the single-person operation.

In this study, we created a sport which applied with human augmentation technology, that is not affected by the differences in the participants' lower limbs, using a wheelchair-type body-enhancing medium that provides a new motor ability called drifting that was not possible before. The sport was played by six players, three people with non lower limb disabilities and three without lower limb disabilities, and the results showed a change in the added ability especially for their motions' accuracy. Some of these were not related to the differential status of the lower limb, while others may have been.

Music serves as a tangible embodiment of the performer’s expression, bearing a distinct musicality that transcends auditory perception. This study delves into the distinctive musicality inherent to musicians through physical data analysis. It introduces a novel approach to the auditory experience by incorporating tactile stimulation via vibration and pressure to present an innovative channel for conveying musicality to the audience. The primary aim is to enhance the realm of music appreciation by amplifying the scope of performers’ expressive capacities, thereby revolutionizing the conventional paradigm of music experiences.

Preference and tolerance towards spicy food may vary depending on culture, location, upbringing, personality and even gender. Due to this, spicy food can often effect the social interaction on the dining table, especially if it is presented as a cultural dish. We propose SpiceWare, a thermally adjustable spoon that alters the perception of spice to improve cross-cultural communication. SpiceWare is a 3D-printed aluminium spoon that houses a thermal peltier that provides thermal feedback up to 45°C which can alter the taste perception of the user. As an initial evaluation, we conducted a workshop among participants of varying cultural backgrounds and observe their interaction when dining on spicy food. We found that the overall interaction was perceived to be more harmonious, and we discuss potential future works on improving the system.

"Synced Drift"provides a new inclusive sport experience that allows two individuals to play a drifting race together while sharing a single body. Synced drift leverages a system consisting of a large-diameter omni-wheel mechanism enabling omnidirectional movement, two sensing seats that detects the user's center of gravity shift, a mobile vehicle, and a viewpoint sharing system. The movement of the mobile vehicle is achieved by proportional control of the body movements acquired by the two sensing seats, one placed on the vehicle and one remote, from two or more individuals. The speed and direction are adjusted based on the harmonization of the inputs from the two users. This system allows people who were previously unable to participate in sports due to physical limitations, for example people with spinal cord injuries, or those with who had a stroke, to participate equally, regardless of individual impairments. Using the above system, the rules of the sport were designed and implemented. In this demonstration, the system explores the possibility of people coming together, racing and competing as one, transcending physical characteristics and physical locations.

We propose the Tactile Music Toolkit, a wearable device that delivers audio feedback to aid in communication and interaction for autistic children. The toolkit is wireless and is packaged into a compact module to promote more physical interaction among autistic children which can aid in their communication and social development. Our preliminary observations showed that our toolkit was fun to use and it helped them in their willingness to engage in physical contact.