Published papers

This paper proposes a thin-film sensor that visualizes heat transfer at contact surfaces through color changes, using thermochromic pigments specifically designed for material identification. By capturing the color change as time-series image data, the spatial distribution of heat transfer across the contact surface can be obtained, enabling differentiation of contact materials. The sensor provides a visual representation of heat flow across the interface between the sensor and the object. Experimental results demonstrate the potential of the proposed sensor for material identification and spatial heat transfer analysis.

Thermal sensing enables the acquisition of thermal properties such as the material distribution and internal state of a contacted object, which cannot be obtained through tactile or visual sensing. However, conventional thermal sensors require an array of multiple sensors to improve spatial resolution, making it challenging to achieve high spatial resolution sensing at the contact surface. In this study, we developed a sensor capable of spatially visualizing heat transfer at the contact surface. We proposed a method to extract the material distribution of the contact object using the ratio of color changes in the sensor. The proposed sensor is constructed with color schemes and layered structures that establish a correlation between temperature and the LAB color space while reducing the influence of ambient light. Dynamic color variations are converted into the spatial heat transfer at the contact surface, allowing the discrimination of different contact materials. Moreover, the sensor enables visualization of the internal material properties of an object enclosed with different materials. The effectiveness of the proposed method is experimentally validated through distributed material identification of contact objects.

Robot skin is essential for achieving physical human-robot interaction. In particular, a warm feeling with a soft tactile impression is necessary for a human-friendly robot design. Contact sensing for safe interaction is also important; however, adding sensors to the skin surface can compromise its soft, smooth feel and create uncomfortable tactile sensations for humans. The paper proposes the design of a fluid-based soft robot skin that can simultaneously give thermal stimulation and detect human contact. We tested a prototype of the robotic skin, confirming that the fluid-based control successfully enabled both thermal display and contact detection capabilities.

Sensing and rendering thermal sensation is important to the multimedia, communication, and medical field. For example, human health care system for monitoring and treating a body temperature distribution can be estimated by using the technique. In this research, the thermal interface using multiple heat sources for rendering thermal sensation is treated. In order to control a lot of heat sources in the thermal interface, the distributed control method is required to render spatial continuous sensation. The paper defined "Distributed thermal conductance" that the relationship between heat sources, and heat flow on the palm can be freely changed depending on the connectivity between heat sources. The proposed method can be controlled the desired temperature distribution on the surface of the interface.

Recognition techniques allow robots to make proper planning and control strategies to manipulate various objects. Object recognition is more reliable when made by combining several percepts, e.g., vision and haptics. One of the distinguishing features of each object's material is its heat properties, and classification can exploit heat transfer, similarly to human thermal sensation. Thermal-based recognition has the advantage of obtaining contact surface information in real-time by simply capturing temperature change using a tiny and cheap sensor. However, heat transfer between a robot surface and a contact object is strongly affected by the initial temperature and environmental conditions. A given object's material cannot be recognized when its temperature is the same as the robotic grippertip. We present a material classification system using active temperature controllable robotic gripper to induce heat flow. Subsequently, our system can recognize materials independently from their ambient temperature. The robotic gripper surface can be regulated to any temperature that differentiates it from the touched object's surface. We conducted some experiments by integrating the temperature control system with the Academic SCARA Robot, classifying them based on a long short-term memory (LSTM) using temperature data obtained from grasping target objects.

Thermal sensation is important for communication tool and multimedia fields as an element of tactile sensation. This research focuses on thermal sensation, and aims to render the sensation spatially. Most of conventional studies present thermal sensation by directly contacting thermal device such as a Peltier device. In this case, it is difficult to get thermal information by the movements of their hands such as tracing. In this paper, a control method for rendering thermal sensation with the consecutive detection of contacting points is proposed. In order to consider temperature distribution of the system, a copper plate attached to the Peltier devices is regarded as a thermal system. A method to detect multiple contacting points is proposed, which uses the value of heat inflow estimated from heat inflow observer, and temperature control at the detected points is conducted. This method enables the system to render spatial thermal sensation in response to the movements of finger. Experimental results show the viability of the proposed control system.

To encourage interaction between a human and a robot, a technique of giving a thermal sensation is needed. In particular, the technique is important for the case of offering a sense of security such as nursing care robot and rehabilitation robot. Thermal sensation is perceived from the temporal and spatial information of heat sources. Therefore, the model included such information is needed to control multiple heat sources as a single system. In this paper, the fingerstall attached four heat sources was developed and its control method for rendering spatial thermal sensation is proposed. The four heat sources are attached to the copper plate, and the human finger is thermally stimulated by touching the plate. The temperature distribution on the copper plate is modeled based on a two-dimensional thermal diffusion equation, and spatial thermal sensation can be rendered using the proposed system. Some experiments were conducted to verify the proposed thermal system and its control system.

The heat generated inside robots originates mainly from energy loss in actuators, onboard electronic circuits, and computation processing units. Typically, it can be dissipated by forced air (most commonly used in robotics) or fluid convection. In this study, we developed a recycled inner-generated heat system inspired by biological thermoregulation mechanism, utilizing the inner heat to the robot's thermal perceptions of a finger-pad (by analogy to human fingertip) for object haptic recognition. Among the fingertips' thermal perceptions, material recognition identifies and distinguishes touched objects, even if the color, stiffness, or roughness are similar. This recognition approach requires a heat source to induce temperature changes at the contact surface to recognize ambient (room) temperature objects. We use an actuator that generates heat in a closed-flow water circuit (by analogy to human cardiovascular system) to induce heat at contact like humans with body temperature and touch. Our thermal method is assessed through experimental simulations of robotic water circulation and a pump system (by analogy to human heart) with the developed finger-pad. The proposed strategy enables it to completely classify three kinds of material covering the same material in 0.7 sec touch.

Human sensing and tactile transmission utilizing thermal interface are necessary to support human life or for communicating with people in remote locations. For instance, the devices are used for remote health care, telesurgery, and so on. Many studies have utilized a Peltier device as the thermal interface, and there are many studies on the control and sensing of the device. However, in most of those studies, the heat propagation in the system is not considered, and only the point that is attached to the sensor can be observed and controlled. In particular, this is a problem for health care systems, because it is difficult to obtain information about the inside of the human body where it is hard to attach the sensor. In order to solve this problem, this study considers the heat propagation of materials such as the human body by developing a model based on the thermal diffusion equation. In the case of cooling material, heat propagation changes depending on the material condition, for example, the presence of heat source such as a cancer cell. Based on the thermal characteristics, a method to obtain the heat flow of material using heat propagation is proposed in this paper. It is possible to find a heat source in deep layers of a material by using the proposed method without attaching any sensor inside the material. Some experiments are conducted using aluminum cylinder as a propagation material to verify the validity of the proposed method.

Heat sensing is necessary for human health care and industrial fields. In many cases, the abnormality of human body or machine is appeared as a heat source. For example, a cancer cell is higher temperature by up to 2 or 3 degree on the skin surface over the tumor. Therefore, abnormal detection is possible to sense through the heat distribution in the material. There are many sensors for observing temperature such as thermocouple, thermistor, thermography, and so on. However, sensing in deep layer of material using these sensors are difficult. This paper proposed the sensing method for detecting heat source inside the material by using heat flow. The method uses an algorithm that heat flow is changed depending on temperature difference. Some experiments are conducted to verify the proposed method.

This paper presents a method of simultaneously presenting tactile and thermal sensations using multilateral tele-operation. Communication using haptic sensation is expected to be a next-generation communication tool replacing telephone or television. A control technique named multilateral control can transmit haptic sensation to multiple points. In this study, robot manipulators are used for tactile sensation presentation, and Peltier devices are used for rendering thermal sensation. A position encoder is used to measure the position of a robot and also estimate the external force, while temperature and heat flow sensors are used for rendering thermal sensation. A routing method to decouple controllers compensates the effect of a time delay between the systems. Defining a modal transformation matrix in a network system eliminates the interference effect of controllers, which is a well-known problem in bilateral and multilateral teleoperation. The matrix shows how to route the information to realize mode-decoupled control in network systems. To consider the motion range of a robot manipulator, the scaling gain is inserted into the modal transformation matrix. The method also compensates the difference between the response speeds of robot manipulators and Peltier devices. This should be considered because the response speed of a Peltier device is slow compared to that of a robot manipulator. The validity of the proposed method is confirmed through experiments.

AbstractAlmost all robotic systems in use have hard shells, which is limiting in many ways their full potential of physical interaction with humans or their surrounding environment. Robots with soft-shell covers offer an alternative morphology which is more pleasant in both appearance and for haptic human interaction. A persisting challenge in such soft-shell robotic covers is the simultaneous realization of softness and heat-conducting properties. Such heat-conducting properties are important for enabling temperature-control of robotic covers in the range that is comfortable for human touch. The presented soft-shell robotic cover is composed of a linked two-layer structure: (1) The inner layer, with built-in pipes for water circulation, is soft and acts as a thermal-isolation layer between the cover and the robot structure, whereas (2) the outer layer, which can be patterned with a given desired texture and color, allows heat transfer from the circulating water of the inner part to the surface. Moreover, we demonstrate the ability to integrate our prototype cover with a humanoid robot equipped with capacitance sensors. This fabrication technique enables robotic cover possibilities, including tunable color, surface texture, and size, that are likely to have applications in a variety of robotic systems.

Technique of rendering thermal sensation can connect people who are in remote locations, such as teletherapy and a multimodal system. Humans feel detailed temperature distribution and recognize the characteristic of a contacting object. Although, conventional method can only render the thermal sensation of a fingertip or a part of a hand. In the case of combining these techniques with robot for human support, the robot can only touch one point of a material. This paper proposed the method of temperature control for rendering thermal sensation considering spatial diffusion. The method can spread the area of rendering and help rendering spatial sensations. The validity of the proposed method is verified through some experiments.

Recently, tactile transmission utilizing wearable devices are necessary to support human life or communicating with people in remote locations. One key factor of transmitting tactile sensation is thermal sensation. Flexible thermal devices have been developed, however, precise temperature distribution cannot be reproduced due to the large size and the flexibility of the devices. Therefore, a technique of rendering thermal sensation considering heat propagation is required to obtain the desired temperature distribution. In this paper, a method to control temperature on a curved surface for implementing to wearable devices is proposed. This method can be applied to various shapes of devices because the deformation of a device can be observed by the propagation delay of the heat flow. Experiments are conducted to verify the validity of the proposed method.

Rendering technique of tactile sensation is necessary to communicate and share feelings between remote locations. This research focuses on thermal sensation, and not only precise rendering of tactile information but also environmental identification is conducted. Thermal sensation is changed depending on thermal impedance such as thermal resistance and thermal capacitance. In this paper, the identification method of thermal impedance is proposed, and rendering the identified thermal parameters is conducted. The method consists of identification using thermal conductivity and reproduction based on thermal impedance control. In order to identify thermal parameters of an object, an iterative least square technique is used. The proposed method is able to render thermal response precisely for human because thermal impedance is identified based on heat flow in the same way as human skin sensation. The validity of the proposed method is verified through some experiments.

Controlling heat flow is necessary in many fields, such as the communications, health care, and manufacturing industries. In the communication field, the techniques for the reproduction of thermal sensations have been researched for many years. In order to extend the area of controlling the heat, the thermal diffusion in time and space needs to be considered. However, attaining precise control based on conventional methods is difficult because it requires several sensors and devices to monitor and control the temperature distribution. In this study, the thermal system consists of a metal wire and a Peltier device attached to the end of the wire, and a one-dimensional thermal propagation system is constructed. Modeling the thermal diffusion equation, it was found that the thermal system has characteristics of damping and propagation delay. Therefore, a control method using these system characteristics is proposed. The thermal system is treated as a time delay system with damping, which is compensated by a time delay and damping compensator. The thermal diffusion through the system can be considered by the proposed method. Experimental results show the viability of the proposed control system.

Actuator arrays that consist of a planar arrangement of actuators are widely used for transportation of objects or shaping a rough surface by controlling the actuators cooperatively. In particular, thermal displays that consist of multiple heat sources spread over a surface have been developed for many years. Generally, the expression ability is determined by the number of actuators in actuator arrays, and accordingly, thermal displays require many heat sources to ensure a high spatial resolution of temperature distribution. However, the number of usable heat sources is limited because of mechanical issues. In order to solve this problem, a control method that employs a model derived from a thermal diffusion equation and is used between two heat sources, is proposed in this paper. By using the proposed method, any selected point of the heat conduction can be expressed using only one heat source. Therefore, infinite number of nodes of propagation can be reproduced by selecting a desired node. The method is expected to extend the technology for controlling multiple actuators or heat sources on a surface.

A thermoelectric device called a Peltier device is widely used in many fields. For example, the device is used for a cooling system such as in systems consisting of medical reagent coolants and refrigerators. However, controlling the device is difficult because thermal characteristics of the device change depending on its temperature. Notably, the amount of heat flow generated from the device varies with the amount of current applied. Therefore, a thermal nonlinear behavior has to be considered in a thermal model. Most studies do not consider the nonlinear characteristics and treat the device as a linear system. Therefore, robustness against disturbance is not high because of modeling error. This paper presents a heat disturbance observer considering the nonlinear characteristics of a Peltier device. Variable parameters depending on the temperature and current of the device are used, and a variable heat disturbance observer is proposed. The thermal model used in the proposed observer includes a few nonlinear elements such as Joule heat and heat flow inside the device. The objective of this study is to improve the response of a Peltier device. Some experiments were conducted to verify the validity of the proposed method. Besides, the effect of modeling error on the stability of the observer is discussed.

Developing a thermal interface to share a thermal sensation between remote locations can lead to significant progress in the field of haptics. The research focuses on a technique of rendering thermal sensation that is an essential element of the haptics for touching human and machines each other. The thermal technique is required in case of offering a sense of security and sharing a precise hand with thermal sensations between remote places such as nursing care and rehabilitation robot. This paper proposed a method to control heat conduction on a palm by setting virtual thermal conductance between two fingers of the glove. Two thermal gloves are used as a master/slave systems, and the virtual thermal conductance can change heat conduction on the palm. The proposed interface and its control algorithm can be used to share a spatial thermal sensation on a palm. The experimental results indicate the validity of the proposed method. The interface and its control system are expected to connect people in remote locations, and the developed system can potentially be used in medical and multimedia applications.

This paper describes the cybernetic avatar system developed by Team JANUS for connectivity, exploration, and skill transfer: the core domains targeted by the ANA Avatar XPRIZE competition, for which Team JANUS was a finalist. We used as an avatar a humanoid robot with a human-like appearance and shape that is capable of reproducing facial expressions and walking, and built an avatar control system that allowed the operator to control the avatar through equivalent mechanisms of motion; that is, by replicating the upper-body movement with naturalness and by stepping to command locomotion. In this way, we aimed to achieve high-fidelity telepresence and managed to be well evaluated from the point of view of the operator during the competition. We introduce our solutions to the integration challenges and present experimental results to asses our avatar system, together with current limitations and how we are planning to mitigate them in future work.