October 11, 2024
Keio University
Future University Hakodate
University of Tsukuba
A research group—comprising Associate Professor Yuta Sugiura and Visiting Researcher Sora Amasaka from the Faculty of Science and Technology at Keio University; Shunta Suzuki, a first-year master's student in the Graduate School of Science and Technology; Associate Professor Hiroki Watanabe from the School of Systems Information Science at Future University Hakodate; and Professor Buntarou Shizuki from the Faculty of Engineering, Information and Systems at the University of Tsukuba—has developed a new method called "EarHover." Focusing on the potential to repurpose sound leakage from hearable devices (high-functionality earphone-type devices) as a signal source, this method uses machine learning to detect and classify mid-air gestures performed near the device.
While the mainstream method of operation for many hearable devices involves contact with the device using touch sensors, enabling the recognition of mid-air gestures performed above and near the device makes it possible to operate them without direct physical contact.
The research group focused on the phenomenon of sound leakage from hearable devices, noting that it can be utilized for sensing the area around the device. This method leverages the Doppler effect, which occurs when an inaudible ultrasonic signal played from the hearable device reflects off a hand performing a mid-air gesture. Furthermore, this method can be used in specific environments where conventional methods are difficult to use, such as when the user's hands are dirty. In addition, because it can be implemented using only the built-in speakers and sound-capturing microphones of hearable devices, low-cost integration into commercial products is anticipated.
This research was supported in part by JSPS KAKENHI (JP23KJ1884, JP21H03485) and JST PRESTO (JPMJPR2134, JPMJPR2138). The results of this research have been accepted at "UIST '24: The ACM Symposium on User Interface Software and Technology," one of the premier international conferences in the field of human-computer interaction, and has been selected to receive a Best Paper Award.
For the full press release, please see below.