Presentations

フランスパリにて開催された国際音楽情報処理学会にて、ハーバードビート評価テストのiOSアプリケーションの信頼性についてブランドアルトマン解析と級内相関係数を用いて検討した結果を報告した。

Playing musical instruments involves auditory-motor coordination and has been essential to our human culture. Researchers have been trying to understand the developmental origin of our musicality to play the instruments, yet early manifestations of auditory-motor interactions have not been fully investigated. In this study, we aimed to investigate the change of limb movements in response to auditory feedback of percussion sounds in three-months old infants. In the experiment, we attached a virtual drum-kit device to the limbs to provide feedback of percussion sounds. First, we measured the infant’s movements without the feedback for two minutes (the baseline phase or ‘Base’). Next, feedback was provided to the infants for eight minutes (the feedback phases, divided into four two-minutes phases or ‘Feedback 1-4’). After that, the feedback was removed for two minutes (the washout phase or ‘Post’). The movements were measured with a video camera and the data were analyzed with OpenPose software to detect the left and right wrist and ankle joint positions. We assessed the amount of movement of seven infants by calculating the length of the limb trajectories. Using one-way analysis of variance (ANOVA), we found a significant increase of the limb movement after the second phases of feedback (Feedback 2-4 and Post) compared with the first phase of the feedback (Feedback 1), p < 0.05, showing that the movements became more active after the Feedback 2. The results suggest that three-months-old infants already primed their bodies to interact with the auditory feedback of percussion sounds.

Music is present in every known culture, implying some biological basis. Yet the nature and extent of biological constraints have remained unclear, in part because cross-cultural comparisons have been limited. We measured a signature of mental representations of rhythm in over 500 participants from 13 countries on four continents, spanning modern societies and traditional indigenous populations belonging to 27 subgroups with varied musical expertise. Listeners were asked to reproduce random ‘‘seed’’ rhythms; their reproductions were fed back as the stimulus (as in the game of “telephone”), such that their biases (the prior) could be estimated from the distribution of reproductions (Jacoby and McDermott 2017). Every tested group showed priors with peaks. These peaks always overlapped with integer ratio rhythms, supporting the idea that rhythm “categories” at integer ratios are universal. However, the relative importance of different integer ratios varied considerably across cultures. Rhythmic prototypes in many cases reflected rhythms prevalent in the musical systems of a participant group’s culture. However, university students in non-Western countries tended to resemble Western participants, underrepresenting the variability evident across indigenous participant groups and highlighting the problematic over-reliance on student participants in cognitive science (Henrich et al. 2010). The results illustrate consistency in rhythm perception amid cultural variation, demonstrating biological constraints and their interaction with culture-specific traditions.

Electrical Muscle Stimulation (EMS) is a technique that allows actuating muscles by means of electrical impulses. This technique has been used for rehabilitation for stroke patients [Hummelsheim, et al., 1997]. More recently, EMS has been used not only for clinical cases but also in interactive systems that stimulate the user’s muscles for a variety of tasks, such as how to operate everyday objects [Lopes et al., 2015] or to practice drumming patterns [Ebisu et al., 2017]. The latter, opens up interesting research questions that remain unanswered: Does the use of EMS improve the performance of rhythmic tasks? In this study, we set out to explore how EMS affects memorization of time intervals. So far, auditory stimulation has been the major method by which researchers explore how we memorize time intervals. However, we argue that stimulating also the subject’s proprioception may reveal some benefits. We compared auditory and EMS conditions using tasks of a previous study by Mioni et al. (2014). After a time duration of 1, 4, 9, 14, 18 sec (inter-stimulus interval: ISI), participants were asked to memorize for either the intervals of 0, 5, and 15 sec (memory interval: MI), then reproduce the ISI. We recruited five participants (3 female, age = 21~23). By analyzing the difference between the time reproduction and target duration (absolute error), the ratio between time reproduction and target duration (RATIO) using repeated measures ANOVA, we found that there were no significant interactions between ISI × conditions and MI × conditions nor significant main effect of condition in both absolute error and RATIO. We further conducted an additional experiment of detecting small changes in the target duration to assess the ability of time duration perception, which showed that there was no difference between EMS and auditory conditions. These results suggest that EMS may not have positive effect on time duration perception and reproduction.

Expert drummers exhibit virtuoso skills such as rapid tapping with a drumstick. As a reference, the winner of a contest to find the world’s fastest drummer (WFD) performed a record of about 1200 taps with hands in a single minute [Fujii, et al., Neurosci. Lett., 2009], which equals 10 Hz movement with each hand over the 60 sec. Acquiring such very rapid drumming skill is laborious for untrained individuals and beginner-level drummers. While previous studies showed augmented feedback modalities (such as auditory feedback [Fujii, et al., Front. Neurosci., 2016]) can boost the motor learning experience; we believe that an active approach, i.e., one that stimulates directly the learner’s muscles, can provide even more benefits. As such, we turned to Electrical Muscle Stimulation (EMS) as a means to assist in learning very rapid drumming. We present an exploratory study, where we trained three participants that were new to drumming. One participant was trained using an EMS system we developed that contracted the participant’s wrist flexor at 10 Hz (same speed as WFD); while the other participants formed our control group that performed an unassisted training. In the first day prior to the start of the study, we measured participants’ drumming speed using the standard 60s drumming task (same as the official WFD contest). Then, participants trained for 3 days, performing 48 trials in a day. In each day, participants practiced by alternating hands, one hand at a time for 10s (15s rest). At the last day, we measured their drumming speed for the 60s task. Our results show both in feedback and non-feedback, individuals improved their drumming speed. In particular, we found that the participants who trained with EMS improved their ability to perform: (1) more symmetrical drumming between hands and (2) more stable inter-tap interval compared to the control group.

Throughout human history, music has been used to boost cohesiveness of groups. From sacred songs in religious rituals, to marching in armies, to dancing at rock concerts, music has been used to bring a sense of unity to diverse groups. Several previous experimental studies have suggested that rhythmic synchronization is the key mechanism by which music facilitates cooperation (Wiltermuth & Heath, 2009, Psychological Science; Reddish et al., 2013, PLOS ONE; Mogan et al., 2017, J. Experimental Social Psychology). However, these experiments were not preregistered, had small effects, and did not dissociate the effects of rhythm, melody, lyrics, and movement. Thus, if and how music has a causal effect on cooperation remains poorly understood. We performed a (non-preregistered) pilot study tentatively confirming that music (singing “Twinkle Twinkle Little Star” in pairs) can cause a slight increase in cooperation as measured through an economic task, relative to a null control (n =103 participants, t = 2.1, p = .03). By SMPC we plan to perform additional pilot experiments using a more nuanced paradigm comparing a null control against the sequential addition of 1) lyrics, 2) rhythm, 3) melody, 4) movement, and 5) large groups We predict that each of these factors will result in a sequential increase in cooperation, but that the increase will be strongest for rhythm. Based on these pilot experiments and feedback from SMPC we plan to conduct a full preregistered experiment to disentangle the multidimensional mechanisms by which music may facilitate cooperation.

Musical pitches are one of the core building blocks of almost all of the world’s music (Savage et al., 2015, PNAS). In many cultures, different relationships between pitches are associated with different aesthetic responses. For example, in most Western music, consonant intervals based on simple integer ratios (e.g., octaves, perfect fifths) are perceived as pleasant while dissonant intervals based on more complex ratios (e.g., 2nds, 7ths) are perceived as unpleasant (McDermott et al., 2010, Current Biology). However, there remains limited experimental data for intervals outside of the standard 12-note Western chromatic scale. Here we present experimental data on aesthetic responses to all possible interval dyads within a single octave based on a quartertone (50-cent) 24-note equal-tempered scale, including both melodic and harmonic intervals. Pilot data from four Japanese participants suggest that for both melodic and harmonic intervals, microtonal intervals are perceived as less pleasant than chromatic intervals (melodic: t = 2.5, p = .02; har- monic: t = 3.9, p = .0008) and that aesthetic ratings are correlated with the degree of harmonic similarity between overtone spectra (melodic: r = .68, p = .0002; harmonic: r = .42, p = .04). These results provide tentative support for the role of general psychoacoustic harmonicity principles in shaping aesthetic preferences to music, with the caveat that these pilot data come from a small sample with extensive exposure to Western musical systems. By SMPC we plan to collect additional data from Japanese participants both with and without extensive training in Japanese traditional music to investigate the role of musical experience in shaping aesthetic responses to musical intervals.

Background: Manual skills of professional musicians can be regarded as a model to investigate human motor skill acquisition after prolonged practice. Previous study showed that pianists was able to perform single finger tapping faster than non-pianists especially with the ring finger (Aoki, et al., Motor Control, 2005). Double-finger tapping of pianists was also faster than that of non-pianist. Although tapping skills of drummers were investigated previously (Fujii et al., Neurosci Lett, 2009), finger coordination skill of hand percussionist is still largely unknown. In this study, we investigated the finger tapping skill of professional hand percussionists (Darbuka players). Methods: Eight professional percussionists and eight amateur percussionists participated in this study. All participants were right-handed. They were asked to perform finger tapping as fast as possible for 12 seconds. There were eleven finger tapping tasks: four single-finger tapping task (i.e., each of the left-index, left-ring, right-index, right-ring fingers), six double-finger tapping task (i.e., all combinations of the above four fingers), and one triple-finger tapping task (i.e., right-index, left-index, and left-ring finger coordination). The tasks were repeated three times each. The mean tapping frequency was calculated for each trial and averaged over the three trials. Results: As for the single-finger tapping task, there was no significant difference between the professional and amateur percussionists. As for the double- and triple-finger tapping tasks, professional percussionists performed faster than the amateur percussion- ists (P < 0.05). Conclusion: The professional percussionists showed faster performance in the double- and triple-finger tapping tasks but not in the single-finger tapping task compared with the amateur percussionists. The result suggest that professional percussionists have acquired organized finger coordination skill over the prolonged practice.

Background: Amusia is often found in patients with schizophrenia. Although previous studies suggested that there is a close link between amusia and cognitive dysfunction, the relationship between beat deafness and cognitive dysfunction in schizophrenia has not yet been fully elucidated. Patients with schizophre- nia can be subdivided into Treatment-Resistant Schizophrenia (TRS) and non-TRS based on antipsychotic treatment response. However, there has been no study investigating the difference of beat-processing ability between TRS and non-TRS. Thus, we aimed to investigate the relationship between beat deafness and cognitive dysfunctions in patients with schizophrenia. Methods: Fifty-eight patients with schizophrenia (27 TRS and 31 non-TRS) and thirty healthy controls (HC) participated in this study. To assess beat- processing ability and cognitive performance, we used the Harvard Beat Assessment Test (H-BAT) and Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), respectively. H-BAT has subtests for assessing beat perception and production abilities. We performed a two-way analysis of vari- ance and post-hoc analyses for the H-BAT measures. The within-subject factor was beat processing [perception/production] and the between-subject factor was group [TRS/non-TRS/HC]. We conducted the partial correlation analyses between the difference of perception and production abilities and cognitive performance, controlling for severity of extrapyramidal impairment as well as chlorpromazine equivalent dose. Re- sults: The ANOVA showed a significant interaction between beat-processing ability [perception/production] and groups[TRS/non-TRS/HC] (F2,64=4.84, p=0.016). Compared with HC, TRS group had lower beat perception and production ability while non-TRS group had lower beat perception ability (F2,84=6.45, p=0.002). There were correlations between the beat processing ability and RBANS scores in patients with non-TRS (r=0.55, p=0.003). Conclusions: There may be a significant difference in the beat-processing ability be- tween TRS and non-TRS, suggesting a close link between the beat-processing and the pathophysiology of antipsychotic response in patients with schizophrenia. Future studies are needed in order to clarify the neural underpinnings.

Music listening sometimes induces the sensation of frisson, a pleasant tingling feeling accompanied with raised body hairs and gooseflesh. Some believe that we feel an emotion and physiological response occurs subsequently. On the contrary, the James-Lange theory proposes that physiological change is primary, and emotion is then experienced. Considering the James-Lange theory, we hypothesized that providing cold stimulation to a skin while listening to sounds may change our sound perception. Here we tested this hy- pothesis by providing cold stimulus to skin during listening to consonant and dissonant intervals. Ten healthy students participated in the pilot study. We used perfect fifth and minor second. Each of the chord had two different root notes in order to counterbalance the pitch. A cold stimulation was provided to the mastoid using a cooling device. As a control sham condition, we touched the skin with another cooling device while turning off the power. Participants rated the degree of frisson and pleasantness using the Visual Analog Scale. We performed a two-way repeated measures analysis of variance (ANOVA) with the factors of musical interval (consonant/dissonant) and stimulus condition (cold/control). The ANOVA showed no significant interaction between the musical interval and stimulus condition. As for the frisson rating, the main effect of stimulus condition was significant (F(1,9)=10.744 , p<0.01, η2=0.544), showing that the cool stimulus increased the frisson rating. The main effect of interval was not significant (F(1,9)=3.011, p=0.117, η2=0.251). As for the pleasantness rating, there was no significant main effect of stimulus condition (F(1,9)=0.003, p=0.957, η2<0.001). There was a significant main effect of interval (F(1,9)=18.116, p<0.01, η2=0.668), showing that perfect fifth was perceived as more pleasant than minor second. The results suggest that cold stimulation may increase the subjective frisson rating but not affect the pleasantness rating while listening to intervals. Our finding partially supported the James-Lange theory of emotion.

Music copyright lawsuits often result in multimillion dollar settlements, yet there are few objective guidelines for applying copyright law in infringement claims involving musical works. Recent research has attempted to develop objective methods based on automated melodic similarity algorithms (Müllensiefen & Pendzich, 2009, Musicae Scientiae; Savage et al., 2018, Proc. Folk Music Analysis), but there remains almost no perceptual data on the role of melodic similarity in music copyright decisions (Lund, 2011, Virginia Sports and Entertainment Law Journal). We conducted a pilot experiment (n = 19 participants) collecting perceptual judgments of copyright infringement for 13 copyright cases from the Music Copyright Infringement Resource database (mcir.edu) involving musical similarity for which we could prepare both full-audio and melody-only (MIDI) versions. Due to the historical emphasis in legal opinions on melody as the key criterion for deciding infringement (Fishman, 2018, Harvard Law Review), we predicted that listening to melody-only versions would result in perceptual judgments that more closely matched actual past legal decisions. Surprisingly, however. the reverse was true: participant judgments more closely matched past decisions when listening to full audio including non-melodic factors such as timbre, instrumentation, and lyrics (paired t = 2.2, p = 0.049). By SMPC we plan to expand this pilot study to include more copyright cases. Our findings have important practical implications regarding whether jury members should be allowed to listen to full audio recordings during copyright cases – a point of major contention during recent high-profile cases such as the dispute involving Blurred Lines.

Background: People with amusia show cortical thinning in the medial orbital frontal gyrus, anterior cingulate gyrus, and inferior temporal gyrus. It was noted that two thirds of patients with schizophrenia showed symptoms of amusia and that they showed malformations of cortical thickness in the frontal and temporal lobes. However, the link between beat processing ability and cortical thickness remains unclear in the schizophrenia. Thus, we investigated the relationship between beat processing ability and the cortical thickness in patients with schizophrenia. Methods: Fifty-five patients with schizophrenia and age- and sex- matched 25 healthy controls participated in this study. We used the Harvard Beat Assessment Test (H-BAT), the Positive and Negative Syndrome Scale (PANSS), and the Simpson-Angus Scale (SAS) to assess beat-processing ability, clinical severity, and extrapyramidal symptoms, respectively. Participants were scanned with a GE 3.0T MRI scanner and the cortical thickness was evaluated with the FreeSurfer software. Pearson’s correlation coefficients were calculated between the cortical thickness and H-BAT scores in the control group. For the patient group, the partial correlation coefficients were calculated controlling for PANSS and SAS scores, and daily antipsychotic dose. Results: The cortical thickness of medial orbitofrontal cortex was negatively correlated with the H-BAT scores in the patient group (r=-0.45, p<0.001), and positively in the control group (r=0.46, p=0.02). No other region showed these correlations. Conclusion: Cortical thinning in the medial orbitofrontal cortex may relate to impaired beat-processing ability in patients with schizophrenia. Increased cortical thickness may also explain the individual difference in the beat processing ability in healthy controls. Future research will be needed for the interpretation of the relationship. This research will ultimately contribute to the elucidation of the relationship between the symptoms of amusia and pathophysiology of schizophrenia.

Human music throughout the world uses diverse scales, but there are also some commonalities shared throughout much of the world’s scales (Savage et al., 2015, PNAS). Are there musical laws or biological constraints that underlie this diversity? The origin of scales have been based on perceptual theories involv- ing small integer ratio of interval since the time of Pythagoras (Bowling & Purves, 2015, PNAS). However, these theories are generally based on tunable instruments, and some are skeptical as to whether this theory applies to vocal song, recognized as the most ancient and universal instrument of human music. One al- ternative theory is that scales arise not from perceptual constraints regarding integer ratios but instead due to production constraints on how precisely the voice can generate pitches (Pfordresher & Brown, 2017, J. Cognitive Psychology). To investigate this theoretical dispute, we conducted automatic comparative analy- sis of 100 matched children’s and adult songs (n = 50 songs each) from around the world using automated pitch analysis (Six et al., 2013, J. New Music Research). We found that children’s songs tend to have more imprecise tuning and fewer scale degrees than adult songs (precision: t = 2.0, p = .009; scale degrees: t = 1.7, p = .007), consistent with motor constraints due to their earlier developmental stage. On the other hand, we also found that both adult and children songs throughout the world shares some common tuning intervals such as perfect 4th (4:3) and perfect 5th (3:2), consistent with sensory theories of integer-ratio preference. These results suggest that some universal aspects of musical scales may be caused by a combination of sensory and motor mechanisms, which we unify into a new sensorimotor hypothesis.

The Harvard Beat Assessment Test (H-BAT) is a battery of tests to assess individual differences in the abilities to perceive, produce, and synchronize with a music beat [Fujii & Schlaug, 2013]. However, the original version of H-BAT is difficult to distribute because it requires particular hardware setups. Thus, we developed an iOS application of H-BAT and reported the results of the reliability previously [Konno, et al., International Conference on Music Information Retrieval, 2018]. However, the number of participants was limited in the previous report, and the reliability across the different iOS devices had not been tested. Here, we increased the number of participants and compared the reliability across different iOS devices to further test the reliability of the H-BAT iOS application. The H-BAT was run on iPad Pro 9.7 inch and 10.5 inch (iOS 11 and iOS 10, respectively), and the data from 21 participants were recorded. Ten participants were tested with the iPad 9.7 inch while eleven participants were tested with the iPad 10.5 inch. For each of the iPad, tapping was recorded on both of the iOS and Matlab setups. The reliability between the iOS and Matlab setups were evaluated with the Intra-class Correlation Coefficients (ICC) and the Bland-Altman methods. As for the inter-tap interval measure, the ICC was very high between the iOS and Matlab setups on both iPad devices (iPad Pro 9.7 inch, ICC = 0.996~0.997; iPad Pro 10.5 inch, ICC = 0.999~0.999, respectively). We also confirmed that the asynchrony measure (i.e., the synchronization error between a tap and a beat) was highly correlated between the iOS and Matlab setups on both iPad devices (iPad Pro 9.7 inch, ICC = 0.995~0.995; iPad Pro 10.5 inch, ICC = 0.997~0.997, respectively). These results suggest that the H-BAT iOS application can be reliably used regardless of different iPad devices.

カナダモントリオールで開催された国際音楽知覚認知学会にて、ハーバードビート評価テストを用いた治療抵抗性統合失調症患者の判別分析の結果について成果を発表した。

カナダモントリオールで開催された国際音楽知覚認知学会にて、統合失調症患者のリズム知覚生成能と言語機能の関連について、脳構造イメージング解析の結果と共に成果を発表した。

身体運動可聴化技術を用いた乳児の音遊び行動とその生理的特性について発表した。