Published papers

Nonnative listeners are generally not as good as native listeners in perceptually restoring degraded speech and understand what was being said. The current study investigates how nonnative listeners of English (namely, native Japanese speakers who learned English as a second language) perceptually restore temporally distorted speech in their L2 English as compared with native English listeners (L1 English) reported in Ishida et al. (Cognition, 151, 68-75, 2016), and as compared with the listeners' native tongue (L1 Japanese). In the experiment, listeners listened to locally time-reversed words and pseudowords in their L2 English and L1 Japanese where every 10, 30, 50, 70, 90, or 110 ms of speech signal was flipped in time-these stimuli contained either many fricatives or stops. The results suggested that the intelligibility of locally time-reversed words and pseudowords deteriorated as the length of reversed segments increased in both listeners' L2 English and L1 Japanese, while listeners understood locally time-reversed speech more in their L1 Japanese. In addition, lexical context supported perceptual restoration in both listeners' L1 Japanese and L2 English, while phonemic constituents affected perceptual restoration significantly only in listeners' L1. On the other hand, locally time-reversed words and pseudowords in L1 Japanese were much more intelligible than those in L1 English reported in Ishida et al. It is possible that the intelligibility of temporally distorted lexical items depends on the structure of basic linguistic units in each language, and the Japanese language might have a unique characteristic because of its CV and V structure.

Listeners can understand speech even when its temporal structure is acoustically distorted. Ishida et al. (Frontiers in Psychology, 9, 1749, 2018) reported that native English speakers could comprehend English sentences using two types of temporal distortions: (1) speech signals divided into equally timed segments, with each segment reversed in time (locally time-reversed speech), and (2) speech signals with reduced modulation-frequency components shaping the amplitude envelope (modulation-filtered speech). While the results showed a similar pattern of intelligibility decline across these two conditions in English (a syllable-oriented language with consonant clusters) when degradation increased in six steps, it remained unclear whether this pattern holds in a linguistically distinct language like Japanese (a mora-oriented language with CV and V as basic linguistic units). The current study investigates how native Japanese speakers comprehend Japanese sentences under the same temporal distortions. In Experiment 1, participants listened to locally time-reversed Japanese sentences with segment intervals reversed at 10 ms, 30 ms, 50 ms, 70 ms, 90 ms, and 110 ms. In Experiment 2, the same participants listened to modulation-filtered Japanese sentences, where the modulation frequency components were low-pass filtered at cut-off frequencies of 32 Hz, 16 Hz, 8 Hz, 4 Hz, 2 Hz, and 1 Hz. Results showed that the intelligibility of locally time-reversed and modulation-filtered Japanese sentences decreased as distortion increased, with longer reversed segment lengths and lower cut-off frequencies. However, the patterns of intelligibility degradation in Japanese differed significantly from those in English. Thus, perceptual restoration may function differently depending on the basic linguistic units (mora vs. syllable).

People can understand speech even when the speech signal is divided into equally long segments and each segment is reversed in time (locally time-reversed speech). In addition, Ishida (Attention, Perception & Psychophysics, 83(6), 2675–2693, 2021) reported that Japanese words – composed of consonant–vowel (CV) units – were significantly more intelligible than English words when locally time-reversed. The current study investigates how tolerant and robust Japanese words are under more severe temporal distortions. In Experiment 1, native Japanese speakers listened to Japanese words and pseudowords that were locally time-reversed at intervals of 100, 120, 140, 160, 180, and 200 ms, which had not been previously examined. These lexical items contained either many fricatives or stops. Results showed that Japanese words were highly tolerant of local time reversal, even at these extreme durations. Perceptual restoration was sustained by dominant phoneme type (fricative-dominant > stop-dominant) and lexicality (words > pseudowords). In Experiment 2, participants listened to stop-dominant Japanese words and pseudowords, which were more susceptible to temporal distortion in Experiment 1. Temporal distortion was further increased by introducing extreme speech rates (fast vs. slow) while reversing the signal at 10, 30, 50, 70, 90, and 110 ms, commonly used intervals with normal speech rates. Results showed that stop-dominant Japanese words remained intelligible with increasing distortions, while pseudowords remained intelligible only up to 50 ms in the slow condition and became unintelligible in the fast condition. Overall, recognition of Japanese CV-based words was highly tolerant of severe temporal distortion, with perceptual restoration supported by dominant phoneme type, slower speech rate, and lexicality.

This study investigates how similarly present and absent English phonemes behind noise are perceived by native and non-native speakers. Participants were English native speakers and Japanese native speakers who spoke English as a second language. They listened to English words and non-words in which a phoneme was covered by noise (added; phoneme + noise) or replaced by noise (replaced; noise only). The target phoneme was either a nasal (/m/ and /n/) or a liquid (/l/ and /r/). In experiment, participants listened to a pair of a word (or non-word) with noise (added or replaced) and a word (or non-word) without noise (original) in a row, and evaluated the similarity of the two on an eight-point scale (8: very similar, 1: not similar). The results suggested that both native and non-native speakers perceived the ‘added’ phoneme more similar to the original sound than the ‘replaced’ phoneme to the original sound. In addition, both native and non-native speakers restored missing nasals more than missing liquids. In general, a replaced phoneme was better restored in words than non-words by native speakers, but equally restored by non-native speakers. It seems that bottom-up acoustic cues and top-down lexical cues are adopted differently in the phonemic restoration of native and non-native speakers.

Speech is intelligible even when the temporal envelope of speech is distorted. The current study investigates how native and non-native speakers perceptually restore temporally distorted speech. Participants were native English speakers (NS), and native Japanese speakers who spoke English as a second language (NNS). In Experiment 1, participants listened to "locally time-reversed speech" where every x-ms of speech signal was reversed on the temporal axis. Here, the local time reversal shifted the constituents of the speech signal forward or backward from the original position, and the amplitude envelope of speech was altered as a function of reversed segment length. In Experiment 2, participants listened to "modulation-filtered speech" where the modulation frequency components of speech were low-pass filtered at a particular cut-off frequency. Here, the temporal envelope of speech was altered as a function of cut-off frequency. The results suggest that speech becomes gradually unintelligible as the length of reversed segments increases (Experiment 1), and as a lower cut-off frequency is imposed (Experiment 2). Both experiments exhibit the equivalent level of speech intelligibility across six levels of degradation for native and non-native speakers respectively, which poses a question whether the regular occurrence of local time reversal can be discussed in the modulation frequency domain, by simply converting the length of reversed segments (ms) into frequency (Hz).

People can understand speech under poor conditions, even when successive pieces of the waveform are flipped in time. Using a new method to measure perception of such stimuli, we show that words with sounds based on rapid spectral changes (stop consonants) are much more impaired by reversing speech segments than words with fewer such sounds, and that words are much more resistant to disruption than pseudowords. We then demonstrate that this lexical advantage is more characteristic of some people than others. Participants listened to speech that was degraded in two very different ways, and we measured each person's reliance on lexical support for each task. Listeners who relied on the lexicon for help in perceiving one kind of degraded speech also relied on the lexicon when dealing with a quite different kind of degraded speech. Thus, people differ in their relative reliance on the speech signal versus their pre-existing knowledge.

This study investigates how similarly a person hears an existing and non-existing speech sound behind noise in L2, as compared to L1 reported in Mattys, Barkan, and Samuel (2014). Participants were Japanese native speakers who spoke English as a second language. They listened to English words and non-words in which a phoneme was covered by noise (added) or replaced by noise (replaced). The target phoneme was either a liquid or a nasal. In experiment, participants listened to a pair of a word with noise (added or replaced) and a word without noise (normal) in a row, and evaluated the similarity of the two by using an 8-point scale. The results suggested that L2 listeners perceived the added and replaced sound significantly differently. L2 listeners found the added sound (a phoneme + noise) more similar to a normal sound than the replaced sound (noise only), as was also reported in L1 listeners. At the same time, they also perceived the illusory sound of a missing phoneme in the replaced condition. A missing nasal was significantly more restored than a missing liquid. There was no lexical effect in perceptual restoration of phonemes among L2 listeners, although it was reported among L1 listeners.

The current study explores how native and non-native speakers cope with simultaneous articulatory and acoustic distortion in speech perception. The articulatory distortion was generated by asking a speaker to articulate target speech as fast as possible (fast speech). The acoustic distortion was created by dividing speech signals into small segments with equal time duration (e.g., 50 ms) from the onset of speech, and flipping every segment on a temporal axis, and putting them back together (locally time-reversed speech). This study explored how "locally time-reversed fast speech" was intelligible as compared to "locally time-reversed normal speech" measured in Ishida, Samuel, and Arai (2016). Participants were native English speakers and native Japanese speakers who spoke English as a second language. They listened to English words and pseudowords that contained a lot of stop consonants. These items were spoken fast and locally time-reversed at every 10, 20, 30, 40, 50, or 60 ms. In general, "locally time-reversed fast speech" became gradually unintelligible as the length of reversed segments increased. Native speakers generally understood locally time-reversed fast spoken words well but not pseudowords, while non-native speakers hardly understood both words and pseudowords. Language proficiency strongly supported the perceptual restoration of locally time-reversed fast speech.