Language and Cognition

Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no direct access to speech data

Gašper Beguš
Alan Zhou
2022

Human speakers encode information into raw speech which is then decoded by the listeners. This complex relationship between encoding (production) and decoding (perception) is often modeled separately. Here, we test how encoding and decoding of lexical semantic information can emerge automatically from raw speech in unsupervised generative deep convolutional networks that combine the production and perception principles of speech. We introduce, to our knowledge, the most challenging objective in unsupervised lexical learning: a network that must learn unique representations for...

Toward understanding the communication in sperm whales

J. Andreas
Gašper Beguš
M. Bronstein
R. Diamant
D. Delaney
S. Gero
S. Goldwasser
D. Gruber
S. de Haas
P. Malkin
N. Pavlov
R. Payne
G. Petri
D. Rus
P. Sharma
D. Tchernov
P. Tønnesen
A. Torralba
D. Vogt
R. Wood
2022

Machine learning has been advancing dramatically over the past decade. Most strides are human-based applications due to the availability of large-scale datasets; however, opportunities are ripe to apply this technology to more deeply understand non-human communication. We detail a scientific roadmap for advancing the understanding of communication of whales that can be built further upon as a template to decipher other forms of animal and non-human communication. Sperm whales, with their highly developed neuroanatomical features, cognitive abilities, social structures, and discrete...

Berkeley linguists published in Journal of Language Evolution

May 1, 2022

A new paper by Berkeley linguists and colleagues has just appeared in the Journal of Language Evolution:

Noga Zaslavsky*, Karee Garvin* (PhD 2021), Charles Kemp, Naftali Tishby, and Terry Regier. 2022. The evolution of color naming reflects pressure for efficiency: Evidence from the recent past. Journal of Language Evolution. (* = co-first authors, contributed equally)

Click here for the preprint PDF. Congrats to all!

Didn't hear that coming: Effects of withholding phonetic cues to code-switching.

Alice Shen
Susanne Gahl
Keith Johnson
2020

Code-switching has been found to incur a processing cost in auditory comprehension. However, listeners may have access to anticipatory phonetic cues to code-switches (Piccinini & Garellek, 2014; Fricke et al., 2016), thus mitigating switch cost. We investigated effects of withholding anticipatory phonetic cues on code-switched word recognition by splicing English-to-Mandarin code-switches into unilingual English sentences. In a concept monitoring experiment, Mandarin–English bilinguals took longer to recognize code-switches, suggesting a switch cost. In an eye tracking experiment, the...

Twenty-eight years of vowels

Gahl, Susanne
Baayen, Harald
2019

Research on age-related changes in speech has primarily focused on comparing “young” vs. “elderly” adults. Yet, listeners are able to guess talker age more accurately than a binary distinction would imply, suggesting that acoustic characteristics of speech change continually and gradually throughout adulthood. We describe acoustic properties of vowels produced by eleven talkers based on naturalistic speech samples spanning a period of 28 years, from ages 21 to 49. We find that the position of vowels in F1/F2 space shifts towards the periphery with increasing talker age. Based on...

Didn't hear that coming: Effects of withholding phonetic cues to code-switching.

Alice Shen
Gahl, Susanne
Johnson, Keith
2020

Code-switching has been found to incur a processing cost in auditory comprehension. However, listeners may have access to anticipatory phonetic cues to code-switches (Piccinini & Garellek, 2014; Fricke et al., 2016), thus mitigating switch cost. We investigated effects of withholding anticipatory phonetic cues on code-switched word recognition by splicing English-to-Mandarin code-switches into unilingual English sentences. In a concept monitoring experiment, Mandarin–English bilinguals took longer to recognize code-switches, suggesting a switch cost. In an eye tracking experiment, the...

The processing of pseudoword form and meaning in production and comprehension: A computational modeling approach using linear discriminative learning

Chuang, Y. Y., Vollmer, M. L., Shafaei-Bajestan, E., Gahl, S., Hendrix, P., & Baayen, R. H.
2020

Pseudowords have long served as key tools in psycholinguistic investigations of the lexicon. A common assumption underlying the use of pseudowords is that they are devoid of meaning: Comparing words and pseudowords may then shed light on how meaningful linguistic elements are processed differently from meaningless sound strings. However, pseudowords may in fact carry meaning. On the basis of a computational model of lexical processing, linear discriminative learning (LDL Baayen et al., Complexity, 2019, 1–39,...

Bilingualism as a Purported Risk Factor for Stuttering

Gahl, Susanne
2020

Berkeley linguists published in PNAS

December 8, 2021

A new article has been published in Proceedings of the National Academy of Sciences, co-authored by four current and former Berkeley linguists (the middle four authors). Congrats, all!

Francis Mollica, Geoff Bacon (PhD 2020), Noga Zaslavsky, Yang Xu, Terry Regier, and Charles Kemp. (2021). The forms and meanings of grammatical markers support efficient communication. Proceedings of the National Academy of Sciences, 118, e2025993118. [Preprint]

Identity-Based Patterns in Deep Convolutional Networks: Generative Adversarial Phonology and Reduplication

Gašper Beguš
2021

This paper models unsupervised learning of an identity-based pattern (or copying) in speech called reduplication from raw continuous data with deep convolutional neural networks. We use the ciwGAN architecture (Beguš, 2021a) in which learning of meaningful representations in speech emerges from a requirement that the CNNs generate informative data. We propose a technique to wug-test CNNs trained on speech and, based on four generative tests, argue that the network learns to represent an identity-based pattern in its latent space. By manipulating only two...