The Lexicon is Shaped for Incremental Processing in a Noisy Channel

Abstract

Human language is a constantly evolving system, with properties of a language’s grammar being shaped, among other things, to benefit efficient communication (Zipf, 1949; Köhler, 1987; Gibson et al., 2019). One important area of a language’s grammar is its lexicon, or set of words and their corresponding phonological forms, and there is a great deal of evidence that the lexicons of the world’s languages are structured to be similar to abstract, maximally efficient communicative codes (e.g., Zipf 1935; Ferrer-i Cancho and Solé 2003; Piantadosi et al. 2009, 2011; Mahowald et al. 2018. In this dissertation, I will present additional evidence that the lexicons of natural languages are structured for efficiency, moving past abstract codes and focusing on how listeners process and identify words in speech. Primarily, I will show that the distribution of inter-lexical contrasts, i.e., phonemes, in a language is such that the average Shannon information (Shannon, 1948) of phonemic contrasts is greater than would be expected otherwise, using a typologically and geographically diverse dataset of 25 languages. In addition, I will show that the increased informativeness of lexical contrasts does not interfere with a lexicon’s potential for accurate communication. Together, these offer strong support that languages evolve to be efficient communication systems, tailored to human users.