Acoustic Analysis and Automatic Classification of Personality and Individual Expression from Speech

Abstract

Perceptions of the personality traits an individual displays through their speech and actions have many and varied applications. Because individuals constantly make assumptions about those around them during daily interactions, gaining insight into how these assessments are made may help researchers from both social science and computational backgrounds. This dissertation is comprised of related experiments that examine personality trait recognition for Big Five personality traits (John and Srivastava, 1999) and related evaluations of personal expression, such as emotion, sentiment, and truthfulness, in order to examine these topics through both social science and computational perspectives.In the first experiment, I examine the changes in ratings of perceived personality traits as the length and speech type of stimulus to which listeners are exposed changes, finding an effect of speech length for agreeableness, conscientiousness, and openness to experience, and an additional effect of speech style for female speakers for conscientiousness and neuroticism. This experiment furthermore examines the correlation between acoustic features and personality trait ratings, identifying intensity, speech rate, and harmonic-to-noise ratio as features of particular importance. I then present investigations into the importance of accurate transcriptions of speech in the automatic identification of personality, emotion, and sarcasm, adding to this an evaluation of how the way a personality trait identification task is formulated affects outcomes. I find that providing accurate transcriptions has some effect on model success, although this effect depends upon the particular task of interest. I furthermore identify that formulating personality trait identification as a five-class classification task of predicting the personality trait with the highest rating, or dominant personality trait, is more challenging than providing predictions about the level in which an individual possesses each personality trait with five-task binary or ternary classification systems. The final set of experiments examine methods for improving multitask neural network architectures aiming to predict personality traits and related personal expression tasks. The results of these experiments highlight the importance of careful task selection, task-dependent care with feature selection and consideration of dataset size and class imbalances. Together, the results of these experiments provide information to be used in future research in the form of recommendations for experiment design, dataset curation, and computational model building.