Augmenting Human Intellect: Automatic Recognition of Nonverbal Behavior with Application in Deception Detection

Abstract

Humans have long sought to use technology to augment human abilities and intellect. However, technology is traditionally employed only to create speedier solutions or more-rapid comprehension. A more challenging endeavor is to enable humans with technology to gain additional or enhanced comprehension that may not be possible to acquire otherwise. One such application is the use of technology to augment human abilities in detecting deception using nonverbal cues. Detecting deception is often critical, whether an individual is communicating with a close friend, negotiating a business deal, or screening individuals at a security checkpoint.The detection of deception is a challenging endeavor. A variety of studies have shown that humans have a hard time accurately discriminating deception from truth, and only do so slightly better than chance. Several deception detection methods exist; however, most of these are invasive and require a controlled environment.This dissertation presents a technological approach to detecting deception based on kinesic (i.e., movement-based) and vocalic (i.e., sounds associated with the voice) cues that is firmly grounded in deception theory and past empirical studies. This noninvasive approach overcomes some of the weaknesses of other deception detection methods as it can be used in a natural environment without cooperation from the individual of interest.The automatable approach demonstrates potential for increasing humans' ability to correctly identify those who display behaviors indicative of deception. The approach was evaluated using experimental and field data. The results of repeated measures analysis of variance, linear regression and discriminant function analysis suggest that the use of such a system could augment human abilities in detecting deception by as much as 15-25%. While there are a number of technical challenges that need to be addressed before such a system could be deployed in the field, there are numerous environments where it would be potentially useful.