Algebraic Visual Design for Deep Learning

Abstract

Deep learning has attracted voluminous attention from the academy and industry. However, understanding and tuning it to good use requires both domain knowledge in the applied field and deep understanding of model training strategies. Visual data analytics has played an important role in this regard, both helping domain experts to tune their models and deep learning experts to better reveal the model behavior. Typically, the quality of such visual analytic tools is evaluated by user studies or expert review. Although these stud- ies very specifically evaluated certain tasks performed with a visual system, less study has been focused on the proper use of visual elements in data analytics. Algebraic visual design (AVD) fills this gap by introducing a vivid language to think and reason about the design of visualizations. AVD takes an algebraic approach to consider the correspondence between data and visualization and develops simple yet effective tests to evaluate the quality of a particular design. In this dissertation, I consider three use cases of algebraic visual design for interpreting deep learning. The first project demonstrates the power of linear projection methods for understanding internal behavior of a single small neural network. The second project improves the technique used in the first visually compare the state-of-the-art neu- ral network architectures. The third project extends the scope of the method from neural network classifiers to generative adversarial networks. Through the three projects, I care- fully consider the structure of data produced in deep learning, and find appropriate visual counterparts that best reflect important aspects of the data.