Constructing and Assessing Surrogates for Volume Visualization Using Neural Networks

Abstract

Surrogates for volume visualization refer to a type of approach that utilizes a proxy for visualization, instead of rendering the underlying volume directly. They have seen increasing attention in recent years as they help mitigate challenges brought by the ever-growing scale of volumetric datasets. In the meantime, deep learning models have shown their excellence in representing complex structures, and have been frequently adopted for visualization problems. This dissertation aims to advance the use of surrogates for volume visualization by utilizing neural networks, and the contribution is two-fold: 1. We show that high quality compressive surrogates for volume visualization can be built with neural networks, and study the benefits and drawbacks of such design. 2. We build an interactive assessment interface for studying the impact of using surrogates for volume rendering. The cornerstone of the interface is an uncertainty-calibrated classification model that is tasked to differentiate images rendered with the surrogate from images rendered with the original volume. The interface complements the surrogate volume visualization pipeline by providing rich information for identifying and reasoning about visual artifacts caused by the surrogates.