Methods of Enhancing Performance in Integral Imaging Based Light Field Head Mounted Displays

Abstract

Integral imaging based (InI-based) light field (LF) displays are considered as one of the most promising methods for solving the well-known vergence-accommodation conflict (VAC) problem, which refers to the mismatch between the vergence and accommodation depth cues rendered by a display system. They offer a great opportunity to achieve true 3D scene rendering with correct focus cues by reproducing the directional rays apparently emitted by 3D points of different depths of the 3D scene, and therefore are capable of rendering correct focus cues similar to those observed from natural 3D scenes and address the VAC problem plaguing conventional stereoscopic 3D displays.Several pioneering works have already demonstrated examples of InI-based light field head-mounted display (HMD) designs for both immersive virtual reality (VR) and optical see-through augmented or mixed reality (AR/MR) applications. However, there are some key challenges yet to be solved to achieve a high performance wearable InI-based light field HMD. From the perspective of optical performance, the state-of-the-art InI-based LF-HMD prototypes are subject to the limits of narrow depth of field (DOF), low spatial resolution in both lateral and longitudinal directions, low image contrast, and unnatural out-of-focus blurring effects mostly due to inadequate elemental view number. From the perspective of viewing experiences, it also is subject to the limits of small viewing window size, large form factor, and view misalignment. In this dissertation, time multiplexed methods are adapted to enhance the optical performance and form factor of InI-based light field HMDs and a pipeline for calibrating an InI-based LF-HMD is proposed to account for the assembly misalignments. Based on time multiplexed schemes, two novel architectures are developed to overcome the performance limitations. The first one mainly focuses on depth enhancement, and the depth of field was extended without sacrificing spatial resolution or system form factor by incorporating a digitally switchable MLA. The second scheme can alleviate the tradeoff between the viewing number and spatial resolution by utilizing a programmable shutter array. The different applications and experimental validations of the time-multiplexed InI-based light field displays are also presented and discussed. To improve the view experience of an InI-based light field HMD system, the factors which affect rendering process are discussed and a mathematical model is developed to describe the whole rendering process. Based on the model, a camera-based calibration method is proposed and the parameters which precisely describe an InI-based system can be obtained. With these accurate parameters, a well-converged and undistorted 3D scene can be rendered.