Toward Joint Scene Understanding Using Deep Convolutional Neural Network: Object State, Depth, and Segmentation
MetadataShow full item record
PublisherThe University of Arizona.
RightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction, presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
EmbargoRelease after 12/11/2021
AbstractSemantic understanding is the foundation of an intelligent system in the field of computer vision. Particularly, the real-time usage of the automation systems, such as robotic vision, auto-driving, and surgical training applications, has been in high demand. The models require to capture this variability of scenes and their constituents (e.g., objects or depth) given the limited memory and computation resources. To achieve the goals of real-time usage in semantic understanding, we propose a series of novel methods for object state, depth, and segmentation in this dissertation. We first present a semantic object model for simplifying the object state detection process. We then propose a novel method of monocular depth estimation to retrieve the 3D information effectively. Lastly, this dissertation presents a multi-task model for semantic segmentation and depth estimation. We train and verify the proposed method by using two public datasets of outdoor scenes that are meant to be applied to auto-driving applications. Our method successfully achieves 60 frames per second with a competitive performance compared to the current state-of-the-art in the benchmark. In the empirical experiments, we have applied our method to a simulated laparoscopic surgical training system: Computer Assisted Surgical Trainer (CAST). One of the CAST training tasks, Peg Transfer Task, is selected to be the evaluation platform. In this experiment, our method has demonstrated promising results for supporting a real-world application in medicine.
Degree ProgramGraduate College
Electrical & Computer Engineering