Conditional Graph Generative Models for Code and Texture Generation

Abstract

In this dissertation, I present two novel graph generative frameworks, the first for code abstract syntax tree (AST) generation from input binary code sequences, and the second for novel texture generation for input 3D meshes. Both frameworks are developed using insights from advances in neural machine translation, in the form of Transformers (multi-headed local and global self-attention), and conditional graph generative neural networks. In the first part of the dissertation, I describe the framework for inferring ASTs from binary sequences. Existing approaches to building decompilers and binary analysis tools are time-consuming and labor-intensive to implement and are not easily adapted to new languages. Existing neural approaches tend to be limited to small code sequences. To address these challenges, we develop a novel and state-of-the-art framework for learning to generate abstract syntax trees from input binary sequences. In the second part of the dissertation, I describe the development of the framework for novel but high-quality texture generation for input 3D meshes. Creating high-quality texture assets for a given 3D mesh model is tedious and time-consuming but has numerous applications in 3D simulation, gaming, and augmented and virtual reality. Existing automated approaches to this problem either require expensive 3D part segmentation or deform the original input mesh. To address these challenges, we develop a new framework that can learn to generate novel textures for 3D mesh models from a collection of 3D meshes and 2D real-world images without any input mesh deformation. Both frameworks are demonstrated to achieve state-of-the-art performance in their respective applications while also providing increased adaptive flexibility.