Date of Award

5-2024

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Mathematics and Systems Engineering

First Advisor

Ryan T. White

Second Advisor

Xianqi Li

Third Advisor

Munevver Mine Subasi

Fourth Advisor

Venkat Keshav Chivukula

Abstract

This dissertation explores applications of representation learning and generative models to challenges in healthcare, astronautics, and aviation.

The first part investigates the use of Generative Adversarial Networks (GANs) to synthesize realistic electronic health record (EHR) data. An initial attempt at training a GAN on the MIMIC-IV dataset encountered stability and convergence issues, motivating a deeper study of 1-Lipschitz regularization techniques for Auxiliary Classifier GANs (AC-GANs). An extensive ablation study on the CIFAR-10 dataset found that Spectral Normalization is key for AC-GAN stability and performance, while Weight Clipping fails to converge without Spectral Normalization. Analysis of the training dynamics provided further insights into the impact of regularization on AC-GAN convergence behavior and stability.

The second part explores approaches for adapting Large Language Models (LLMs) to specialized domains. One study focused on fine-tuning BERT variants to identify suicidal ideation in social media posts. The fine-tuned models were interpreted using Layer Integrated Gradients to understand the linguistic cues used to detect suicidal ideation. Token attributions were leveraged as features for an efficient lightweight classifier. Another study adapted BERT to the aerospace domain through continual pre-training on an aerospace corpus (aeroBERTv2). Aerospace-specific pre-training yielded significant gains over general-purpose checkpoints, especially for smaller models, highlighting the value of tailoring representations to the target domain.

The third part applies 3D Gaussian Splatting to reconstruct satellites from 2D images under the computational constraints of spacecraft hardware. The approach achieves nearly 100x speedup over prior neural rendering methods, enabling real-time performance on-board spacecraft. The learned 3D representations are leveraged to generate synthetic satellite images for data augmentation, significantly improving the performance of object detection models for recognizing key spacecraft components.

Together, these studies advance representation learning and generative modeling techniques to derive insights and solve practical problems across healthcare, astronautics, and aviation. The work provides novel technical contributions while yielding broader insights into the behavior of generative models, the adaptation of LLMs to specialized domains, and the application of 3D reconstruction to space robotics.

Download

Share

COinS