Date of Award
12-2024
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Mathematics and Systems Engineering
First Advisor
Ryan T. White, Ph.D.
Second Advisor
V. Keshav Chivukula, Ph.D.
Third Advisor
Jewgeni H. Dshalalow, Dr. Sci.
Fourth Advisor
Xianqi Li, Ph.D.
Abstract
This dissertation addresses critical challenges in neural network design by leveraging entropy-based techniques to improve model efficiency, interpretability, and bias reduction. Focusing on the unique demands of computer vision applications, particularly object detection and classification for real-time systems, this work introduces a series of innovative methods centered on information theory. At the core of these methods is the Probabilistic Explanations of Entropic Knowledge (PEEK) framework, a tool developed to analyze and visualize entropy distributions across feature maps. PEEK offers insights into information flow within neural networks, making it possible to pinpoint layers that contribute meaningfully to decision-making or identify those that are redundant. By integrating our novel entropy-based loss functions into neural network training, this work enhances model convergence speed and accuracy. These losses enforce more efficient information flow, reducing unnecessary complexity in representations while preserving critical data. In real-world tasks such as image classification and object detection, the incorporation of entropy-guided loss terms led to models that achieve better performance with fewer parameters and fewer training epochs. Further, this research extends PEEK to enable efficient model pruning by identifying low PEEK variance layers that contribute minimally to overall performance. This variance-driven pruning optimizes network architecture, reducing GFLOPS and memory footprint to make models suitable for deployment on low-compute devices, such as Raspberry Pi and NVIDIA Jetson, without sacrificing accuracy. These entropy-based methods advance our capacity to build reliable, transparent, and resource-efficient AI systems, with significant implications for the deployment of autonomous systems in fields like aerospace and healthcare. In offering a structured framework for understanding, optimizing, and deploying neural networks, this dissertation contributes toward the development of scalable, interpretable, and fair AI applications.
Recommended Citation
Meni, Mackenzie J., "Decoding Neural Networks: An Information-Theoretic Guide to Interpretability, Error Analysis and Efficiency" (2024). Theses and Dissertations. 1515.
https://repository.fit.edu/etd/1515
Included in
Artificial Intelligence and Robotics Commons, Data Science Commons, Mathematics Commons, Other Statistics and Probability Commons, Theory and Algorithms Commons