ETD

• Model Selection
• Neural Network
• Probabilistic Model
• Unbounded Depth
• Unbounded Width
• Variational Inference

Model selection is an essential component of machine learning, upon which depends a model's capacity to generalize, its computational efficiency, and its ultimate performance on a given task. However, the current landscape of Deep Learning research is dominated by an exponential scaling trajectory, often requiring models with billions of parameters. At this scale, exhaustive hyperparameter searches to determine the optimal architecture become computationally prohibitive, with practitioners often having to rely on heuristics or suboptimal configurations scaled up from smaller experiments.
Trying to address some of these limitations, this thesis introduces the \acl{model} (\acs{model}), a novel variational framework that decouples machine learning models from the rigid requirement of a pre-fixed topological structure. By formulating the network's internal topology as a set of latent variables, \acl{model} jointly adapts both its depth and layer width to the specific complexity of the task at hand with standard gradient-based optimization. This probabilistic approach significantly reduces the computational burden of manual model selection. Experimental results demonstrate that our adaptive architecture reliably matches or exceeds the performance of fixed-topology baselines on tabular benchmarks and qualitative analysis further inspect the behavior of the model under different scenarios, providing a robust foundation for future research on dynamic and integrated approaches to model selection.