Understanding Practical Optimization in Machine Learning

Overview

Modern AI systems are built by training large and increasingly complex machine learning models, making optimization one of the central algorithmic challenges behind their success. Many practical optimization methods work well empirically before they are fully understood theoretically. Moreover, these methods often differ substantially from the idealized algorithms used in classical analyses: they operate on sketches or submodels of the model, modify gradients through clipping or normalization, use layer-wise non-Euclidean update directions, or rely on approximate oracle computations. This thesis studies these discrepancies through the lens of optimization. Its guiding principle is to analyze the objective, update, assumptions, and oracle induced by the implemented method, rather than by a simplified proxy.

The first part studies submodel and sparse training. We analyze Independent Subnetwork Training and show that computing gradients after sketching the model can shift the fixed point of the method, creating an irreducible error for the original objective. We then study Model-Agnostic Sparsified Training, a sketch-aware formulation that explains dropout-like and sparse methods as optimizing an objective that explicitly incorporates the sketching mechanism.

The second part focuses on private optimization. We analyze adaptive quantile clipping, a widely used technique motivated by the difficulty of tuning clipping thresholds in differentially private learning. The analysis shows that adaptive clipping reduces the tuning burden, but does not remove the bias introduced by clipping. We then propose smoothed normalization with error compensation and obtain convergence guarantees for distributed private optimization under standard assumptions.

The third part studies modern LMO-based optimizers and the assumptions needed to analyze them faithfully. We develop Gluon, a layer-wise framework for Muon/Scion-style methods that revisits standard smoothness assumptions through generalized, layer-wise models supported by trajectory measurements. We also analyze inexact Muon updates, showing how approximate orthogonalization affects convergence, step sizes, momentum, and hyperparameter sensitivity. Overall, the thesis argues that practical optimization methods in machine learning can be better understood, and ultimately improved, by explicitly modeling the structure that makes them practical.