After reviewing and updating the theory of energy-conserving Hamiltonian dynamics for optimization and sampling, I'll explain a new application of Energy Conserving Descent (ECD) optimization to precision scientific data analysis in which NN initialization variance has been a bottleneck. Specifically, we choose a particular ECD Hamiltonian whose measure on phase space concentrates the results in a controlled way, and describe a simple prescription for hyperparameter defaults. In a set of experiments on likelihood ratio estimation, using small simulated and real (Aleph) particle physics data sets, we find this reduces the error as predicted, performing better than Adam in this regard.
Time permitting, I'll discuss some separate ideas on machine learning theory, at an early stage of work in progress, including a novel architecture with a major reduction in parameter count based on the Abel-Jacobi map between a two-dimensional Riemann surface with many handles and its high-dimensional Jacobian torus.
Based in part on
Optimizers for Stabilizing Likelihood-free Inference – INSPIRE with G. B. De Luca, B. Nachman, and H. Zheng
Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for Optimization ,
Microcanonical Hamiltonian Monte Carlo JMLR 22-1450.pdf ,
[2306.00352] Improving Energy Conserving Descent for Machine Learning: Theory and Practice
Biography:
Professor Silverstein conducts research in theoretical physics -- particularly gravitation and cosmology, as well as recently developing new methods and applications for machine learning.
What are the basic degrees of freedom and interactions underlying gravitational and particle physics? What is the mechanism behind the initial seeds of structure in the universe, and how can we test it using cosmological observations? Is there a holographic framework for cosmology that applies throughout the history of the universe, accounting for the emergent effects of horizons and singularities? What new phenomena arise in quantum field theory in generic conditions such as finite density, temperature, or in time-dependent backgrounds?
Professor Silverstein attacks basic problems in several areas of theoretical physics. She develops concrete and testable mechanisms for cosmic inflation, accounting for its sensitivity to very high-energy physics. This has led to a fruitful interface with cosmic microwave background research, contributing to a more systematic analysis of its observable phenomenology.
Professor Silverstein also develops mechanisms for stabilizing the extra dimensions of string theory to model the accelerated expansion of the universe. In addition, Professor Silverstein develops methods to address questions of quantum gravity, such as singularity resolution and the physics of black holes and cosmological horizons.
Areas of focus:
- optimization algorithms derived from physical dynamics, analyzing its behavior and advantages theoretically and in numerical experiments
- UV complete mechanisms and systematics of cosmic inflation, including string-theoretic versions of large-field inflation (with gravity wave CMB signatures) and novel mechanisms involving inflaton interactions (with non-Gaussian signatures in the CMB)
-Systematic theory and analysis of primordial Non-Gaussianity, taking into account strongly non-linear effects in quantum field theory encoded in multi-point correlation functions
-Long-range interactions in string theory and implications for black hole physics
- Concrete holographic models of de Sitter expansion in string theory, aimed at upgrading the AdS/CFT correspondence to cosmology
- Mechanisms for non-Fermi liquid transport and $2k_F$ singularities from strongly coupled finite density quantum field theory
- Mechanisms by which the extra degrees of freedom in string theory induce transitions and duality symmetries between spaces of different topology and dimensionality (Source)
