Matthew Senese
University of Notre Dame
3:30 PM
154 Hurley Hall
Memorization in Score-Based Generative Modeling and Neural Net Regularization
Generative diffusion models have grown significantly in popularity as a reliable and effective way to generate data from an unknown distribution. Score-based Generative Modeling (SBGM) is a specific kind of generative modeling process that manipulates data through stochastic differential equations (SDEs). SBGM is done by progressively adding noise to a piece of data in a forward-time SDE, and then using the gradient of the log of the density function (∇ log p), referred to as the “score”, in a reverse-time SDE to progressively remove noise and gen- erate new data. This score is typically approximated, where the accuracy of the estimated score (s) to the true score is measured by a loss function of the form: L = ∥s − ∇ log p∥2. Despite being an estimation, we can determine a unique minimizer of this loss, allowing the model to “memorize” the training data sam- ples rather than create novel data. Neural networks have the remarkable ability to implicitly regularize a learned score function compared to other explicit regularization techniques. We will investigate how different types of neural networks and explicit algorithms can be implemented into SBGM, as well as how these models memorize or regularize the training data they learn.
View Poster