Mathematical Physics In-Person Seminar
Upcoming Events
- Sep 24 2026
Anirvan Sengupta - In-context Denoising, Attention and Diffusion
- Information
- Thursday, September 24, 2026 - Thursday, September 24, 2026
- 12:10 PM - 1:10 PM
- Hill 705
Anirvan Sengupta – Rutgers University
Date/Time/Location
Thursday, September 24, 2026, 12:10 pm; Hill Center 705
In-context Denoising, Attention and DiffusionModern neural networks can have enough parameters to memorize their training data, yet often generalize remarkably well. I will discuss this tension from a statistical-physics perspective, focusing on attention mechanisms and their relation to associative memory. I will first consider an in-context denoising problem for a one-layer transformer. For several simple data models, the Bayes-optimal denoiser can be derived explicitly and represented by linear or softmax attention, providing a direct connection to modern Hopfield models. In this picture, generalization is associated with a crossover of the Hopfield energy landscape from isolated stored patterns to a representation of the underlying data distribution. I will then discuss multilayer attention-only dynamics, where all samples evolve together. In an appropriate continuum limit, the resulting self-attractive dynamics can be related to anti-diffusion, connecting attention to score-based denoising and empirical Bayes. This suggests a unified view of associative memory, attention, and diffusion-based generative modeling.
- Oct 01 2026
Kasper Larsen - Strict SDE Comparison for Cusp Coefficients and Counterexamples
- Information
- Thursday, October 1, 2026 - Thursday, October 1, 2026
- 12:10 PM - 1:10 PM
Kasper Larsen – Rutgers University
Date/Time/Location
Thursday, October 1, 2026, 12:10 pm; Hill Center 705
Strict SDE Comparison for Cusp Coefficients and CounterexamplesWe provide a tractable sufficient condition for strict comparison for solutions of the one-dimensional stochastic differential equation dX = sigma(X)dB, with initial condition x in an interval I, for sigma positive and continuous. Our proof is based on a two-dimensional Lyapunov argument, which allows us to prove strict comparison for some coefficients in the local Sobolev space W1p, for p between 1 and 2. We illustrate using sigma of x equal to 1 plus the absolute value of x to the power beta, for beta between 0 and 1, and show that strict comparison holds if and only if beta is at least one half. We give examples showing that neither local W1p regularity for p between 1 and 2 nor Hölder regularity of order beta between one half and 1 is sufficient for strict comparison, even when combined with boundedness, uniform ellipticity, global strong existence, and pathwise uniqueness.