Localized Dudley Entropy Integral
Published:
Chaining across all resolution scales at once: process complexity is the integral of root-entropy, localized to a ball.
Posts by Tags
Chaining
Concentration
McDiarmid’s Bounded Differences Inequality
Published:
Any function of independent inputs that no single input can move very far concentrates like a Gaussian.
Sub-Gaussian Maximal Inequality
Published:
The expected max of n sub-Gaussian variables costs only √(2 log n) — no independence required.
Bernstein’s Inequality
Published:
Variance-aware concentration for bounded random variables: the Gaussian rate until the range term takes over.
Dimension Reduction
PCA through the lens of Linear Algebra
Published:
Principal Component Analysis from a linear algebraic perspecive.
Empirical Processes
Talagrand’s Contraction Lemma
Published:
Composing with a κ-Lipschitz function costs at most a factor κ in Rademacher complexity.
Symmetrization & the Rademacher Generalization Bound
Published:
The ghost sample trick: uniform deviations are controlled by the class’s ability to correlate with random signs.
Localized Dudley Entropy Integral
Published:
Chaining across all resolution scales at once: process complexity is the integral of root-entropy, localized to a ball.
Learning Theory
Rademacher Complexity Bounds for Concrete Models and Losses
Published:
Annotated study notes on Ma Chapter 5: margin theory, Rademacher complexity of linear models and two-layer nets, and the covering-number route to deep nets.
Learning as Optimization: the Excess Risk Decomposition
Published:
Why ‘learning is optimization’: no free lunch, the three-term excess risk decomposition, and the coupling that makes deep learning theory interesting.
Linear Algebra
PCA through the lens of Linear Algebra
Published:
Principal Component Analysis from a linear algebraic perspecive.
Optimization
Learning as Optimization: the Excess Risk Decomposition
Published:
Why ‘learning is optimization’: no free lunch, the three-term excess risk decomposition, and the coupling that makes deep learning theory interesting.
Rademacher Complexity
Rademacher Complexity Bounds for Concrete Models and Losses
Published:
Annotated study notes on Ma Chapter 5: margin theory, Rademacher complexity of linear models and two-layer nets, and the covering-number route to deep nets.
STAT 210B
Talagrand’s Contraction Lemma
Published:
Composing with a κ-Lipschitz function costs at most a factor κ in Rademacher complexity.
Symmetrization & the Rademacher Generalization Bound
Published:
The ghost sample trick: uniform deviations are controlled by the class’s ability to correlate with random signs.
McDiarmid’s Bounded Differences Inequality
Published:
Any function of independent inputs that no single input can move very far concentrates like a Gaussian.
Sub-Gaussian Maximal Inequality
Published:
The expected max of n sub-Gaussian variables costs only √(2 log n) — no independence required.
Localized Dudley Entropy Integral
Published:
Chaining across all resolution scales at once: process complexity is the integral of root-entropy, localized to a ball.
Bernstein’s Inequality
Published:
Variance-aware concentration for bounded random variables: the Gaussian rate until the range term takes over.
STAT 241A
Rademacher Complexity Bounds for Concrete Models and Losses
Published:
Annotated study notes on Ma Chapter 5: margin theory, Rademacher complexity of linear models and two-layer nets, and the covering-number route to deep nets.
Learning as Optimization: the Excess Risk Decomposition
Published:
Why ‘learning is optimization’: no free lunch, the three-term excess risk decomposition, and the coupling that makes deep learning theory interesting.