Concepts

Short pages for terms that come up in several lectures. Each one has a definition, the key formula and an Appears in list that links to the detailed explanation in the lectures. The list grows with every lecture.

Foundations

• Inductive Bias
• Loss and Risk

Decision theory

• Bayes Classifier
• Bayesian Decision Theory
• Surrogate Loss

Learning from data

• Consistency
• Empirical Risk Minimization
• Estimation and Approximation Error
• Bias-Variance Decomposition
• No Free Lunch Theorem

Linear classifiers and optimization

• Perceptron
• Margin
• Stochastic Gradient Descent
• Leave-One-Out Error

Learning theory

• Hoeffding Inequality
• Uniform Convergence
• Generalization Bound
• Shattering Coefficient
• VC Dimension
• Rademacher Complexity

Stability and optimization

• Algorithmic Stability
• Strong Convexity
• Smoothness