The lecture notes for this course were prepared by Alexander Rakhlin and Wen Dong, students in the class.
Complete set of lecture notes in one file (PDF - 4.8 MB)
| LEC # | TOPICS |
|---|---|
| 1 | Introduction |
| 2 | Voting classifiers, training error of boosting (PDF) |
| 3 | Support vector machines (SVM) (PDF) |
| 4 | Generalization error of SVM (PDF) |
| 5 | One dimensional concentration inequalities. Bennett's inequality (PDF) |
| 6 | Bernstein's inequality (PDF) |
| 7 | Hoeffding, Hoeffding-Chernoff, and Khinchine inequality (PDF) |
| 8 | Vapnik-Chervonenkis classes of sets (PDF) |
| 9 | Properties of VC classes of sets (PDF) |
| 10 | Symmetrization. Pessimistic VC inequality (PDF) |
| 11 | Optimistic VC inequality (PDF) |
| 12 | VC subgraph classes of functions. Packing and covering numbers (PDF) |
| 13 | Covering numbers of the VC subgraph classes (PDF) |
| 14 | Kolmogorov's chaining method. Dudley's entropy integral (PDF) |
| 15 | More symmetrization. Generalized VC inequality (PDF) |
| 16 | Consequences of the generalized VC inequality (PDF) |
| 17 | Covering numbers of the convex hull (PDF) |
| 18 | Uniform entropy condition of VC-hull classes (PDF) |
| 19 | Generalization error bound for VC-hull classes (PDF) |
| 20 | Bounds on the generalization error of voting classifiers (PDF) |
| 21 | Bounds on the generalization error of voting classifiers (cont.) (PDF) |
| 22 | Bounds on the generalization error of voting classifiers (cont.) (PDF) |
| 23 | Bounds in terms of sparsity (PDF) |
| 24 | Bounds in terms of sparsity (cont.) (example) (PDF) |
| 25 | Martingale-difference inequalities (PDF) |
| 26 | Comparison inequality for Rademacher processes (PDF) |
| 27 | Application of martingale inequalities. Generalized martingale inequalities (PDF) |
| 28 | Generalization bounds for neural networks (PDF) |
| 29 | Generalization bounds for neural networks (cont.) (PDF) |
| 30 | Generalization bounds for kernel methods (PDF) |
| 31 | Optimistic VC inequality for random classes of sets (PDF) |
| 32 | Applications of random VC inequality to voting algorithms and SVM (PDF) |
| 33 | Talagrand's convex-hull distance inequality (PDF) |
| 34 | Consequences of Talagrand's convex-hull distance inequality (PDF) |
| 35 | Talagrand's concentration inequality for empirical processes (PDF) |
| 36 | Talagrand's two-point inequality (PDF) |
| 37 | Talagrand's concentration inequality for empirical processes (PDF) |
| 38 | Applications of Talagrand's concentration inequality (PDF) |
| 39 | Applications of talagrand's convex-hull distance inequality. Bin packing (PDF) |
| 40 | Entropy tensorization inequality. Tensorization of Laplace transform (PDF) |
| 41 | Application of the entropy tensorization technique (PDF) |
| 42 | Stein's method for concentration inequalities (PDF) |