Lecture notes for "Statistical machine learning" (University of
Chicago Stats 37700, Winter 2007)
- Lecture 1: introduction, decision
trees. References: [1] (chap. 2), [2], [3] (8.1-3), [4] (chap. 9)
- Lecture 2: introduction to statistical
learning theory. Reference: [2]
- Lecture 3: statistical analysis of nearest
neighbors rules. References: [1] (chap. 5-6), [3] (4.4-6)
- Lecture 4: classical linear and quadratic
discrimination rules. References: [1] (chap. 4), [3] (2.6, 3.8,
chap.5), [4] (chap. 4), [5] (p.12-15)
- Lecture 5: statistical learning theory II:
Vapnik-Chervonenkis theory Reference: [2]
- Lecture 6: support vector machines
(linear) Reference: [5]
- Lecture 7: kernels Reference: [6]
- Lecture 8: ensemble methods References:
[4] (chap. 10), [7] (p. 118-183)
- Lecture 9: statistical learning theory
III: Rademacher complexities References: [2], [6] (chap. 4), [a]
for the simple proof of the Rademacher contraction property
- Lecture 10: randomized classifiers (and
multiple testing).
- Lecture 11: Bayesian methods, Gaussian
processes References: [8] (p. 41-71), [b]
General references:
- [1] A Probabilistic Theory of Pattern Recognition, by
L. Devroye, L. Györfy and G. Lugosi, Springer.
- [2] Pattern classification and learning theory, by G. Lugosi,
in Principles
of non Parametric Learning, L. Györfy editor, Springer.
- [3] Pattern classification, by R. Duda, P. Hart and D. Stork,
Wiley.
- [4] The Elements of Statistical Learning, by T. Hastie,
R. Tibshirani, and J. Friedman, Springer.
- [5] An introduction to Support Vector Machines, by
N. Cristianini and J. Shawe-Taylor, Springer.
- [6] Kernel methods for pattern analysis, by
N. Cristianini and J. Shawe-Taylor, Cambridge university press.
- [7] Advanced Lectures on Machine Learning, S. Mendelson,
A. Smola editors, Springer LNAI 2600.
- [8] Advanced Lectures on Machine Learning, O. Bousquet, U. von
Luxburg, G. Rätsch editors, Springer LNAI 3176.
Some specific references: