Mathematics for Artificial Intelligence 2
These lectures are offered in the M1 tracks of the Master Mathematics
and Applications of Paris Saclay
Part I: January 4 - February 12
Part II: March 1 - April 9
French or English, depending on the audience.
Probability theory, linear algebra, convex analysis
Please download the lecture notes.
This course MathIA-2 can be followed independently of the course MathIA-1 from the
The lectures are intended as an introduction to the mathematical tools
involved in the analysis of Machine Learning algorithms.
The course has two parts. A first part is focused on the problem of
sequential learning. The mathematical tools involved are mainly probabilistic, with
some convexity arguments. The second part covers some questions
where matrix analysis and linear algebra play a central role.
Part 1: Sequential learning
Part 2: Matrix analysis for Machine Learning
- Complement on probability: Sub-Gaussian random vector,
sub-Exponential random variables, concentration inequalities.
- Sequential learning problems, Stochastic Gradient Optimisation,
connection with classical optimisation problems.
- Learning with expert advices, adversarial environnement, mirror
- Learning in an unknown environnement. Exploration/exploitation
trade-off, Information-Theoretic lower bounds.
- Matricial problems in ML: dimension reduction, clustering,
- Singular value decomposition, matrix norms, perturbation bounds
- Concentration of the operator norm of random matrices,
application in ML.
Mid term exam February 15, room 0A2, 14h--17h,
Final exam May 21, room 1A7, 14h--17h.
Exercises for the exam: Sections 1.3.1, 2.3.1, 2.3.2, 3.4.3 [including Pinsker inequality for arbitrary probability distributions], 4.4., 6.4.2, 7.4.1, 7.4.2, 8.4.1.
Discovering more on the topic
For a deeper investigation of the first part, I recommend the M2 course
'Apprentissage et optimisation sequentiels'. For the second part, I
recommend the M2 course 'High-dimensional probability and
statistics'. Both courses are given in the master program
'mathématiques de l'aléatoire'.
For exploring more widely on mathematics and machine learning, you may have a look at the nice blogs of Sebastien Bubeck and
The Institut de Mathématiques d'Orsay is located in Building
307, Orsay campus, 5min walk from RER station Orsay-Ville.