Classification and statistical machine learning

Classification problems arise in various domains such as bioinformatics, computer vision or text analysis. We will consider the binary supervised classification problem, where the goal is to learn how to assign a label (0 or 1) to any observation X from a sample X_1, ..., X_n of labelled observations. For instance, X can be the output of a DNA microarray experiment, an image, a video sequence or an email. After formalizing the classification problem, we will identify what are the main challenges of classification, from the limits of universal consistency (there is no free lunch!) to the trade-off between overfitting and underfitting. Then, we will overview several classical approaches to classification that correspond to very different ways of modelling the data (empirical risk minimization and model selection, support vector machines, random forests), and some of the main  mathematical results available for their analysis. Finally, we will discuss some key practical issues such as the computational complexity (in particular in the big data framework) and hyperparameters tuning.