Fundamentals of Machine Learning

Summary

This course aims to provide a comprehensive introduction to the fundamentals of machine learning, combining both theoretical insights and practical applications. The curriculum covers essential topics such as supervised (regression, classification, decision trees, SVMs, ensemble methods like random forests and gradient boosting, and neural networks for supervised tasks) and unsupervised learning (k-means, hierarchical clustering, dimensionality reduction methods such as PCA and t-SNE), model evaluation, optimization techniques, and common machine learning frameworks. Particular emphasis will be placed on the practical implementation of the various algorithms seen during the course (using Python).

Organization

  • Lecturers: Titouan Vayer (Inria, ENS Lyon, LIP), Mathurin Massias (Inria, ENS Lyon, LIP), Elisa Ricietti (ENS Lyon, LIP).
  • Duration: total 20 * 2 hours = 10 * 2 hours of lectures + 10 * 2 hours of practical sessions with exercises + Python sessions.

Outline

  1. Introduction (slides)
    • An introduction to machine learning (data, supervised and unsupervised learning, classification, regression)
    • Model selection and validation
    • A glimpse of statistical learning theory
  2. Linear regression (slides)
    • What is linear regression ?
    • Variables selection
    • Ridge/Lasso
  3. Dimension reduction (slides)
    • Curse of dimensionality
    • Principal components analysis
    • Non-linear dimension reduction (t-SNE)
  4. Clustering (slides)
    • K-means algorithm and variations (K-medoids)
    • Spectral clustering
    • Hierarchical clustering
  5. Linear classification (slides)
    • Linear regression
    • LDA/QDA
    • SVM and kernels (slides)
  6. Ensembles methods (slides)
    • Trees and forest
    • Boosting
  7. Neural networks and deep learning (slides 1) (slides 2)
    • Introduction to neural networks
    • Automatic differentiation
    • CNN, GANs
  8. Density estimation and introduction to generative modeling (slides)
    • Gaussian mixture model
    • Introduction to optimal transport, divergence minimization