Machine Learning in Montpellier, Theory & Practice

Overfitting on training data is typically assumed to be a bad practice. However, modern machine learning models are highly over-parameterised, to the extent that they can overfit on training data. In this talk, I will discuss new theories that debunk the myths that: (1) large models with too many parameters always overfit the training data; and (2) models that perfectly fit the training data cannot predict well on new data.I will then present our recent works on generalisation and learning dynamics of over-parameterised models, including (1) the double descent phenomenon in causal inference and (2) the neural tangent kernel approximation for semi- and self-supervised models. I will highlight how some of our results resolve conjectures in machine learning, and also provide new practical tools.,,