Room 02.022, Building 5, St Priest campus Machine Learning in Montpellier, Theory & Practice Personal data is being collected at an unprecedented scale by businesses and public organizations, driven by the progress of data science and machine learning. While such data can be turned into useful knowledge about the global population by computing aggregate statistics or training machine learning models, this can also lead to undesirable disclosure of personal information. We must therefore deal with two conflicting objectives: maximizing the utility of data while protecting the privacy of individuals whose data is used in the analysis. In this talk, I will present differential privacy (DP), a statistical definition of privacy which comes with rigorous guarantees as well as an algorithmic framework that allows the design of practical privacy-preserving algorithms. I will then discuss the application of DP to machine learning, and some related open questions. Machine Learning in Montpellier, Theory & Practice

Room 109, IMAG, Triolet campus Machine Learning in Montpellier, Theory & Practice Machine learning's success relies on the use of datasets that typically hold sensitive information about people. To prevent leakage of personal data, differentially private optimization methods have recently been developed. While these methods offer strong guarantees on data privacy, they suffer from degraded performance. This incurs a trade-off between data privacy and model utility. In this talk, I will describe the difficulties faced when training a model with differential privacy guarantees, in particular for high-dimensional models. I will then show how coordinate descent methods can exploit structural properties of the problem to improve the privacy-utility trade-off. Machine Learning in Montpellier, Theory & Practice

Room 02.022, Building 5, St Priest campus Machine Learning in Montpellier, Theory & Practice Sampling from a target measure when only partial information is available (e.g. unnormalized density as in Bayesian inference, or true samples as in generative modeling ) is a fundamental problem in computational statistics and machine learning. The sampling problem can be formulated as an optimization over the space of probability distributions of a well-chosen discrepancy (e.g. a divergence or distance). In this talk, we'll discuss several properties of sampling algorithms for some choices of discrepancies (well-known ones, or novel proxies), both regarding their optimization and quantization aspects.,, Machine Learning in Montpellier, Theory & Practice

Room 02.249, Building 5, St Priest campus Machine Learning in Montpellier, Theory & Practice Machine Learning in Montpellier, Theory & Practice

Room 01.124, Building 5, St Priest campus Machine Learning in Montpellier, Theory & Practice In this talk, I present the objective of conformal prediction and overview some properties of the well-studied split estimator. Then, I introduce a quantile-of-quantiles estimator that allows constructing prediction sets in a one-shot federated learning setting. We investigate the properties of this estimator and how it can be used to build confidence sets with probabilistic coverage being marginally or training-conditionally valid. Over a set of experiments, we empirically verify the quality of our results and show that it is possible to output prediction sets with desired coverage, in only one round of communication, while recovering performances close to the centralized split method. Machine Learning in Montpellier, Theory & Practice

Room 03.124, Building 5, St Priest campus Machine Learning in Montpellier, Theory & Practice This talk is about Lipschitz constraints in the context of deep learning. Lipschitz constrained neural networks are of huge interest in the context of optimal transport (through Kantorovich-Rubinstein duality) and certifiability against adversarial attacks. First, I will explain how to implement these constraints in practice by performing a brief literature reviews of the dominant aproaches and methods. I will also present the DEEL-LIP library that contains some of these implementations in a Python package. Then, I will present some of my contributions to this field. I will show that these class of functions can be used to solve any classification task, with robustness guarantees, generalization guarantees (some of which being architecture-independant), and I will emphasize the importance of entropic regularization to chose an accuracy/robustness tradeoff on a Pareto front. After, I will present the explainability properties of these networks when they are trained with the Hinge-Kantorovich-Rubinstein (HKR) loss, inspired by optimal transport. Their saliency maps are more aligned with humans perception than any other network, including on challenging datasets like ImageNet. The HKR loss and Lipschitz neural networks can also be used to compute Signed Distance Function (SDF) in an unsupervised manner, relieving the need for a ground truth based on nearest neighbor. The resulting algorithm can be seen as a parametric K-nearest neighbor, or a robust one-class classifier. Furthermore, the SDF is also of huge interest in the computer graphics community since it allow for implicit surface parametrization with formal guarantees during raytracing. Finally, I will show how Lipschitz constraints w.r.t the input can be converted easily into Lipschitz constraints w.r.t the parameters, using a "backpropagation for bounds" algorithm, which opens path for deep learning with privacy guarantees without the need for the expensive gradient clipping operation of DP-SGD. Machine Learning in Montpellier, Theory & Practice

Room 109, IMAG, Triolet campus Machine Learning in Montpellier, Theory & Practice We consider a gradient-based optimization method applied to a function [image: \mathcal{L}] of a vector of variables [image: {\theta}], in the case where [image: {\theta}] is represented as a tuple of tensors [image: (\mathbf{T}_1, \cdots, \mathbf{T}_S)]. This framework encompasses many common use-cases, such as training neural networks by gradient descent. First, we propose a computationally inexpensive technique providing higher-order information on [image: \mathcal{L}], especially about the interactions between the tensors [image: \mathbf{T}_s], based on automatic differentiation and computational tricks. Second, we use this technique at order 2 to build a second-order optimization method which is suitable, among other things, for training deep neural networks of various architectures. This second-order method leverages the partition structure of [image: \theta] into tensors [image: (\mathbf{T}_1, \cdots, \mathbf{T}_S)], in such a way that it requires neither the computation of the Hessian of [image: \mathcal{L}] according to [image: \theta], nor any approximation of it. The key part consists in computing a smaller matrix interpretable as a ``Hessian according to the partition’’, which can be computed exactly and efficiently. In contrast to many existing practical second-order methods used in neural networks, which performs a diagonal or block-diagonal approximation of the Hessian or its inverse, the method we propose does not neglect interactions between layers. GitHub: \url{https://isdm.umontpellier.fr/github.com/p-wol/GroupedNewton%7D. arXiv https://isdm.umontpellier.fr/arxiv.org/abs/2312.03885 Machine Learning in Montpellier, Theory & Practice