Machine Learning in Montpellier, Theory & Practice
Remote sensing images, produced in large quantities, contain information that is already being used to monitor climate change, improve security and understand the environment. However, this data is difficult to interpret and often requires manual processing. As the amount of data increases, interpretation becomes a limiting factor not only in terms of time, but also in terms of the areas in which these images can be used. While the data is there, a wide audience cannot take advantage of it. In this presentation, we detail recent work on automatic response to visual questions. The aim of this work is to provide an answer in natural language to a question (also formulated in natural language) relating to remote sensing images, opening up the extraction of information from these images to as many people as possible. 10h30 – 11h, Konstantinos Panousis, INRIA Post-doc Title: Sparse Linear Concept Discovery Models Abstract: The recent mass adoption of DNNs, even in safety-critical scenarios, has shifted the focus of the research community towards the creation of inherently intrepretable models. Concept Bottleneck Models (CBMs) constitute a popular approach where hidden layers are tied to human understandable concepts allowing for investigation and correction of the network’s decisions. However, CBMs however usually suffer from: (i) performance degradation and (ii) lower interpretability than intended due to the sheer amount of concepts contributing to each decision. In this work, we propose a simple yet highly intuitive interpretable framework based on Contrastive Language Image models and a single sparse linear layer. In stark contrast to related approaches, the sparsity in our framework is achieved via principled Bayesian arguments by inferring concept presence via a data-driven Bernoulli distribution. As we experimentally show, our framework not only outperforms recent CBM approaches accuracy-wise, but it also yields high per example concept sparsity, facilitating the individual investigation of the emerging concepts. 11h – 11h15 PAUSE 11h15 – 11h45, Diego Marcos, INRIA CPJ Title: PDiscoNet: Semantically consistent part discovery for fine-grained recognition Abstract: Fine-grained classification often requires recognizing specific object parts, such as beak shape and wing patterns for birds. Encouraging a fine-grained classification model to first detect such parts and then using them to infer the class could help us gauge whether the model is indeed looking at the right details better than with interpretability methods that provide a single attribution map. We propose PDiscoNet to discover object parts by using only image-level class labels along with priors encouraging the parts to be: discriminative, compact, distinct from each other, equivariant to rigid transforms, and active in at least some of the images. In addition to using the appropriate losses to encode these priors, we propose to use part-dropout, where full part feature vectors are dropped at once to prevent a single part from dominating in the classification, and part feature vector modulation, which makes the information coming from each part distinct from the perspective of the classifier. Our results on CUB, CelebA, and PartImageNet show that the proposed method provides substantially better part discovery performance than previous methods while not requiring any additional hyper-parameter tuning and without penalizing the classification performance. 11h45 – 12h15 Ananthu Aniraj – INRIA PhD student Title: Masking Strategies for Background Bias Removal in Computer Vision Models Abstract: Models for fine-grained image classification tasks, where the difference between some classes can be extremely subtle and the number of samples per class tends to be low, are particularly prone to picking up background-related biases and demand robust methods to handle potential examples with out-of-distribution (OOD) backgrounds. To gain deeper insights into this critical problem, our research investigates the impact of background-induced bias on fine-grained image classification, evaluating standard backbone models such as Convolutional Neural Network (CNN) and Vision Transformers (ViT). We explore two masking strategies to mitigate background-induced bias: Early masking, which removes background information at the (input) image level, and late masking, which selectively masks high-level spatial features corresponding to the background. Extensive experiments assess the behaviour of CNN and ViT models under different masking strategies, with a focus on their generalization to OOD backgrounds. The obtained findings demonstrate that both proposed strategies enhance OOD performance compared to the baseline models, with early masking consistently exhibiting the best OOD performance. Notably, a ViT variant employing GAP-Pooled Patch token-based classification combined with early masking achieves the highest OOD robustness. Best, Cássio F. Dantas INRAE, TETIS [ https://isdm.umontpellier.fr/cassiofragadantas.github.io/ | cassiofragadantas.github.io ]
