François Grolleau

This talk examines two critical aspects of data-driven medicine: personalized treatment strategies using causal inference and the robust evaluation of large language models (LLMs) for clinical applications.

In the first part, we present a novel approach to personalized medicine, applying causal statistical learning to observational data to develop individualized treatment rules. We focus on optimizing the timing of renal replacement therapy initiation in acute kidney injury, demonstrating: (i) the estimation and validation of an optimal dynamic strategy, and (ii) a comprehensive framework for evaluating individualized rules using observational data.

In the second part, we tackle the challenge of evaluating LLMs in medicine, focusing on the generation of hospital course summaries. Current evaluation methods are often either unscalable (physician-led) or untrustworthy for clinical settings (LLM-as-a-judge). We propose a rubric-based approach to LLM evaluation that combines the scalability of automated methods with the trustworthiness demanded by medical applications, paving the way for responsible deployment of LLMs in healthcare.

Data-driven medicine, LLMs, clinical applications, LabéliséHallesIA

Inria Montpellier, St-Priest Campus, Building 5, Room 01/124
Machine Learning in Montpellier, Theory & Practice

Data.table is one of the most efficient open-source in-memory data manipulation packages available today. First released to CRAN by Matt Dowle in 2006, it continues to grow in popularity, and now over 1500 other CRAN packages depend on data.table. This talk will start with data reading from CSV, discuss basic and advanced data manipulation topics, and finally will end with a discussion about how you can contribute to data.table.
https://github.com/tdhock/2023-10-LatinR-data.table?tab=readme-ov-file#source-files-for-latinr-datatable-tutorial-slides

LabéliséHallesIA

Inria Montpellier, St-Priest Campus, Building 5, Room 02/022
Machine Learning in Montpellier, Theory & Practice

Machine learning classifiers often produce probabilistic predictions that are critical for accurate and interpretable decision-making in various domains. The quality of these predictions is generally evaluated with proper losses like cross-entropy, which decompose into two components: calibration error assesses general under/overconfidence, while refinement error measures the ability to distinguish different classes.
In this paper, we provide theoretical and empirical evidence that these two errors are not minimized simultaneously during training. Selecting the best training epoch based on validation loss thus leads to a compromise point that is suboptimal for both calibration error and, mostimportantly, refinement error. To address this, we introduce a new metric for early stopping and hyperparameter tuning that makes it possible to minimize refinement error during training. The calibration error is minimized after training, using standard techniques. Our method integrates seamlessly with any architecture and consistently improves performance across diverse classification tasks.

LabéliséHallesIA

Campus St Priest
Rebecca Farina

Our objective is to construct well-calibrated prediction sets for a time-to-event outcome subject to right-censoring with guaranteed coverage. Our approach is inspired by modern conformal inference literature, in that, unlike classical frameworks, we obviate the need for a well-specified parametric or semi-parametric survival model to accomplish our goal. In contrast to existing conformal prediction methods for survival data, which restrict censoring to be of Type I, whereby potential censoring times are assumed to be fully observed on all units in both training and validation samples, we consider the more common right-censoring setting in which either only the censoring time or only the event time of primary interest is directly observed, whichever comes first. Under a standard conditional independence assumption between the potential survival and censoring times given covariates, we propose and analyze two methods to construct valid and efficient lower predictive bounds for the survival time of a future observation. The proposed methods build upon modern semiparametric efficiency theory for censored data, in that the first approach incorporates inverse-probability-of-censoring weighting (IPCW), while the second approach is based on augmented-inverse-probability-of-censoring weighting (AIPCW). For both methods, we formally establish asymptotic coverage guarantees, and demonstrate both via theory and empirical experiments that AIPCW substantially improves efficiency over IPCW in the sense that its coverage error bound is of second-order mixed bias type, that is doubly robust, and therefore guaranteed to be asymptotically negligible relative to the coverage error of IPCW.

LabéliséHallesIA

Campus St Priest
Machine Learning in Montpellier, Theory & Practice
Jim Smit

Despite several randomised controlled trials (RCTs) on the use of adjuvant treatment with corticosteroids in patients with community-acquired pneumonia (CAP), the effect of this intervention on mortality remains controversial. We aimed to evaluate heterogeneity of treatment effect (HTE) of adjuvant treatment with corticosteroids on 30-day mortality in patients with CAP.

LabéliséHallesIA