Machine Learning in Montpellier, Theory & Practice
In the first part of the talk, I will present some reflections on the purpose of prediction sets and the role that statistics can play in forming useful prediction sets. In particular, I will discuss how prediction sets fit into a decision making pipeline and the different kinds of decisions one may make using a prediction set. In the second part of the talk, I will describe a particular statistically motivated set-generating procedure for the classification setting called clustered conformal prediction, which gives all classes an equal chance of being correctly included in the prediction set (“class-conditional coverage”). This procedure can be useful in situations where it is important to identify instances of all classes, even the rare ones. We demonstrate the performance of this method on ImageNet and other image classification datasets.
