The reject option is an efficient technique to abstain from prediction when the uncertainty about the label is too high. This may happen, for instance, when classes overlap, images contain several objects, or the input is degraded. In this talk, I will first present learning with reject option in its classical use, and then focus on the case where the rejection rate is bounded, which turns this uncertainty into a controlled quantity. The optimal rule is then characterized and estimated by a plug-in procedure in a semi-supervised setting, where unlabeled data are enough to calibrate the rejection threshold; I will discuss distribution-free guarantees on the rejection rate as well as rates of convergence on the excess-risk highlighting when unlabeled data help.
In the second part, I will argue that the rejected set is not a failure but a signal on where to spend a scarce resource. I will illustrate this with two applications of a different nature: active learning, where rejection identifies the region in which labels should be queried, and early exiting in deep neural networks, where rejection decides which inputs deserve more computation under a fixed computational (and energy) budget. I will conclude with a brief overview of recent uses of abstention in other settings.