Uncertainty Estimation Continuing my interest in Active Learning, it is impossible to pass by the topic of Uncertainty Estimation of the models. Actually, this is a quite interesting field of ML and super important for applications. For example, in self-driving cars the model should be sure if there is or not a pedestrian on the road. If it is not sure enough if the road is free, it should pass the control to the person. Another very good example: imagine you train a model to classify images of dogs and cats. But suddenly, the model will get an input the image of a bird. It will return you by design the label if it is either cat of dog, but is it sure about its decision? It is important to indicate how your model deals with out-of-domain (OOD) samples, with shifts or new types of data in the test set. Also, uncertainty can appear from the data itself — if it is inconsistencies, if during measurement there were some noises, if it was unfortunate split into train/test/dev and there are shifts in the batches. My list of sources to learn more about Uncertainty Estimation: 1. Youtube lecture from MIT Deep Learning course "Evidential Deep Learning and Uncertainty" [link]. Very good for the general understanding of the problem. 2. A paper (like a book) "A Survey of Uncertainty in Deep Neural Networks" [link] — a comprehensive overview of types of uncertainties in the models, how to measure and how to deal with them. 3. Another book-typed paper "A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges" [link] — also, very good overview now more focused on the techniques how to measure these uncertainties for different model types and tasks.