Uncertainty Estimation for Trustworthy AI



Short Course on Uncertainty Estimation for Trustworthy AI


Mara Graziani, mara.graziani@hevs.ch

Andrey Malinin, andrey@malinin.io

Nataliia Molchanova, nataliia.molchanova@hevs.ch

Vatsal Raina, vr311@cam.ac.uk

Merixtell Bach Cuadra, merixtell.bachcuadra@unil.ch

Content and organization

Unreliable and weak AI systems are a threat to the safety and wellbeing of the society. As computer scientists working on complex systems integrating deep learning models, such as autonomous driving systems or clinical tools for assisted diagnosis, we must ensure the reliability and robustness of the models. The hardest challenges for reliability arise on out-of-distribution samples, on which a neural network has little experience. Consider inputting an input image of a cat to a model that was only trained on brain MRI images. What should the model do when presented an input coming from such a strongly shifted distribution?

In this short course we will cover the basic topics to get started with evaluating model uncertainty and robustness, a step that should be taken into account in all developments for high-stakes and that would also benefit most low-risk applications.

By participating to the course, you will deepen your technical understanding in the following areas:

(1) Recognising and characterising distributional shift

(2) Estimating uncertainty under domain shift and on in-domain data

(3) Differentiating between aleatoric and epistemic uncertainty

(4) Applicative benefits of implementing uncertainty and confidence score assessment in model development

(5) Hands-on confidence on the real-world use-case scenarios of lesion segmentation in brain imaging and power consumption estimation for vessel cargos


  1. Motivation
  2. What is distributional shift?
  3. Approaches for uncertainty estimation
  4. Real-world application examples and hands-on tutorial


Beginner to Intermediate

Course Duration

3 hours

Course Type

Short Course


English. Experience with Python and PyTorch is recommended.

Modality (online/in person):


Host Institution
HES-SO Valais

Other short courses

10. 04. 2024 Go

Ethics & STICs

01. 03. 2024 Go

Computer Vision

24. 11. 2023 Go

Human Rights Toolbox

21. 02. 2023 Go

Computer Vision

12. 01. 2023 Go