This lecture will provide an short overview of geometric deep learning & manifold learning, a subfield of machine learning that addresses challenges in analyzing non-Euclidean data such as manifolds or graphs. Traditional deep learning methods work well for data in Euclidean spaces, but many practical problems involve non-Euclidean spaces. For example, 360 degree cameras capture spherical images and covariance matrices and rotation matrices appear in fields like 3D reconstruction, robotics and image processing. The lecture first gives an introduction into the key concepts of Riemannian manifolds like tangent space, geodesics and exponential/logarithm map. We will also show how neural network components like convolutions can be extended to manifolds. Furthermore, common applications of geometric deep learning for multimedia including image classification, 3D data analysis, and video processing will be presented. Practical examples will show how geometric deep learning can be employed advantageously, such as using knowledge distillation to adapt CNN models to spherical images from 360 degree cameras, image editing by manipulation of the latent space manifolds and how to merge multiple finetuned models in an optimal way via manifold mixing model soups.The session concludes with a brief overview of open source libraries for geometric deep learning like Geomstats and geoopt.
Hannes Fassold received a MSc degree in Applied Mathematics from Graz University of Technology in 2004. Since then he works at JOANNEUM RESEARCH, where he is currently a senior researcher at the Intelligent Vision Applications Group of the DIGITAL institute. His main research interests are how to employ machine vision and AI methods successfully to solve real-world problems like image and video enhancement, defect inspection, object detection and tracking and so on. He is presenting regularly in renowned computer vision, multimedia & AI conferences like ACM Multimedia, ICIP, ICME, AIVR, MVA, MMSP etc. He coordinates the machine learning workflow as well as the dedicated ML hardware & software infrastructure for the DIGITAL institute.