Image and Video Generation: A deep Learning Approach

20th April 2021

 

Lecture by Prof. Nicu Sebe Image and Video Generation: A deep Learning Approach

Abstract

Video generation consists of generating a video sequence so that an object in a source image is animated according to some external information (a conditioning label or the motion of a driving video). In this talk I will present some of our recent achievements adressing these specific aspects: 1) generating facial expressions, e.g., smiles that are different from each other (e.g., spontaneous, tense, etc.) using diversity as the driving force. 2) generating videos without using any annotation or prior information about the specific object to animate.

Once trained on a set of videos depicting objects of the same category (e.g. faces, human bodies), our method can be applied to any object of this class. To achieve this, we decouple appearance and motion information using a self-supervised formulation. To support complex motions, we use a representation consisting of a set of learned keypoints along with their local affine transformations. A generator network models occlusions arising during target motions and combines the appearance extracted from the source image and the motion derived from the driving video. Our solutions score best on diverse benchmarks and on a variety of object categories.

Short CV

Nicu Sebe is a professor in the University of Trento, Italy, where he is leading the research in the areas of multimedia information retrieval and human-computer interaction in computer vision applications. He received his PhD from the University of Leiden, The Netherlands and has been involved in the past with the University of Amsterdam, The Netherlands and the University of Illinois at Urbana-Champaign, USA. He was involved in the organization of the major conferences and workshops addressing the computer vision and human-centered aspects of multimedia information retrieval, among which as a General Co-Chair of the IEEE Automatic Face and Gesture Recognition Conference, FG 2008, ACM International Conference on Multimedia Retrieval (ICMR) 2017 and ACM Multimedia 2013. He was a program chair of ACM Multimedia 2011 and 2007, ECCV 2016, ICCV 2017 and ICPR 2020. He is a general chair of ACM Multimedia 2022 and a program chair of ECCV 2024. Currently he is the ACM SIGMM vice chair, a fellow of IAPR and a Senior member of ACM and IEEE.

Presentation

More events

Cookie Settings

A AIDA - AI Doctoral Academy may use cookies to remember your login data, collect statistics to optimize the functionality of the site and to perform marketing actions based on your interests.


These cookies are necessary to allow the main functionality of the website and are automatically activated when you use this website.
These cookies allow us to analyze the use of the website, so that we can measure and improve its performance.
Allow you to stay in touch with your social network, share content, send and post comments.

Required Cookies They allow you to personalize the commercial offers that are presented to you, directing them to your interests. They can be own or third party cookies. We warn you that, even if you do not accept these cookies, you will receive commercial offers, but without meeting your preferences.

Functional Cookies They offer a more personalized and complete experience, allow you to save preferences, show you content relevant to your taste and send you the alerts you have requested.

Advertising Cookies Allow you to stay in touch with your social network, share content, send and post comments.