Attention and Transformers Networks

Archives: Resources

Resource description

Attention and Transformers Networks

In this lecture, the limitations of Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs) in effectively processing sequences are emphasized. However, a breakthrough solution known as Transformers is introduced, which addresses these limitations comprehensively. The architecture of Transformers is meticulously described, with a particular emphasis on its fundamental building blocks. These include positional encoding,… Continue reading Attention and Transformers Networks

Convolutional Neural Networks Lecture

Convolutional Neural Networks form the backbone of current AI revolution and are used in a multitude of classification and regression problems. This lecture overviews the transition from multilayer perceptrons to deep architectures. The following topics are resented in detail: Tensors and mathematical formulations. Convolutional layers. Fully connected layers. Pooling. Neural Image Features and their relation… Continue reading Convolutional Neural Networks Lecture

Multilayer perceptron. Backpropagation

This lecture covers the basic concepts and architectures of Multi-Layer Perceptron (MLP), Activation functions, and Universal Approximation Theorem. Training MLP neural networks is presented in detail: Loss types, Gradient descent, Error Backpropagation. Training problems are overviewed, together with solutions, e.g., Stochastic Gradient Descent, Adaptive Learning Rate Algorithms, Regularization, Evaluation, Generalization methods.

1D Convolutional Neural Networks

This lecture overviews 1D Convolutional Neural Networks that has many applications in 1D signal analysis. It covers the following topics in detail: 1D Convolution, 1D CNN Architecture, Convolutional Layer, Fully Connected Layer, Pooling Layers, Activation Functions, Supervised Learning, Classification/Regression, 1D CNN Training, 1D CNN applications (ECG monitoring, Music tagging).

Bayesian Learning

This lecture overviews Bayesian Learning that has many applications in pattern recognition and clustering. It covers the following topics in detail: Bayes probability theorem. Bayes decision rule. Bayesian classification. Maximum A-Posteriori Criterion. Maximum Likelihood Criterion. Normally Distributed Sample Classification. Bayesian clustering.