introduction to deep learning

By attending Introduction to Deep Learning workshop, Participants will: 

  • Linear regression: mean squared error, analytical solution.
  • Logistic regression: model, cross-entropy loss, class probability estimation.
  • Gradient descent for linear models. Derivatives of MSE and cross-entropy loss functions.
  • The problem of overfitting.
  • Regularization for linear models.

The goal of this Introduction to Deep Learning training course is to give learners basic understanding of modern neural networks and their applications in computer vision and natural language understanding. The course starts with a recap of linear models and discussion of stochastic optimization methods that are crucial for training deep neural networks. Learners will study all popular building blocks of neural networks including fully connected layers, convolutional and recurrent layers. 

Learners will use these building blocks to define complex modern architectures in TensorFlow and Keras frameworks. In the course project learner will implement deep neural network for the task of image captioning which solves the problem of giving a text description for an input image.

  • Basic knowledge of Python.
  • Basic linear algebra and probability.

COURSE AGENDA

  • Course intro
  • Linear regression
  • Linear classification
  • Gradient descent
  • Overfitting problem and model validation
  • Model regularization
  • Stochastic gradient descent
  • Gradient descent extensions
  • Multilayer perceptron (MLP)
  • Chain rule
  • Backpropagation
  • Efficient MLP implementation
  • Other matrix derivatives
  • What is TensorFlow
  • Our first model in TensorFlow
  • What Deep Learning is and is not
  • Deep learning as a language
  • Motivation for convolutional layers
  • Our first CNN architecture
  • Training tips and tricks for deep CNNs
  • Overview of modern CNN architectures
  • Learning new tasks with pre-trained CNNs
  • A glimpse of other Computer Vision tasks
  • Unsupervised learning: what it is and why bother
  • Autoencoders 101
  • Autoencoder applications
  • Autoencoder applications: image generation, data visualization & more
  • Natural language processing primer
  • Word embeddings
  • Generative models 101
  • Generative Adversarial Networks
  • Applications of adversarial approach
  • Motivation for recurrent layers
  • Simple RNN and Backpropagation
  • The training of RNNs is not that easy
  • Dealing with vanishing and exploding gradients
  • Modern RNNs: LSTM and GRU
  • Practical use cases for RNNs