Lectures

You can download the lectures here. We will try to upload lectures prior to their corresponding classes.

• (dl-00) Introduction
  tl;dr:
  [slides]
  

  Suggested Readings:

  • Chapter 1 of the AI textbook by Russell and Norvig

  Additional References

  • Know your brain
  • McCulloch-Pitts Neuron, 1943
  • Hebbian Learning Princile by D. Hebb, 1949
  • Neocognitron by Fukushima, 1980 and an excellent Video Demo
  • Backprop for training NNs by Rumelhart et al. 1986
  • Backprop applied to MNIST by Lecun, 1989
  • AlexNet by Krizhevsky et al. 2012
  • ResNet by He et al. 2015
• (dl-01) Artificial Neuron Models (MP neuron and Perceptron)
  tl;dr: The Basic building block of ANNs.
  [perceptron-learning-code] [slides]
  

  Suggested Readings:

  • Chapter-1 of Michael Nielson’s NNDL textbook
  • Convergence proof for Perceptron
  • MP Neuron
• (dl-02) Network of perceptrons
  tl;dr: If not one, can multiple (network) of Perceptrons do the job?.
  [slides]
  

  Suggested Readings:

  • Chapter-1 of Michael Nielson’s NNDL textbook
  • Chapter-4 and 5 of Bishops’ book
• (dl-03) Representational Power of an MLP
  tl;dr: Multi Layered Network of Neurons can represent any artbirary function!
  [ReLU-activation-code] [sigmoid-activation-codes] [slides]
  

  Suggested Readings:

  • Chapter 4 of Michael Nielson’s NNDL book
  • Deep Mind Blog on Universal Approximation Theorem
  • Approximation by Superposition of Sigmoid Functions
  • Approximation Capabilities of Muitilayer Feedforward Networks