Schedule

Suggested Readings:

• Chapter 1 of the AI textbook by Russell and Norvig

Additional References

• Know your brain
• McCulloch-Pitts Neuron, 1943
• Hebbian Learning Princile by D. Hebb, 1949
• Neocognitron by Fukushima, 1980 and an excellent Video Demo
• Backprop for training NNs by Rumelhart et al. 1986
• Backprop applied to MNIST by Lecun, 1989
• AlexNet by Krizhevsky et al. 2012
• ResNet by He et al. 2015

Suggested Readings:

• Chapter-1 of Michael Nielson’s NNDL textbook
• Convergence proof for Perceptron
• MP Neuron

Suggested Readings:

• Chapter-1 of Michael Nielson’s NNDL textbook

Suggested Readings:

• Chapter 4 of Michael Nielson’s NNDL book
• Deep Mind Blog on Universal Approximation Theorem
• Approximation by Superposition of Sigmoid Functions
• Approximation Capabilities of Muitilayer Feedforward Networks

Suggested Readings:

• ML chapter from Goodfellow et al. DL Book
• Numerical Computation chapter from Goodfellow et al DL book

Suggested Reading

• Chapter 2 in Michael Nielson’s NNDL textbook

Additional Readings:

• Paper by Rumelhart et al. that popularized backpropagation

Suggested Reading

• Chapter 2 in Michael Nielson’s NNDL textbook

Additional Readings:

• Paper by Rumelhart et al. that popularized backpropagation

Suggested Readings:

• Chap-3 from Goodfellow et al. DL 2016 book
• Chap-3 from Michael Nielson’s NNDL book

Suggested Readings:

• Ruder’s blog on variants of gradient-based updates
• Chapter on Optimization from Goodfellow et al. book
• Chapter 5 from Michael Nielson’s NNDL book

Suggested Readings:

• Chapter on Regularization from Goodfellow et al. book
• Chapter 5 from Michael Nielson’s NNDL book

Suggested Readings:

• CNNs module from CS231n
• CNNs Chapter from Goodfellow et al. DL book

Suggested Readings:

• CNNs module from CS231n
• CNNs Chapter from Goodfellow et al. DL book

Suggested Readings:

• Chapter 10 from Goodfellow et al DL book
• Denny’s tutorial on RNNs
• Colah’s blog on LSTMs
• GRU by Cho et al. 2014

Suggested Readings:

• Notes by Elena Voita
• GloVe Paper
• Word2Vec Paper

Suggested Readings:

• Lecture Notes/Material on by Elena Voita
• Sequence to sequence learning by Sutskever et al. NeurIPS 2014
• Neural Machine Translation with aligning by Bahdanau et al. ICLR 2015
• Show Attend and Tell by Xu et al. 2015

Suggested Readings:

• Chapter-12 from the Bishop’s book
• The Illustrated Transformer
• Attention is all we need!
• Computational complexity discussion
• My slides on self-attention layer for CNNs

Suggested Readings:

• Chapter-12 from the Bishop’s book

Suggested Readings:

• Chapter-19 from the Bishop’s book
• Chapter-14 from Goodfellow et. al.’s book

Suggested Readings:

• IJCAI-ECAI 2018 Tutorial on Generative Modeling

Suggested Readings:

• Joseph Rocca’s blog

Suggested Readings:

• Dive into DL, chapter on GANs
• DC GAN by Radford et al. NeurIPS 2016

Suggested Readings:

• Understanding Diffusion Models: A Unified Perspective
• Lilian Weng’s blog