CS:4980:006 Deep Learning with Python, Fall 2018

Syllabus, Homeworks, Project, Quiz, Lecture Notes,

Announcements

• 08/21: Welcome to the class! This web site is the place where you will see annoucements, homework assignments, etc.

Homeworks

There are 10 assignments, each counts for 4-8% of the final grade, depending on the difficulty. The percentage of all the homework assignments is 60% of the final grade. Each homework is due in about one or two weeks.

1. (40 points) Homework 1 is due on 9/6/18.
2. (40 points) Homework 2 is due on 9/13/18.
   Here is an implementation of the cross-entropy cost with sigmoid activation: mlnnSGD2.py
3. (40 points) Homework 3 is due on 9/20/18.
4. (80 points) Homework 4 is due on 10/2/18.
5. (40 points) Homework 5 is due on 10/9/18.
6. (80 points) Homework 6 is due on 10/18/18.
7. (80 points) Homework 7 is due on 10/30/18.
8. (80 points) Homework 8 is due on 11/9/18.
9. (40 points) Homework 9 is due on 11/16/18.
10. (80 points) Homework 10 is due on 11/30/18.

Project

Project Proposal Requirement

Project Progress Report

Final Project Report

Lecture Notes

• Lecture 1: Introduction Lecture 1 PDF
• Lecture 2: Gradient Descent Lecture 2 PDF
• Lecture 3: NumPy for NN Lecture 3 PDF mlnn.py
• Lecture 4: SGD: Stochastic Gradient Descent Lecture 4 PDF
• Lecture 5: Improve NN Performance: Gradient Values Lecture 5 PDF mlnnSGD.py
• Lecture 6: Improve NN Performance: Learning Rate and Weight Values Lecture 6 PDF
• Lecture 7: Implement NN in Theano Lecture 8 PDF Shared variable example Logistic example Neural Network example and Test on MNIST
• Lecture 8: Convolutional NN Lecture 8 PDF Testing convolutional function Test CNN on MNIST CNNTF.py
• Guest Lecture: Introduction to Argon Cluster Argon Cluster PDF Dat Hong's tutorial on Argon
• Lecture 9: Improve NN Performance: Ensemble and Dropout Lecture 9 PDF CNNdropout.py
• Lecture 10: Inception and Residual Network Lecture 10 PDF
• Lecture 11: Recurrent Neural Network Lecture 11 PDF rnnecho.py
• Lecture 12: Compressing Neural Networks Lecture 12 PDF
• Lecture 13: Long Short Term Memory RNN Lecture 13 PDF rnnLSTMecho.py
• Lecture 14: Word2Vec Lecture 14 PDF word2vec.py
• Lecture 15: Neural Machine Translation Lecture 15 PDF
• Lecture 16: GRU in seq2seq for Price Prediction Lecture 16 PDF
• Lecture 17: Autoencoders Lecture 17 PDF autoencoder.py denoiseauto.py keras-vae.py
• Lecture 18: Neural Arts Lecture 18 PDF deepdream.py
• Lecture 19: Genrative Adversarial Networks Lecture 19 PDF dcgan.py
• Lecture 20: Reinforcement Learning Lecture 20 PDF paddle_game.py rl_game_train.py rl_game_test.py rl-network.h6
• Lecture 21: Adversarial Games and AlphaGo Lecture 21 PDF
• Lecture 22: Automatically Generating NNsLecture 22 PDF

• Neural Networks and Deep Learning, online book by Michael Nielsen, http://neuralnetworksanddeeplearning.com
• Python Deep Learning, by V. Zocca, G. Spacagna, D. Slater, P. Roelants, 2017, http://www.packtpub.com
• Theano Deep Learning Tutorial, LISA Lab, University of Montreal, http://deeplearning.net/tutorial/deeplearning.pdf
• TensorFlow 1.x Deep Learning Cookbook, by Antonio Gulli, Amita Kapoor, 2017, http://www.packtpub.com

• Top 100 most cited papers in deep learnng
• Deep Learning Reading List by deeplearning.net, http://deeplearning.net/reading-list/
• Practical recommendation for SGD in deep learnng
• On optimization methods for deep learning
• Natural gradient for deep networks
• Deep Learning https://en.wikipedia.org/wiki/Deep_learning
• Deep Learning by Ian Goodfellow, Yoshua Bengjo, Aaron Courville, MIT Press, 2016
• Fathom: Reference Workloads for Modern Deep Learning Methods
• Deep Learning Tutorials
• Deep Learning tutorial
• Theano Documentation
• No free lunch theorems

Interesting Readings

• The Singularity Is Near
• Homo Deus: A Brief History of Tomorrow
• Engineers vs Scientists
• IBM Watson wins the Jeopardy
• AlphaGo Zero

Have a great day!