For the information on Instructor and Teaching Assistant, please see the course website.

Purpose: The main objective of this course is to expose undergraduate and graduate students to one of the hot topics in machine learning: deep neural networks. Deep learning is a powerful tool for modeling and extracting layered high-level representations of data in a way that maximizes performance on a given task. Deep learning is behind many recent advances in AI, including Siri's speech recognition, Facebook's tag suggestions and self-driving cars. We will cover a range of topics from basic neural networks, logistic regression, convolutional and recurrent network structures, deep unsupervised and reinforcement learning, and applications to problem domains like speech recognition and computer vision.

Course Outline (tentative):

• Feed-Forward Neural Networks
• Sigmoid, Tanh, and ReLU Neurons and Softmax Output Layers
• Gradient Descent and The Backpropagation Algorithm
• Learning Rate Adaptation and Stochastic and Minibatch Gradient Descent
• Test Sets, Validation Sets, Overfitting
• Regulation and Momentum-Based Optimization in Gradient Descent
• Convolutional Neural Networks
• Filters and Feature Maps and Max Pooling
• Accelerating Training with Batch Normalization
• Visualizing Learning in Convolutional Networks
• Embedding and Representation Learning
• Models for Sequence Analysis
• Long Short-Term Memory (LSTM) Units
• Memory Augmented Neural Networks
• Deep Reinforcement Learning
• Markov Decision Processes (MDP)
• Policy Learning Versus Value Learning
• Q-Learning and Deep Q-Networks
• Unsupervised Reinforcement and Auxiliary Learning (UNREAL)
• Implementing Neural Networks in Theano
• Implementing Neural Networks in TensorFlow
• Implementing Neural Networks in Keras

There will be ten homework assignments (each 6-8% of the final grade, depending on difficulty) and a term project (40% of the final grade). Most homeworks contain programming in Python.

Students should be prepared to put in considerable time and effort into reading and programming to become familiar with the course's topics, and gain experience with the techniques seen in class.

Prerequisite: In addition to the formal prerequisite that a C- or better in CS:2210 and CS:2230, a strong mathematical background in calculus, linear algebra, and probability and statistics, as well as enjoying programming, is essential for the success of this course. You will learn to use Python if you have not used Python before.

Textbooks:

• Neural Networks and Deep Learning, online book by Michael Nielsen, http://neuralnetworksanddeeplearning.com
• Python Deep Learning, by V. Zocca, G. Spacagna, D. Slater, P. Roelants, 2017, http://www.packtpub.com
• Theano Deep Learning Tutorial, LISA Lab, University of Montreal, http://deeplearning.net/tutorial/deeplearning.pdf
• TensorFlow 1.x Deep Learning Cookbook, by Antonio Gulli, Amita Kapoor, 2017, http://www.packtpub.com

• Deep Learning https://en.wikipedia.org/wiki/Deep_learning
• Deep Learning by Ian Goodfellow, Yoshua Bengjo, Aaron Courville, MIT Press, 2016
• Deep Learning Reading List by deeplearning.net, http://deeplearning.net/reading-list/
• Fathom: Reference Workloads for Modern Deep Learning Methods

Course Accounts: You will need a HawkID and a password to login to ICON to electronically submit your assigned work.

Grading Plus/Minus grading and curving will be used for the course. There are three components that will determine your grade.

• Homework Assignments: 60%

  You will have five homeworks (most of them need Python programming), usually due in one or two weeks. Each homework will be counted for 4-8% of your final score (60% total).

• One Project: 40%

  The details will be provided.

You may discuss homework with others or use online resources. However, do not share your work (including codes) with others or ask others to see their completed assignments since both are considered academic misconduct. Late submissions will not be accepted and in general you will be better off turning in what you have on time rather than seeking extra time to complete your work. Starting early is the key, and the TA and I will be glad to help with any questions you may have on the assignments. So please visit us often during our office hours and if necessary, outside our office hours as well.

Required Legalese: This course is run by the College of Liberal Arts and Sciences. This means that class policies on matters such as requirements, grading, and sanctions for academic dishonesty are governed by the College of Liberal Arts and Sciences. Students wishing to add or drop this course after the official deadline must receive the approval of the Dean of the College of Liberal Arts and Sciences. Details of the University policy of cross enrollments may be found online: https://clas.uiowa.edu/faculty/teaching-policies-resources-syllabus-insert

Student Complaints: If you have any complaints or concerns about how the course is being conducted by me or by the TA please feel free to talk to me. You are also welcome to get in touch the the Computer Science department chair, Prof. Alberto Segre (alberto-segre@uiowa.edu, 14D McLean Hall). Consult the college policy on Student Complaints Concerning Faculty Actions (online version) for more information.