Course Information
About
This course gives a detailed overview of the fundamental concepts and recent advances in deep unsupervised learning where the topics covered include autoregressive models, normalizing flow models, variational autoencoders, generative adversarial networks, energy-based models, generative models with discrete latent variables, self-supervised learning, pretraining language models. The course is designed to familiarize students with the state-of-the-art in deep unsupervised learning by exposing them to a wide range of models and concepts in this new exciting field.
The course is taught by Aykut Erdem.
The course will use Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville as the textbook (draft available online and for free here).
Instruction style: During the semester, students are responsible for studying and keeping up with the course material outside of class time. These may involve reading particular book chapters, papers or blogs and watching some video lectures. After the first three lectures, each week a student will present a paper related to the topics of the week.
Time and Location
Lectures: Mondays and Wednesdays at 08:30-09:45 (online via Zoom)
Communication
The course webpage will be updated regularly throughout the semester with lecture notes, presentations, assignments and important deadlines. All other course related communications will be carried out through Blackboard.
Prerequisites
COMP201 is open to all COMP graduate students. Prospective senior undergraduate students may also enroll to the course but a consent is required. Similarly, non-COMP graduate students should also ask the course instructor for approval before the add/drop period. The prerequisites for this course are:
- Programming (you should be a proficient programmer to work out the assignments and to implement your course project.)
- Calculus (differentiation, chain rule) and Linear Algebra (vectors, matrices, eigenvalues/vectors) (MATH107)
- Basic Probability and Statistics (random variables, expectations, multivariate Gaussians, Bayes rule, conditional probabilities) (ENGR200)
- Machine Learning or Deep Learning (you can still survive this course without a machine learning course before, but it is highly recommended (ENGR421, COMP541)
- Optimization (cost functions, taking gradients, regularization)
Course Requirements and Grading
Grading for COMP547 will be based on
- Assignments (40%) (4 assignments x 10% each)
- Midterm Exam (10%)
- Course Project (presentations and reports) (32%),
- Paper Presentations (9%),
- Weekly Quizzes (9%),
Reference Books
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, MIT Press, 2016 (draft available online)
Schedule
| Week | Topic | Assignments |
| Feb 15-17 | Introduction | |
| Feb 22-24 | Neural Networks Basics | |
| Mar 1-3 | Autoregressive Models | Assignment 1 out |
| Mar 8-10 | Normalizing Flow Models | |
| Mar 15-17 | Variational Autoencoders | Assg 1 due, Assg 2 out |
| Mar 22-24 | Generative Adversarial Networks | Project proposal due |
| Mar 29-31 | Energy-based Models | Assg 2 due, Assg 3 out |
| Apr 5-7 | No classes - Spring Break | |
| Apr 12-14 | Discrete Latent Variable Models | Assg 3 due, Assg 4 out |
| Apr 19-21 | Project Progress Presentations | Project progress reports due |
| Apr 26-28 | Self-Supervised Learning | Assg 4 due |
| May 3-5 | Strengths and Weaknesses of Current Models | |
| May 10-12 | Pretraining Language Models | |
| May 17 | Final Project Discussions | |
| May 24-26 | Midterm Exam | |
| May 24-26 | Final Project Presentations | |
| May 24-26 | Final Project Presentations |