COMP 690: Deep Generative Models

Overview

This course provides an advanced introduction to deep generative models — the algorithms behind modern image, video, 3D, action, and language generation systems. It is open to graduate students and senior undergraduates. The course is split roughly in half: Part I covers the core principles and theory behind the major model families (autoregressive models, variational autoencoders, normalizing flows, generative adversarial networks, energy-based and score-based models, diffusion models, and flow matching), along with discrete generative models and techniques for efficient training and fast inference; Part II turns to applications, exploring how these techniques drive current state-of-the-art systems in unified multimodal (omni) models, video synthesis, 3D content creation, robotics, and the sciences. Students will read and present recent papers and complete a research-style final project.

Logistics

Time

Tuesday & Thursday, 5:00 – 6:15 PM

Dates

August 20 – December 10, 2026

Location

FB 007

Instructor

Jason Ren

Office Hours

Tuesdays 4:00 – 5:00 PM, FB 240

Prerequisites

Familiarity with probability, linear algebra, and deep learning fundamentals (e.g., a prior ML/DL course)

Grading

• 50% — Final project
  • 5% — Proposal
  • 20% — Presentation
  • 25% — Final artifacts
• 25% — Paper reviews
• 25% — Paper presentation

Plus a 5% bonus for active class participation. Good discussions happen when everyone shows up having read the papers, so please come prepared and jump in when you have thoughts or questions. Don't worry if a paper isn't fully clear to you — you're probably not the only one, and asking about it is often exactly what gets a discussion going.

Final Project

The final project is the heart of this course — a chance to turn what you've learned about generative modeling into a piece of original research. We're far more excited by an ambitious, original idea than by a safe, predictable one. We strongly encourage novel ideas, algorithms, or applications — even (especially!) if your experiments are still preliminary by the end of the semester. Simply reproducing numbers from an existing paper, running an existing pipeline on a similar new dataset, or tuning parameters/prompts on top of someone else's method is unlikely to be very exciting to us, and won't be evaluated as favorably as a genuine attempt at something new. Take the risk — we'd rather see you chase a hard, interesting question than play it safe.

• Proposal: 5-minute in-class presentation, plus a written proposal (up to 2 pages) covering:
  • Overview: What problem are you tackling, and what's your core idea?
  • Related work: What relevant work exists (not limited to papers covered in this course)? What makes your project distinct from what's already been done?
  • Approach: What representations and algorithms will you use? What's the technical novelty — can you explain it clearly, and ideally formalize it?
  • Experiments: How will you evaluate your approach — through ablations, comparisons to existing methods, or both? What data will you use, and what baselines will you compare against? If you plan to collect your own data, explain how.
  • Others: What do you hope to learn or produce from this project? What's still uncertain at this stage? Share any early results if you have them.
• Final artifacts:
  • A report (up to 8 pages) covering:
    • Introduction: Summarize the problem, your main idea, and your results.
    • Related work: Give a thorough account of related papers (not limited to those in the schedule). If you're proposing something new or extending prior work, contrast it directly against that work; if you're analyzing existing techniques, situate them relative to the broader literature.
    • Approach: Describe your method in enough detail that a reader could re-implement it without consulting outside sources — clearly state the inputs, outputs, and any assumptions or design decisions you made along the way.
    • Experiments & Results: For each experiment, explain what you ran, why you ran it, and what you learned from the outcome. Back this up with figures, tables, and qualitative examples wherever they help make the point.
    • Limitations: Be upfront about where your approach falls short and what constraints shaped your results.
    • LLM usage: Use of LLMs is allowed — describe how they were used throughout the project.
    • Team contributions: Clearly state each team member's individual contributions and how the work was split.
    • Conclusion: Recap the main idea and findings, and discuss what you'd explore next.
  • A results webpage showcasing qualitative results (visualizations, images/videos, figures)
  • Source code
  • Dataset & Benchmark (if applicable)
• Project presentation: 10-minute talk + 5-minute Q&A
• Team size: Up to 3 people

All written components (proposal and final report) should use the CVPR template.

You're welcome to stop by office hours at any point in the semester to talk through your project — whether that's brainstorming ideas, debugging a puzzling result, working through a failure, or just planning next steps.

Paper Reviews

Before each Readingclass, every student must submit a short review of one of the suggested papers via [Google Form link — TBD]. The form will contain the following questions:

• What's the main contribution or novelty of this paper?
• What's the most questionable claim in the paper, and how would you attack it?
• What are the strongest and weakest results in the paper, and how would you improve them?
• Do the experiments actually support the paper's claims?
• How could this work be extended?
• What questions do you have for the instructor/presenter — confusing points, open questions, etc.?
• ...

Reviews are due 11:59 PM ET the day before class. We don't accept late submissions, but everyone gets two missed reviews forgiven over the semester. You may skip submitting a review for the class in which you are presenting a paper.

Paper Presentation

For each paper covered in a Readingsession, either a student or the instructor will present it — one person per paper — and lead the in-class discussion. The instructor will then close out the session with a 10-minute summary discussion tying all the papers together.

• Sign up for papers here: [Google Sheet link — TBD]
  • Each student will present X papers over the semester (the exact number will be determined once final enrollment is set)
  • The instructor will present any papers that remain unclaimed
• Aim for roughly a 20-minute talk (about 15 slides) that's well-organized and polished, covering:
  • A clear statement of the problem
  • Why the problem matters — what makes it interesting, important, and/or hard
  • The paper's key contributions and technical ideas
  • The experimental setup and results
  • Strengths and weaknesses
  • Open questions, possible extensions, and/or applications worth discussing
• Keep slides visual — lean on figures, animations, and videos rather than text. Feel free to pull material from the authors' project page or webpage, and it's fine to reuse the authors' own slides, but clearly cite any slide that isn't originally yours.
• [Required!] Email the instructor a draft of your slides two days before your presentation (Sun 5 PM for Tue classes, Tue 5 PM for Thu classes) to get feedback.
• [Required!] Email your final slides to the instructor before class, making sure you have incorporated that feedback.

Schedule

The schedule is tentative and subject to change.

Resources

Related courses

• Stanford CS236: Deep Generative Models
• Cornell Tech/UCLA/Stanford/Caltech: Deep Generative Models
• MIT 6.S978: Deep Generative Models
• CMU 18-789: Deep Generative Modeling
• Berkeley CS294-158: Deep Unsupervised Learning
• MIT 6.S184: Generative AI with Stochastic Differential Equations
• CMU 16-726: Learning-Based Image Synthesis

Blogs & tutorials

• Lilian Weng, What are Diffusion Models?
• Lilian Weng, Flow-based Deep Generative Models
• Yang Song, Generative Modeling by Estimating Gradients of the Data Distribution
• Sander Dieleman's blog (extensive writing on diffusion models)
• Calvin Luo, Understanding Diffusion Models: A Unified Perspective
• Fjelde et al., An Introduction to Flow Matching
• Hugging Face, The Annotated Diffusion Model

Misc.

• Honor Code: All students are expected to abide by the UNC Student Code of Conduct.
• Academic Integrity: For your presentations and projects, you are allowed to use materials from external sources. However, you must clearly acknowledge those sources.