I Semester 2023-24

Differentiable graphics is an emerging family of techniques that enable graphics algorithms to be combined with machine learning for many cutting-edge applications in computer vision, robotics, fabrication, and other fields. While traditional computer graphics solves the forward problem of predicting the appearance and dynamics of objects given the description of a virtual scene, differentiable graphics also provides the derivatives of the output image or motion with respect to the scene parameters. This allows gradient-based techniques to be applied to optimize any desired objective, such as recovering the shape and material of a real object from one or more input photographs, or computing control parameters for robotic manipulation of soft materials. This course will cover the basic theoretical foundations and implementation techniques underlying differentiable algorithms for rendering and simulation, and survey some of the key recent advances in this emerging field.

**Texts:** We will begin by covering the material from a
few recent courses presented at SIGGRAPH:

- Öztireli et al.,
*TensorFlow Graphics: Differentiable Computer Graphics in TensorFlow*, SIGGRAPH 2021 - Zhao et al.,
*Physics-Based Differentiable Rendering: A Comprehensive Introduction*, SIGGRAPH 2020 - Coros et al.,
*Differentiable Simulation*, SIGGRAPH 2021

The rest of the reading material in this course will primarily consist of recent research papers. A list of such papers will be added shortly.

**Prerequisites:** Students are expected to have taken a
course on at least one of the following topics: (i) computer graphics,
(ii) computer vision, (iii) machine learning. If you are not sure
whether you have adequate background, please contact the instructor.

The course is expected to consist of about 10-12 sessions of 1.5 hours each. They will start around mid-August and continue until late September.

The first few sessions will be lectures delivered by the instructor introducing the key ideas underlying differentiable rendering and simulation. After this, the remaining sessions will be conducted in seminar style, with student presentations and discussions of various recent research papers in this area. Students will also write a short report on one of the presented papers, summarizing its key ideas for a general computer science audience.

**Lecture slides:**

- 19 Aug: Introduction
- 22 Aug: Scene representations
- 29 Aug: Physics-based rendering
- 1 Sep: Differentiable simulation

**Paper presentations:**

- Presentation: 40%
- Report: 30%
- Participation: 30%

**Presentation format:** Presenters should read their
chosen paper in detail and prepare a 15-20 minute presentation on it.
The presentation should clearly describe: the *problem* the paper
is trying to solve, the fundamental *challenges* that make it a
hard problem, the *key ideas* that make the paper work, and the
*tradeoffs and limitations* inherent to the approach.

All other students are expected to have gone through the paper at
least superficially, and prepared some pertinent questions related to it
(these need not just be about *how* does the method work, but
also *why* this approach was chosen, or *why not* use a
different/simpler approach?). We will discuss these questions after the
presentation.

**Report format:** Report writers are expected to read
their assigned paper in detail, attend the associated presentation, take
notes of the discussion, and finally write a 2-3 page report on the
paper. The report may be based on the presentation but should not be
restricted to it; it is a report on the *paper*, not the
*presentation*. In particular, it should also incorporate any
interesting points raised in the post-presentation discussion.

The report should be submitted within 1 week of the presentation. For uniformity, students should use this template based on the ACM publication format.

**Grading:** Following institute policy, a minimum of
80% marks are required for an A grade, and minimum 30% marks for D.

**Audit policy:** A minimum of 40% marks and 75%
attendance is required for audit pass.

**Attendance policy:** Attendance lower than 75% will
result in a one-grade penalty (e.g. A to A-).