Abstract
This course covers the core technologies required to model and simulate digital humans. The curriculum includes human body modeling, human motion capture, data-driven human motion synthesis, and ML-based generative models. Each topic will be extensively illustrated with examples to provide a comprehensive understanding of the subject matter.
Objectives
Students will learn how to estimate human pose, shape, and motion from videos and create basic human avatars from various visual inputs. Students will also learn how to represent and algorithmically generate motions for digital characters. To help students prepare for the midterm exam, four ungraded exercises will be released following the corresponding lectures and will be reviewed during the tutorial sessions.Content
- Basic concepts of 3D representations- Human body/hand models
- Human motion capture
- Neural rendering
- Transformers
- Generative models for digital humans
Lecture Notes
Lecture and tutorial slides will be available on moodle.Prerequisites
Experience with python and C++ programming, numerical linear algebra, multivariate calculus and probability theory. Some background in deep learning, computer vision, physics-based modeling, kinematics, and dynamics is preferred.Administration
| Number | 263-5806-00L |
|---|---|
| Lecturer | Prof. Dr. Siyu Tang, Dr. Sergey Prokudin |
| Assistants | Kaifeng Zhao (head TA) Zinuo You (head TA) Gen Li Xiaozhong Lyu Yutong Chen Frano Rajič Yan Wu Johannes Weidenfeller Malte Prinzler Bahri Batuhan Bilecen |
| Location and Time | Lecture: Tue 14:15-17:00 LFO C 13 Tutorial: Thu 16:15-18:00 ETF E 1 |
| Moodle | https://moodle-app2.let.ethz.ch/course/view.php?id=27533 |
| ECTS Credits | 8 |
| Exam | The grade will be determined by 40% interim examination and 60% final project presentation and report. |
Schedule
Lecture
| Week | Date (14pm-17pm) | Topic |
|---|---|---|
| 01 | 17-Feb | Introduction |
| 02 | 24-Feb | Human body models |
| 03 | 3-Mar | From Images to Human Models |
| 04 | 10-Mar | Mesh-based Human Avatars |
| 05 | 17-Mar | Volumetric Human Avatars (Neural Fields) |
| 06 | 24-Mar | Point-based Human Avatars (3D Gaussian Splats) |
| 07 | 31-Mar | Generative Models |
| 08 | 7-Apr | Easter Break (no class) |
| 09 | 14-Apr | Midterm Exam |
| 10 | 21-Apr | Project presentation |
| 11 | 28-Apr | Project Office Hour |
| 12 | 5-May | Project Office Hour |
| 13 | 11-May | Project Office Hour |
| 14 | 19-May | Project Office Hour |
| 15 | 26-May | Project presentation |
Tutorial
| Week | Date (16pm-18pm) | Topic |
|---|---|---|
| 01 | 19-Feb | |
| 02 | 26-Feb | |
| 03 | 5-Mar | Exercise 1 |
| 04 | 12-Mar | Exercise 2 |
| 05 | 19-Mar | Exercise 3 |
| 06 | 26-Mar | Project Introduction |
| 07 | 2-Apr | Exercise 4 |
| 08 | 9-Apr | Easter Break (no class) |
| 09 | 16-Apr | Pytorch Tutorial |
| 10 | 23-Apr | Cluster Tutorial |
| 11 | 30-Apr | Project Office Hour |
| 12 | 7-May | Project Office Hour |
| 13 | 14-May | Knabenschiessen (no class) |
| 14 | 21-May | Project Office Hour |
| 15 | 28-May | Project Office Hour |
Assistants:
Kaifeng Zhao
PhD student CAB G 65
PhD student CAB G 65
Zinuo You
PhD student CNB G 100.5
PhD student CNB G 100.5
Gen Li
PhD student CAB G 82.1
PhD student CAB G 82.1
Xiaozhong Lyu
PhD student CAB G 89
PhD student CAB G 89
Yutong Chen
PhD student CAB G 65
PhD student CAB G 65
Frano Rajič
PhD student CAB G 65
PhD student CAB G 65
Yan Wu
PhD student CAB G 82.1
PhD student CAB G 82.1
Johannes Weidenfeller
PhD student CNB G 100.5
PhD student CNB G 100.5
Malte Prinzler
PhD student CAB G 65
PhD student CAB G 65
Bahri Batuhan Bilecen
PhD student CNB G 100.5
PhD student CNB G 100.5