進階三維電腦視覺 (英文授課)

The deep learning techniques have brought magic leap to the research area of computer vision, especially the remarkable advances in 2D vision tasks such as image recognition, detection, and segmentation. However, the challenge for duplicating the success of deep learning in 2D vision to 3D visual data has just begun, due to the difficulties stemmed from the irregularity and multi-modality of 3D data as well as the hardship for collecting the groundtruth annotations. Coming with the challenge, the chance/potential of integrating deep learning with 3D vision for the wide applications such as graphics, robotics, content creation, mixed reality has attracted lots of research efforts, we indeed witness quite some progresses in 3D vision in recent years. Therefore, the goal of this course is to give students a big picture of the latest techniques and trends in learning-based 3D vision, and build up the ability to discover or warm start their own research projects on the related topics. Particularly, my research group recently has quite some research projects and publications on learning-based 3D vision, the knowledge that having been accumulated would be quite beneficial to the students who are interested in such field and eliminate the burden for them to get hands dirty.

Linear Algebra, Probability, Calculus, Machine Learning, Deep Learning, Computer Vision Basics, Computer Graphics Basics

Course-related material will be uploaded onto E3New.

(Provisional) #Programming homework assignments (40~50%) #Paper study and presentation (10~20%) #Term project (35~45%) #Class participation (bonus)

• Introduction to Essentials for 3D Geometry: Projection and 3D Reconstruction
• Introduction to Multi-modal 3D Data, Ranging from Passive Stereo Cameras to Active Depth Sensors
• Multi-modal 3D Visual Data Fusion
• Supervised Learning for 2.5D and 3D Vision
• Unsupervised and Self-Supervised (Representation) Learning for 3D Vision
• Rendering of 3D Data
• Explicit and Implicit 3D Models, View Synthesis (e.g. SRN and NeRF)
• Generative Models for 3D Data
• Introduction to 3D Perception for Humans

The course ideally would run as a mixture between regular lectures (together with potential guest lectures) and student presentations, where the materials are mainly from the latest publications in top conferences and journals of computer vision or artificial intelligence fields. Though there is no textbook available for these recent advances, students can find some basics of 3D geometry from the following books: “Hartley and Zisserman, Multiple View Geometry in Computer Vision” “Ma, Soatto, Košecká, Sastry, An Invitation to 3-D Vision: From Images to Models” The students will also be encouraged to follow some online seminars/lectures, such as https://3dgv.github.io/

地點

EC526

時間

To be announced

聯絡方式

walon@cs.nctu.edu.tw