機器人視覺

Course Description: This course aims at training students to participate in world-class competition, and we will target the RobotX Competition (https://www.robotx.org/) in 2022. This course will cover the fundamental and advanced domains in vision for mobility tasks, including the challenges combining object recognition, pose estimation, motion planning, and deep reinforcement learning. We will systematically study each component from previous winning teams in 2014, 2016, and 2018, as well as cutting-edge methods and advanced topics using Gazebo, 3D perception, and deep learning. Students will form teams to develop term projects, and we encourage in-class discussions and presentation skills. This course is a "learning-by-doing," and includes in-class lab/tutorial in each teaching module. We will use Duckietown and Duckiepond as learning platforms, and Xilinx KV260 for perception tasks. We will host a “When Robots Meet AI Chips” in mid-June, based on the course modules developed in this course.

Prerequisite: ● This course involves a fair amount of probability, linear algebra, and programming. Students are required to take image processing or computer vision. Being familiar with Unix-like systems, Git, C++, Robot Operation System (ROS) and Python is needed. ● Due to the tight competition schedule, we wish to cover more advanced topics directly relevant to the competition. We require prerequisites and assume that you are familiar with ROS and CNN. ● Due to the limited robots/computing resource for Gazebo and Deep Learning, we will ask all attending students to fill in a self-evaluation form and motivation statement. We can only enroll a limited number of students according to students' 1) motivation, 2) knowledge of Unix/Ubuntu, Vim, Git, 3) ROS, 4) computer vision and image processing, and 5) your own computing resource (such as a native Ubuntu machine with GTX 1060 or above). ● We will not host audit students due to limited resources and in-class discussions in small groups/teams. ● The order of registration may be used in a potential lottery.

Class Organization: ● All lectures and discussions will be online events. Two hours per week. ● Learning by Doing: We have lecture and lab sections, and include hand-on tutorial/lab materials each week. We also wish to train students' presentation skills via the presentation of the tutorials. ● In-class Discussion: We encourage discussions and presentation skills using a shared document each week for class participation. ● Hands on with real robots. Each student will arrange 6 times of real robot experiments or demos with TA as course requirements. ● Project-based learning: each team will work on a term project related to RobotX Competition or other mobility tasks. 注意事項: ● 我們課程的先修課程與技能為ROS, 影像處理等,也需要對Ubuntu, Git, Python等有ㄧ定熟悉度 ● 上課第一週會請所有同學填寫自評表,將以動機、對先修課程與技能的熟悉度高的學生優先,若人數過多則以抽籤決定。

Assignments: We have an in-class tutorial/lab each week, and the lab materials typically include 3 specific tasks (programming/algorithm/system work). and students are expected to finish the lab materials during the class. Exams: There will be no midterm/final exams. Evaluations: Class Participation (10%) In-Class Lab/Tutorials (40%), Midterm Project Proposal (10%) Final Presentation and Term Paper/Report (40%)

Textbooks and Resources: 1. Computer Vision: Algorithms and Applications, Richard Szeliski, Springer, 2010. 2. Robotics, Vision, and Control, Peter Croke, Springer, 2011. 3. Introduction to Autonomous Robots, Nikolaus Correll, 2015. 4. RobotX Competition https://www.robotx.org/

時間

By appointment

聯絡方式

Instructor: Prof. Nick Wang