感測與智慧系統

This course covers the sensing and intelligence aspects of robots, including the fundamental and advanced topics in deep learning and object recognition, 3D perceptions and pose estimation. We focus on cutting-edge methods and implementations, and will include 50% lecture and 50% hands-on lab tutorials in mobile manipulation. We will systematically study each components from previous winning teams of the Amazon Robotics Challenge (https://www.amazonrobotics.com/#/roboticschallenge) as well as underwater missions in RobotX Competition (https://robotx.org/).

This course involve a fair amount of probability, linear algebra, and programming. Students who took image processing, computer vision, and creative software project are encouraged to join. Being familiar with Unix-like system, Robot Operation System (ROS), C++, and Python are desirable but not required. * 本課程為機器人碩士學位學程必修課3選1課程 * 本課程為電控所博士班資格考課程之一

Github: https://github.com/Sensing-Intelligent-System/ Homepage: https://sensing-intelligent-system.github.io/ FAQ Q: is this course English mediated? A: Officially no, but all written materials, slides, exams will be in English. Q: I don’t have much experience about building/developing robots, should I enroll in the class? A: Yes, you are welcome to join the class. We will cover the most-needed technical skills, and real robot platforms will be provided. Q: Is this course teaching-oriented or research-oriented? A: first of all, a quick self evaluation: how many research papers you have read/written? Do you know problem formulation? Have you designed an experiment with a hypothesis? If you don’t know much about those, you come to the right class. We wish to make this course a transition from teaching to research. We will still cover fundamental materials in the first half, with hands-on materials from MIT 2.12. In the 2nd half it will be project-based and research-oriented. Q: Why hands-on materials? A: the goal of the class is to let you build an AI-powered mobile manipulator. I believe watching a robot you build in action is the most rewarded moment, and is the best way to motivate students. Q: why should we have a final exam? A: this course is one of the qualifying exam subjects in the PhD program of the Electrical and Control Engineering. I believe the current topics about deep learning and Intelligent robots, as well as how to read paper are must-know for any graduate student. Q: Why ROS? A: it’s the most commonly used middleware in robotics. You will be able to control lots of robots existing on earth. You can just learn it in our tutorial covered in this class, or learn it via our Duckietown course (hope you and your friends have joined our AI Driving introductory course). Q: what deep learning frameworks are used in the class and why? A: we will cover Caffe/Caffe2 and PyTorch. We want to run deep learning on embedded computer, which is a good reason to use Caffe/Caffe2. PyTorch is popular for all areas in machine learning, such as vision, NLP, and RL; we love Python. We will use onnx to convert the models. We also wish you could train your own models on GPU workstations. In fact, PyTorch team decided to marry PyTorch and Caffe2 which gives the production-level readiness for PyTorch. https://pytorch.org/2018/05/02/road-to-1.0.html

1.學期作業: Students form teams, with 2-3 people, to work on hands-on tutorials, including a few in-class activities and (take home) assignments. Each team should choose a mobile manipulation platform (ground or underwater vehicles) and develop a solution related to the competition tasks. Each team should present the project proposal in midterm and carry out systematic evaluations as term project in final presentation. There is no final report. 2. 考試狀況: There will be a final exam (open book and note) for individuals. The exam will focus on the topics of deep learning and object recognition, 3D perceptions, and the recent research papers covered in the course. Students are suggested to read all papers in the given list, about 10-15 papers from top conferences or journals in related fields. Students will be asked to write essays or draw tables/diagrams with pen and paper, and/or use individual laptop without internet communication. The questions will include (but not limited to) compare algorithms, state the limitations and challenges of existing problems, or analyze the experiment results for the papers in the given list. The exam may include a mock review process for a short article not in the given list. The evaluations of the exam will include at least two domain experts. All writing should be in English. The exam duration is 3 hours. * The format of the doctoral qualifying exam will be similar to the final exam. 3.評量方法: Hands-on Tutorials and Assignments (40%), Midterm Presentation (15%) Final presentation (20%) Final Exam (25%)

1. Computer Vision: Algorithms and Applications, Richard Szeliski, Springer, 2010. 2. Robotics, Vision, and Control, Peter Croke, Springer, 2011. 3. Introduction to Autonomous Robots, Nikolaus Correll, 2015.

地點

EE-759

時間

By appointment

聯絡方式

Email: hchengwang@gmail.com