感測與智慧系統

This project-based course aims at training students to participate world-class competition. This course will cover the sensing and intelligence aspects of robots, including the challenges combining object recognition, pose recognition, grasp planning, compliant manipulation, motion planning, task planning, task execution, and error detection and recovery. We will introduce cutting-edge methods and prepare virtual environment (Gazebo) for a virtual challenge in class. 本課程為專題導向,課程前半部將以LoCoBot Competition的五項任務進行評分,課程後半段為修課同學分組報告介紹state-of-the-art演算法,以感測、深度學習智慧系統、機器人領域為主題。

This course involve a fair amount of probability, linear algebra, and programming. Being familiar with Unix-like system, Robot Operation System (ROS), Python, and basic knowledge of Deep Learning and Convolutional Neural Network (CNN) are required. * 本課程屬於進階課程,需有基礎ROS及CNN先備知識,先備知識考試為個人考試,繳交期限為2/28,未繳交或未通過將視為退選。考試形式為使用Colab Notebook,填寫Python程式碼後上傳E3系統,本課程將以nbgrader (https://nbgrader.readthedocs.io/en/stable/) 進行自動評分。 * 不熟悉Python同學建議參考 MIT OCW https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-0001-introduction-to-computer-science-and-programming-in-python-fall-2016/index.htm

M567時段(注意:本學期課程講授皆為線上進行:meet.google.com/gxj-pnbo-umx): 1-1 課程講授(Lecture):含理論介紹、Colab程式碼導讀 1-2 Colab 與 Python程式演練、專題討論 學生將分組進行課程討論,每位同學上課須攜帶筆電,筆電有Ubuntu系統,或可連線至個人Ubuntu工作站為佳。 本課程將使用Google Colab。 本學期將安排實作、實體機器人展示,強調課程介紹之演算法如何用於機器人任務。 2022課程網路資料夾: https://drive.google.com/drive/folders/1XiT5QbJ_vkH8E19sR0iqCYJnvqHtLOlx?usp=sharing 期末專題主題原則(參考MIT6.811課程 ),要點如下: https://manipulation.csail.mit.edu/Fall2020/project.html * 本課程專題應與人工智慧、機器人、感測主題相關。 * 課程專題建議以2-3人小組進行,專題的評分會根據人數進行調整(人數越多應有更多產出) * 專題應有適當的規模,可在一學期內完成,工作環境設置(Ubuntu, Simulation, GPU Workstation等)應儘早預備。 * 專題內容應反映出小組對課程內容的理解,如進行演算法實作,建議能將已有的Github Repo程式碼跑起來,重現已發表重要論文之方法,包含實驗結果作為baseline/benchmark。. * 本課程鼓勵期末專題與個人研究、參與研究室之研究主題相關,然需於計畫書說明本課程專題與其他專題的關聯,本課程專題內容應為學期初尚未完成之項目,不可為(個人、上學期修課同學、或研究室學長姐)已完成內容。Some of you will be able to do a project that is closely related to your current research. This is completely acceptable, as long as the subproblem you are solving is clearly defined. If you are using part or all of your work for research or as a part of another class, you must state this explicitly in your final report. We expect you to do something new that you would not have done if you hadn't taken this class. * 期末專題鼓勵使用開源程式碼,但須明確說明什麼部分是開源程式碼提供,什麼部分是專題成果,若使用GitHub repo應用fork,切勿重新上傳或移除原作者資訊(可能違反學術倫理)。 期中報告Proposal要點 The project proposal is intended as a forcing function for you to crystalize a project idea. Moreover, it gives us a chance to offer you a feedback and make sure that your plan is worthwhile and feasible. It should be approximately one page (at most, say, 500 words) detailing your idea for the project. It should: * Define exactly what the project deliverable is. * Briefly describe why the project is worthwhile and interesting. * List the topics that we have studied (or that we will study) in class that are covered by your project. * Discuss any related prior work you have found that is relevant. * Define specific goals that you expect to have accomplished before each of the progress updates. * It does not have to be extremely polished, but it does need to provide us with enough information to understand what you are hoping to do. Otherwise, we won't be able to help you! * 鼓勵各組以Teleop拍攝影片(under 3 mins, under 25 MB)as supplementary materials 期末報告與書面報告 The project presentation is due by the last day of classes. Extensions may be granted on request. The presentation must have duration of 3 minutes for individual projects and 5 minutes for group projects. Please try to share the 5 minutes evenly between the various members of the group. This year we will require you to upload presentations on E3. Using the IEEE Template for conference proceedings (probably the LaTeX one; we suggest the team lead to create an Overleaf project to share with team members), write a summary of what you accomplished during your project. Write it, as much as possible, like a conference paper. You should include: * An abstract. * An introduction of your project and why you think it is interesting. * A related work / literature review section * Your results (partial or work-in-progress is expected, and completely fine!) * A discussion of your results and potential next steps We don't have any requirements on length, but realize that this is worth a very large portion of your overall grade, so be as thorough as possible in demonstrating mastery of the course material. For reference, reasonable lengths could be anywhere between 3-8 pages, with figures and a few citations where relevant. Videos are encouraged as well, as a way to showcase simulation/hardware results and provide visual explanation. IEEE Latex Template: https://www.ieee.org/conferences/publishing/templates.html LoCoBot Competition Lab Materials: 00-ssh-key.md 00-workstation.md 01-open-gazebo-docker.md 02-project-gazebo-world.md 03-complie-test-ros-packages.md 04-smach-pyrobot.md 05-pushing-example-customized-world.md 06-teleop-docker-gazebo.md 07-navigation.md 08-placing.md 09-integrated.md 10-pose-estimation.md 11-detecion.md 12-virtualenv.md 13-spawn-objects.md 14-placing-arm-operation.md 15-hardware-in-the-loop.md 16-maskRCNN-detection.md 17-maskRCNN-detection-real.md 18-lanefollowing.md

1) 學期作業: 個人作業 There are in-class tutorials (in Python and Jupyter Notebook) 2) 期中報告: 以組為單位繳交書面計畫書(Proposal)一份,LoCoBot Competition任務評分。 3) 期末報告(含口頭報告、書面報告): 可選擇以實體或虛擬環境進行,若因疫情可能全部調整為virtual形式。 4) 期末考:為個人考試,以電控所博士班資格考試形式問答題之紙筆測驗為主。 3.評量方法: - Class Participation 10 (%) - LoCoBot Competition and Jupyter Notebook Exercises (30%) - Project Midterm and Final Report (30%) - Final Exam (30%) 期末考題說明: The exam will focus on the topics of deep learning and object recognition, 3D perceptions, and the recent research papers covered in the course. Students are suggested to read all papers in the given list, about 10-18 papers from top conferences or journals in related fields. Those papers are citing some fundamental papers as references. The questions will include (but not limited to) compare algorithms, state the limitations and challenges of existing problems, or analyze the experiment results for the papers in the given list. The exam may include a mock review process for a short article not in the given list. * Students will be asked to write essays or draw tables/diagrams with pen and paper, and/or use individual laptop without internet (or any) communication. * The exam duration is 3 hours. * As in previous SIS courses (2018, 2019), we will ask each team to present 1-2 papers included in the SIS paper list. The presentations will be arranged in the 2nd half of the course; 2 presentations each week. * Students are encouraged to suggest important literatures or baseline approaches to the SIS paper list, in particular the ones relevant to the final projects. Link to SIS paper list and slides (generated by students): https://drive.google.com/drive/folders/1wQaPNTFXAxjvq1NE5YSmH1xV992MI3hj?usp=sharing

1. Computer Vision: Algorithms and Applications, Richard Szeliski, Springer, 2010. 2. Robotics, Vision, and Control, Peter Croke, Springer, 2011. 3. Introduction to Autonomous Robots, Nikolaus Correll, 2015. 4. Introduction to Computation and Programming Using Python, third edition: With Application to Computational Modeling and Understanding Data by John Guttag