感測與智慧系統

This course is project-oriented, and the first half of the course will focus on the topic of Large Language Models (LLM), following a progressive structure based on the Hugging Face materials at https://huggingface.co/learn/nlp-course/chapter1/1?fw=pt. In the latter part of the course, students will be grouped for presentations covering state-of-the-art algorithms in topics such as sensing, deep learning, large language models, and robotic systems. 本課程為專題導向,課程前半部將以大型語言模型(Large Language Model, LLM)為主題,將以Huggingface教材循序漸進。 https://huggingface.co/learn/nlp-course/chapter1/1?fw=pt 課程後半段為修課同學分組報告介紹state-of-the-art演算法,以感測、深度學習、大型語言模型、機器人系統等為主題。

This course involve a fair amount of probability, linear algebra, and programming. Being familiar with Unix-like system (Ubuntu), version control (Git), Object-oriented Programming and Python, fundamental knowledge of Deep Learning and Convolutional Neural Network (CNN), PyTorch or Tensorflow are required. 本課程屬於進階課程,需有Ubuntu, Git, OOP & Python、CNN、以及PyTorch or Tensorflow先備知識,先備知識考試為個人考試(2/29),考試形式為紙筆考試(是非題)及Python程式(填寫於Jupyter Notebook後上傳E3系統),未通過將視為退選。

M567時段(注意:本學期課程講授皆為線上進行: 1-1 課程講授(Lecture):含理論介紹、程式碼導讀 1-2 Python程式演練、專題討論 學生將分組進行課程討論,每位同學上課須攜帶筆電,筆電有Ubuntu系統,或可連線至個人Ubuntu工作站為佳。 期末專題主題原則(參考MIT6.811課程 ),要點如下: https://manipulation.csail.mit.edu/Fall2020/project.html * 本課程專題應與人工智慧、機器人、感測主題相關。 * 課程專題建議以2-3人小組進行,專題的評分會根據人數進行調整(人數越多應有更多產出) * 專題應有適當的規模,可在一學期內完成,工作環境設置(Ubuntu, GPU Workstation等)應儘早預備。 * 專題內容應反映出小組對課程內容的理解,如進行演算法實作,建議能將已有的Github Repo程式碼跑起來,重現已發表重要論文之方法,包含實驗結果作為baseline/benchmark。. * 本課程鼓勵期末專題與個人研究、參與研究室之研究主題相關,然需於計畫書說明本課程專題與其他專題的關聯,本課程專題內容應為學期初尚未完成之項目,不可為(個人、上學期修課同學、或研究室學長姐)已完成內容。Some of you will be able to do a project that is closely related to your current research. This is completely acceptable, as long as the subproblem you are solving is clearly defined. If you are using part or all of your work for research or as a part of another class, you must state this explicitly in your final report. We expect you to do something new that you would not have done if you hadn't taken this class. * 期末專題鼓勵使用開源程式碼,但須明確說明什麼部分是開源程式碼提供,什麼部分是專題成果,若使用GitHub repo應用fork,切勿重新上傳或移除原作者資訊(可能違反學術倫理)。 期中報告Proposal要點 The project proposal is intended as a forcing function for you to crystalize a project idea. Moreover, it gives us a chance to offer you a feedback and make sure that your plan is worthwhile and feasible. It should be approximately one page (at most, say, 500 words) detailing your idea for the project. It should: * Define exactly what the project deliverable is. * Briefly describe why the project is worthwhile and interesting. * List the topics that we have studied (or that we will study) in class that are covered by your project. * Discuss any related prior work you have found that is relevant. * Define specific goals that you expect to have accomplished before each of the progress updates. * It does not have to be extremely polished, but it does need to provide us with enough information to understand what you are hoping to do. Otherwise, we won't be able to help you! 期末報告與書面報告 The project presentation is due by the last day of classes. Extensions may be granted on request. The presentation must have duration of 3 minutes for individual projects and 5 minutes for group projects. Please try to share the 5 minutes evenly between the various members of the group. This year we will require you to upload presentations on E3. Using the IEEE Template for conference proceedings (probably the LaTeX one; we suggest the team lead to create an Overleaf project to share with team members), write a summary of what you accomplished during your project. Write it, as much as possible, like a conference paper. You should include: * An abstract. * An introduction of your project and why you think it is interesting. * A related work / literature review section * Your results (partial or work-in-progress is expected, and completely fine!) * A discussion of your results and potential next steps We don't have any requirements on length, but realize that this is worth a very large portion of your overall grade, so be as thorough as possible in demonstrating mastery of the course material. For reference, reasonable lengths could be anywhere between 3-8 pages, with figures and a few citations where relevant. Videos are encouraged as well, as a way to showcase simulation/hardware results and provide visual explanation. IEEE Latex Template: https://www.ieee.org/conferences/publishing/templates.html

1) 學期作業: 個人作業 There are in-class tutorials (in Python and Jupyter Notebook) 2) 期中報告: 以組為單位繳交書面計畫書(Proposal)一份。 3) 期末報告(含口頭報告、書面報告)。 4) 期末考:為個人考試,以電控所博士班資格考試形式問答題之紙筆測驗為主。 3.評量方法: - Class Participation 10 (%) - Jupyter Notebook Exercises (30%) - Project Midterm and Final Report (30%) - Final Exam (30%) 期末考題說明: The exam will focus on the topics of deep learning and object recognition, 3D perceptions, large language model (LLM), and the recent research papers covered in the course. Students are suggested to read all papers in the given list, about 10-18 papers from top conferences or journals in related fields. Those papers are citing some fundamental papers as references. The questions will include (but not limited to) compare algorithms, state the limitations and challenges of existing problems, or analyze the experiment results for the papers in the given list. The exam may include a mock review process for a short article not in the given list. * Students will be asked to write essays or draw tables/diagrams with pen and paper, and/or use individual laptop without internet (or any) communication. * The exam duration is 3 hours. * Students are encouraged to suggest important literatures or baseline approaches to the SIS paper list, in particular the ones relevant to the final projects.

1. Speech and Language Processing (3rd ed. draft) by Dan Jurafsky and James H. Martin https://web.stanford.edu/~jurafsky/slp3/ 2. On the Opportunities and Risks of Foundation Models https://arxiv.org/pdf/2108.07258.pdf 3. Introduction to Computation and Programming Using Python, third edition: With Application to Computational Modeling and Understanding Data by John Guttag