可信任與節能機器學習
Trustworthy and Green Machine Learning
| 節 | 週二 |
|---|---|
5 13:20–14:10 | 可信任與節能機器學習 CM215 3 節連堂 |
6 14:20–15:10 | |
7 15:30–16:20 |
* 根據陽明交大上課時間表所列
This course consists of two parts: trustworthy machine learning algorithms and efficient algorithm development and implementation. In the first part, we will discuss algorithms that provide reliable confidence scores for their predictions in a systematic and analytical manner, including Gaussian Process regression and classification. In the second part, we will explore efficient accelerated computation, covering training, fine-tuning, and inference of large language models on consumer-grade computers.
Basic machine learning concepts, such as overfitting, gradient descent, neural networks etc..
Websites and reference books. 1. Hennig, P., 2020. Probabilistic Machine Learning. lecture course, University of Tübingen, URL = https://uni-tuebingen.de/en/180804 2. Kevin P. Murphy, Probabilistic Machine Learning: An introduction, MIT Press, 2022, URL = https://probml.github.io/pml-book/book1.html URL = https://probml.github.io/pml-book/book2.html
Homework: 30% Project: 50% Attendance: 20% Ex1. Basic Bayesian inference. (Coding) Ex2. Gaussian linear regression (Coding) Ex3. Gaussian process regression (Coding) Ex4. Integrating Gaussian process classifier and deep neural networks. (Coding) Ex6. Integrating CNN and Gaussian process classifier. (Coding and report) Ex7. Memory-efficient LLM Training with GaLore (Coding and Report) Ex8. Memory-efficient LLM Training with PowerInfer (Coding and Report)
| 週次 | 主題 |
|---|---|
| 第 1 週 | Course outline |
| 第 2 週 | Reasoning under uncertainty Bayesian theorem |
| 第 3 週 | Monte Carlo sampling |
| 第 4 週 | Bayesian inference, Gaussian distribution and Gaussian process |
| 第 5 週 | Understanding kernels and similarity metrics |
| 第 6 週 | Example: Financial Application of Gaussian processes |
| 第 7 週 | Gaussian process classification |
| 第 8 週 | Generalized linear model and Exponential families |
| 第 9 週 | Towards Bayesian Neural network: Last-layer Laplace approximation vs. deterministic uncertainty estimation |
| 第 10 週 | Review and Report |
| 第 11 週 | Probabilistic classifier integrating deep neural network and Gaussian process (I) |
| 第 12 週 | Probabilistic classifier integrating deep neural network and Gaussian process (II) |
| 第 13 週 | Memory-efficient LLM Training |
| 第 14 週 | Memory-efficient LLM Inference |
| 第 15 週 | Large Foundation models |
| 第 16 週 | Term project report |
No textbook for this course. Pioneering and contemporary papers will be discussed in the class.
- 地點
- in the class
- 時間
- By appointment or after class every week.
- 聯絡方式
- machingwen@nycu.edu.tw