可信任節能生成式AI
Trustworthy Green Generative AI
| 節 | 週二 |
|---|---|
5 13:20–14:10 | 可信任節能生成式AI CM216 3 節連堂 |
6 14:20–15:10 | |
7 15:30–16:20 |
* 根據陽明交大上課時間表所列
This course (Trustworthy Green Generative AI) consists of two parts: trustworthy machine learning algorithms and efficient algorithm development and implementation. In the first part, we will discuss algorithms that provide reliable confidence scores for their predictions in a systematic and analytical manner, including Gaussian Process regression and classification. In the second part, we will explore efficient accelerated computation, covering training, fine-tuning, and inference of large language models on consumer-grade computers.
Basic machine learning concepts, such as overfitting, gradient descent, neural networks etc..
Websites and reference books. 1. Hennig, P., 2020. Probabilistic Machine Learning. lecture course, University of Tübingen, URL = https://uni-tuebingen.de/en/180804 2. Kevin P. Murphy, Probabilistic Machine Learning: Advanced Topics, MIT Press, 2022, URL = https://probml.github.io/pml-book/book2.html 3. Up to date AI papers
Homework: 30% Project: 50% Attendance: 20% Ex1. Basic Bayesian inference. (Coding) Ex2. Gaussian linear regression (Coding) Ex3. Gaussian process regression (Coding) Ex4. Integrating Gaussian process classifier and deep neural networks. (Coding) Ex5. GPs help us train large models with fewer expensive experiments. (Coding and Report) Ex6. Uncertainty quantification vis attention chain (Coding and report) Ex7. Response Uncertainty and Probe Modeling: Two Sides of the Same Coin in LLM Interpretability (Coding and Report) Ex8. Linear transformer (Coding and Report)
| 週次 | 主題 |
|---|---|
| 第 1 週 | Course outline |
| 第 2 週 | Reasoning under uncertainty Bayesian theorem |
| 第 3 週 | Monte Carlo sampling Data Generation Concepts |
| 第 4 週 | Bayesian inference, Gaussian distribution and Gaussian process |
| 第 5 週 | Understanding kernels and similarity metrics |
| 第 6 週 | Application: GPs help us train large models with fewer expensive experiments. |
| 第 7 週 | Gaussian process classification |
| 第 8 週 | Generalized linear model and Exponential families |
| 第 9 週 | Towards Bayesian Neural network: Last-layer Laplace approximation vs. deterministic uncertainty estimation |
| 第 10 週 | Review and Report |
| 第 11 週 | Probabilistic classifier integrating deep neural network and Gaussian process |
| 第 12 週 | uncertainty quantification vis attention chain |
| 第 13 週 | Response Uncertainty and Probe Modeling: Two Sides of the Same Coin in LLM Interpretability |
| 第 14 週 | Linear Transformer and delta Rules |
| 第 15 週 | Low rank computation: LoRA and Galore |
| 第 16 週 | Term project report |
No textbook for this course. Pioneering and contemporary papers will be discussed in the class.
- 地點
- in the class
- 時間
- By appointment or after class every week.
- 聯絡方式
- machingwen@nycu.edu.tw