邊緣人工智慧
Edge AI
| 節 | 週四 |
|---|---|
8 16:30–17:20 | 邊緣人工智慧 ED117 2 節連堂 |
9 17:30–18:20 |
* 根據陽明交大上課時間表所列
邊緣人工智慧：邊緣人~工作的智慧？邊緣人工~的智慧？邊緣的~人工智慧！ 不管怎麼解讀，概念都離不開 efficient (高效率) 這個詞，尤其對 CS 領域的邊緣人來說，能躺著幹就不坐起來，能坐著幹就不站起來，實為 efficient 一詞的極致表現。身為 CS 領域的(邊緣)人，如果你對人工智慧有興趣，但不想再調參建模了，此課必修！ ======== 當人工智慧越來越普及，越來越多的人工智慧服務將被去中心化，也就是邊緣化，邊緣化的人工智慧服務仰賴邊緣(終端)裝置獨立運作，可不仰賴網路連接至大型雲端伺服器或資料中心，更不須分享機敏資料。相較於集中式的、雲端的人工智慧服務，邊緣化的人工智慧服務(此後簡稱為邊緣人工智慧)有以下幾項優點： 1. 高隱私性、無機敏資料洩漏之疑慮 2. 無(低)網路需求、不受網路頻寬影響 3. 無(低)雲端伺服器或資料中心之需求 但天下沒有白吃的午餐，想當邊緣人，凡事就得自己來；想擁有邊緣人工智慧的優點，凡事只能靠在地的、非雲端的邊緣(終端)裝置。然而，終端裝置通常不具備大規模運算能力，沒有充沛的記憶體資源，且可能沒有穩定的電力來源甚至僅依賴電池供電。因此，邊緣人工智慧的運行，必須高效率，更精確地說，軟體(人工智慧之模型)要小巧但準確，硬體(運行人工智慧之裝置)要又快又省電，缺一不可。 這門課將探討一系列為了實現「邊緣人工智慧」的相關技術，詳細內容請看下面的進度列表。
1. Basic knowledge of neural networks 2. Python (required) and CUDA (preferred) 3. A fresh liver
Lab 0: Running MobileNet on the cloud (with GPU) and at the edge (without GPU) Lab 1: Compressing (Pruning/Sparsifying) MobileNet Lab 2: Running Vision Transformer and Quantizing Vision Transformer Lab 3: Optimizing Lab 2 with FlashAttention (and HQQ) Lab 4: Running a Large Language Model (LLM) and Compressing LLM's KV-Cache Lab 5: Optimizing Lab 4 with PagedAttention Lab 6: Accelerating LLM with speculative decoding Lab 7: LLM Challenge - Joe是要對決：奢華LLM vs. 平價LLM Lab assignments: 95% Participation: 5% Presentation: 5% (bonus) Exam: 0% (hopefully)
| 週次 | 主題 |
|---|---|
| 第 1 週 | Opening and introduction |
| 第 2 週 | Fundamentals of artificial intelligence (AI) and deep learning (DL) |
| 第 3 週 | Review of selected neural network (NN) models: MobileNet, Transformer, LLM, etc. |
| 第 4 週 | Pruning: unstructured vs. structured NN model compression Sparsity: semi-structured NN model compression |
| 第 5 週 | TinyML, ONNC |
| 第 6 週 | AI accelerators (I): general-purpose AI accelerators AI accelerators (II): TPU vs. LPU vs. NPU AI accelerators (III): AI ASICs |
| 第 7 週 | Quantization (I): post-training quantization Quantization (II): quantization-aware training Quantization (III): MXINT vs. MXFP vs. NVFP |
| 第 8 週 | Neural architecture search (I): search and train Neural architecture search (II): train and search |
| 第 9 週 | Wrap-up (halfway) What's next: LLM made faster and smaller (to be deployed at the edge) |
| 第 10 週 | FlashAttention and PagedAttention KV-Cache (compression and quantization) Google's TurboQuant vs. NVIDIA's KVTC |
| 第 11 週 | Speculative decoding, lookahead decoding, offloading Speculative decoding with approximate D/V |
| 第 12 週 | P/D disaggregation D/V disaggregation |
| 第 13 週 | Efficient fine-tuning: LoRA and DoRA Knowledge distillation and transfer learning |
| 第 14 週 | LLM re-architectured: Mixture-of-Experts, DeepSeek, Mamba |
| 第 15 週 | Distributed training, federated learning |
| 第 16 週 | On-device training, privacy-aware model customization/personalization |