遞迴神經網路
Recurrent Neural Networks
| 節 | 週二 |
|---|---|
5 13:20–14:10 | 遞迴神經網路 CM214 3 節連堂 |
6 14:20–15:10 | |
7 15:30–16:20 |
* 根據陽明交大上課時間表所列
This course introduces Recurrent Neural Networks (RNNs). RNNs are a variant of conventional feedforward neural networks that can deal with sequential data and can be trained to hold knowledge about the past. They are well-known for modeling sequential data by learning temporal dependencies via recurrent connections. They have been widely applied to many applications, such as speech recognition, machine translation, generating text, and chatbots. In this course, we will introduce basic RNN, gated RNN (LSTM, GRU, and MGU), and some advanced RNNs such as structured RNN and temporal pyramid RNN. Besides, current research on Nature Language Processing has been changed from RNN to Transformers; thus, we will also cover the recent development of Transformers and Bidirectional Encoder Representations from Transformers (BERT). The course will include lectures, small projects (implementation), paper presentations, and a final project.
None
Instruction and project-based learning. Course material is available on E3 learning platform.
Programming homework.: 60% (4x15%) Paper presentation: 10% Final project: 30%
| 週次 | 主題 |
|---|---|
| 第 1 週 | Course Introduction; RNN: Network Architectures |
| 第 2 週 | RNN: Learning Processes |
| 第 3 週 | RNN: Recurrent Neural Networks |
| 第 4 週 | RNN: Recurrent Neural Networks |
| 第 5 週 | RNN: LSTM |
| 第 6 週 | RNN: LSTM |
| 第 7 週 | RNN: Gated Recurrent Unit (GRU) |
| 第 8 週 | RNN: Minimal Gated Unit (MGU) |
| 第 9 週 | Transformer |
| 第 10 週 | BERT |
| 第 11 週 | Pretraining a RoBERTa Model from Scratch |
| 第 12 週 | Downstream NLP Tasks with Transformers |
| 第 13 週 | Text Generation with OpenAI GPT-2 and GPT-3 Models |
| 第 14 週 | Vision Transformer (ViT), BERT Pre-Training of Image Transformers (BEiT), End-to-End Object Detection with Transformers (DETR), CF-DERT, etc. |
| 第 15 週 | Paper presentation |
| 第 16 週 | Paper presentation, final project |
| 第 17 週 | |
| 第 18 週 |
1. Fathi M. Salem, Recurrent Neural Networks, Springer, 2022. ISBN 978-3-030-89928-8 2. Denis Rothman, Transformers for Natural Language Processing, Packt Publishing, Jan. 2021. ISBN: 9781800565791
- 地點
- ChiMei 303
- 時間
- Tuesday 10:00-12:00AM
- 聯絡方式
- 55729