強化學習原理
Reinforcement Learning
| 節 | 週三 | 週五 |
|---|---|---|
3 10:10–11:00 | 強化學習原理 EC115 2 節連堂 | |
4 11:10–12:00 | ||
7 15:30–16:20 | 強化學習原理 EC115 |
* 根據陽明交大上課時間表所列
- Learn how to model tasks as RL problems. - Understand RL from a theoretical viewpoint - Learn how to systematically solve RL problems by using various RL algorithms and perform analysis of these algorithms - Learn how to implement deep RL algorithms using software packages (e.g. Tensorflow and Pytorch) through team projects
- Some math maturity: Familiarity with calculus and probability (basic understanding of numerical optimization would help) - Programming language: Python (familiarity with Tensorflow/Pytorch would help)
Homework: 40% Theory Project: 25% Team Implementation Project: 35% (including 15% for poster presentations)
| 週次 | 主題 |
|---|---|
| 第 1 週 | - Course Logistics - Markov Decision Process (MDP) |
| 第 2 週 | - Planning in MDPs - Bellman Equations - Value Iteration - Policy Iteration - Regularized MDPs |
| 第 3 週 | - Policy Optimization - Introduction to Optimization (Convexity, Smoothness, Gradient Descent, and Mirror Descent) - Policy Gradient (PG) |
| 第 4 週 | - Stochastic PG (REINFORCE, A2C, and Natural PG) - Variance Reduction |
| 第 5 週 | - Model-Free Prediction - Generalized Advantage Estimation |
| 第 6 週 | - Global Convergence of Policy Gradient - Global Convergence of Natural PG |
| 第 7 週 | - Value Function Approximation |
| 第 8 週 | - Deterministic PG, DDPG, TD3 - Off-Policy Learning via Deterministic and Stochastic Policy Gradients |
| 第 9 週 | - Trust Region Policy Optimization (TRPO) - Global Convergence of TRPO and PPO - Proximal Policy Optimization (PPO) |
| 第 10 週 | - Value-Based Methods and Stochastic Approximation - Sarsa, Expected Sarsa, Q-Learning, and Double Q-Learning |
| 第 11 週 | - Entropy-Regularized RL - Soft Q-learning - Soft Actor-Critic |
| 第 12 週 | - Offline RL - Upside-Down RL - Sequence-to-Sequence Modeling for RL |
| 第 13 週 | - Reinforcement Learning from Human Feedback (RLHF) - Preference-based RL |
| 第 14 週 | - Model-based RL - Model-predictive control (MPC) - TD-MPC |
| 第 15 週 | - Imitation Learning - Inverse Reinforcement Learning (GAIL, AIL, and IQ-Learn) |
| 第 16 週 | - Final Poster Presentations |
- Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, MIT Press, 2nd edition, 2018 - Alekh Agarwal, Nan Jiang, and Sham M. Kakade, Reinforcement Learning: Theory and Algorithms, 2020 - Nocedal, Jorge, and Stephen Wright. Numerical optimization. Springer Science & Business Media, 2006 - Léon Bottou, Frank E. Curtis, and Jorge Nocedal, Optimization Methods for Large-Scale Machine Learning. arXiv 2016 - Tor Lattimore and Csaba Szepesvari, Bandit Algorithms. 2019
- 地點
- EC418
- 時間
- 1pm-2pm on Tuesdays
- 聯絡方式
- By email: pinghsieh@nycu.edu.tw