最佳化演算法
Optimization Algorithms
| 節 | 週一 |
|---|---|
N 12:20–13:10 | 最佳化演算法 ED202 3 節連堂 |
5 13:20–14:10 | |
6 14:20–15:10 |
* 根據陽明交大上課時間表所列
This graduate-level course is designed to introduce a variety of optimization methods for solving engineering problems, such as machine learning, data networks, communication, and other related areas of computer science. We will cover topics from both convex and non-convex optimization, especially for the needs of the problems and applications in machine learning and net-works. The learning objectives can be summarized as follows. 1. Learn how to formulate real engineering problems as optimization problems 2. Learn how to systematically solve optimization problems by leveraging var-ious algorithms and perform convergence analysis of these algorithms 3. Learn how to implement optimization algorithms in a high-level language 4. Learn how to solve optimization problems using optimization solvers (e.g., Gurobi) 5. Learn how to conduct original research and write a technical report
• Undergraduate-level understanding of calculus, probability, and linear algebra • Basic programming skills in a high-level language (e.g., Python or MATLAB)
Homework assignments: 60% (including math problems and programming tasks) Final Project: 40%
| 週次 | 主題 |
|---|---|
| 第 1 週 | Fundamentals: Convex sets, convex functions, conjugates, subdifferentials |
| 第 2 週 | Fundamentals: Weak and strong duality, optimality conditions, and KKT conditions |
| 第 3 週 | Fundamentals: Conditions of nonconvex optimality and stationarity |
| 第 4 週 | First-order methods: Gradient descent and SGD |
| 第 5 週 | First-order methods: Subgradient and accelerated gradient |
| 第 6 週 | First-order methods: Variance reduction for stochastic optimization |
| 第 7 週 | First-order methods for constrained optimization: Frank-Wolfe methods |
| 第 8 週 | First-order methods for constrained optimization: Projected subgradient methods |
| 第 9 週 | Mirror descent (I) |
| 第 10 週 | Mirror descent (II) |
| 第 11 週 | Proximal gradient methods |
| 第 12 週 | Dual and primal-dual methods |
| 第 13 週 | Alternating direction method of multipliers (ADMM) |
| 第 14 週 | Neural network optimization (I) |
| 第 15 週 | Neural network optimization (II) |
| 第 16 週 | Final Presentation |
The course material will be adapted from the following textbooks: • Yurii Nesterov, Lectures on Convex Optimization, Springer, 2nd edition, 2018. • Amir Beck, Introduction to Nonlinear Optimization: Theory, algorithms, and applications with MATLAB, Society for Industrial and Applied Mathematics, 2014. • Leon Bottou, Frank Curtis, and Jorge Nocedal, Optimization Methods for Large-Scale Machine Learning, 2018. • Dimitri Bertsekas, Nonlinear Programming, Athena Scientific, 2nd edition, 1999.
- 地點
- EC713
- 時間
- TBD
- 聯絡方式
- Email: pinghsieh@nycu.edu.tw