消息理論
Information Theory
學期
106-2
學分
3
學分
當期課號
5066
永久課號
ECM5104
開課單位
電信工程研究所
授課教師
李冕
校區
光復
類別
選修
上課時間表
| 節 | 週五 |
|---|---|
2 09:00–09:50 | 消息理論 EDB07 3 節連堂 |
3 10:10–11:00 | |
4 11:10–12:00 |
* 根據陽明交大上課時間表所列
概述
The course covers the fundamentals of information theory, a mathematical theory developed to quantify the amount of information contained in a random source and study how such information can be manipulated. While studying these topics, we will also develop some basic data compression and data transmission algorithms.
先修科目
Probability(UEE2102) Linear Algebra(UEE1061)
教學方式
English medium course
評分方式
40% Final Exam 20% Midterm Exam 20% Homework 20%Class participation Bonus Gym participation (just kidding)
週次計畫
| 週次 | 主題 |
|---|---|
| 第 1 週 | Introduction |
| 第 2 週 | mutual information, Jensen's inequality; data compression: efficient coding of a single RV, trees, leaf-counting and leaf-depth lemma, Kraft inequality Trees with probabilities, path length lemma, leaf-entropy theorem, converse part of coding theorem for a single random message |
| 第 3 週 | Shannon-type codes, Shannon codes, Fano codes, binary Huffman codes, D-ary Huffman codes , Types of codes: McMillan's theorem |
| 第 4 週 | coding of an information source: block–to–variable-length coding, rithmetic coding & decoding, variable-length–to–block coding |
| 第 5 週 | Variable-length–to–block coding, Tunstall coding, performance bounds on Tunstall coding; stochastic processes, stationarity, Markovity, Entropy rate, Cesaro mean |
教科書
Information Theory Lecture Notes from Stefan M. Moser
Office Hours
- 地點
- ED716A
- 時間
- Tuesday and Thursday 1520~1620
- 聯絡方式
- stefano@nctu.edu.tw