資訊理論與壓縮編碼的應用(英文授課)

In the near future, big data analysis will change our daily life significantly. However, in practice, it is possible that several megabytes of data may contain only a few bytes of useful information. Therefore, it is essential to understand the "nature" of information, and how to derive a mathematical model such that the amount of information in a data set can be quantified for analysis. This course is based on the classical Information Theory proposed by C. E. Shannon, with special focus on its application to data compression. Practical algorithms for different types of data compression will be presented. Finally, two case studies on data compression, one for deep-learning neural network models and the other one for audio data, will be investigated.

Linear Algebra, Probability

The E3 website will be used to host all class materials.

Performance evaluation is based on textbook reading assignments, a midterm exam, and a final programming project & report.

• 2. Entropy, relative entropy and mutual Information
• 3. Source coding:from Shanon to Kolmogorov
• 4. Mathematical background on lossless data compression
• 5. Huffman coding, arithmetic coding, and dictionary techniques
• 6. Context-based compression
• 7. Mathematical background on lossy data compression
• 8. Quantization and rate-distortion theory
• 9. Transform-domain and sub-band coding
• 10. Case Study I: Deep-learning neural network model coding
• 11. Case Study II: Audio coding
• 1. Introduction to information theory

1. Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, 2nd Ed., John Wiley & Sons, Inc., 2006。 2. Khalid Sayood, Introduction to Data Compression, Third Edition, Morgan Kaufmann, 2005. Note that both textbooks are available in ebook format in the NCTU university library. These are not the latest editions of the textbooks from the publishers.

地點

EC718

聯絡方式

cjtsai@cs.nctu.edu.tw (03) 573-1628 On campus extension number: 31628