資訊理論與壓縮編碼的應用(英文授課)

In the near future, big data analysis will change our daily life significantly. However, in practice, it is possible that several megabytes of data may contain only a few bytes of useful information. Therefore, it is essential to understand the "nature" of information, and how to derive a mathematical model such that the amount of information in a data set can be quantified for analysis. This course is based on the classical information theory proposed by C. E. Shannon, with special focus on its application to data compression. Practical algorithms for different types of data compression will be presented. Finally, two case studies on data compression, one for deep-learning neural network models and the other one for audio data, will be investigated.

Linear Algebra, Probability

The E3 website will be used to host all class materials.

Performance evaluation is based on some programming assignments, a midterm exam, and a final project. You can use the language of your choice for the assignments.

• 1. Introduction to information theory
• 2. Information measures
• 3. Data compression and Huffman code.
• 4. Entropy Rates
• 5. Kolmogorov data complexity
• 6. Arithmetic coding and dictionary-based techniques
• 7. Context-based compression
• 8. Quantization and rate-distortion theory
• 9. Transform-domain and sub-band coding
• 10. Case Study I: Deep-learning neural network model coding
• 11. Case Study II: Audio coding

1. Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, 2nd Ed., John Wiley & Sons, Inc., 2006. 2. Khalid Sayood, Introduction to Data Compression, Third Edition, Morgan Kaufmann, 2005. Note that both textbooks are available in ebook format in the NCTU university library. They are not the latest editions of the textbooks from the publishers.

地點

EC718

聯絡方式

cjtsai@cs.nctu.edu.tw (03) 573-1628 On campus extension number: 31628