Course Details

Course details of EE5142
Course NoEE5142
Course TitleIntroduction to Information Theory and Coding
Credit12
Course Content1) Entropy, Relative Entropy, and Mutual Information:Entropy, Joint Entropy and Conditional Entropy, Relative Entropy and Mutual Information, Chain Rules, Data-Processing Inequality, Fano’s Inequality 2) Typical Sequences and Asymptotic Equipartition Property:Asymptotic Equipartition Property Theorem, Consequences of the AEP:Data Compression, High-Probability Sets and the Typical Set 3) Source Coding and Data Compression:Kraft Inequality, Huffman Codes, Optimality of Huffman Codes 4) Channel Capacity:Symmetric Channels, Properties of Channel Capacity, Jointly Typical Sequences, Channel Coding Theorem, Fano’s Inequality and the Converse to the Coding Theorem 5) Differential Entropy and Gaussian Channel:Differential Entropy, AEP for Continuous Random Variables, Properties of Differential Entropy, Relative Entropy, and Mutual Information,Coding Theorem for Gaussian Channels 6) Linear Binary Block Codes:Introduction, Generator and Parity-Check Matrices, Repetition and Single-Parity-Check Codes, Binary Hamming Codes, Error Detection withLinear Block Codes, Weight Distribution and Minimum Hamming Distance of a Linear Block Code, Hard-decision and Soft-decision Decoding of Linear Block Codes, Cyclic Codes, Parameters of BCH and RS Codes,Interleaved and Concatenated Codes 7) Convolutional Codes:Encoder Realizations and Classifications, Minimal Encoders, Trellis representation, MLSD and the Viterbi Algorithm, Bit-wise MAP Decoding and the BCJR Algorithm
Course Offered this semesterNo
Faculty Name