Abstract : This short course will give a broad survey of information theory topics, starting from the simplest point-to-point communication networks, working towards small multi-terminal networks like multiple access and broadcast systems, and then discussing methods for generalizing information theoretic tools to derive results for very large network systems. Topics covered will include capacities, source coding bounds, and unifying themes in the tools used to derive them.
Background: 1. Comfort with Probability
2. Elements of Information Theory, Cover & Thomas, 2nd Edition,
Chapters 2 (Entropy, Relative Entropy, and Mutual Information), 3
(Asymptotic Equipartition Property), 7 (Channel Capacity), 8
(Differential Entropy), 9 (Gaussian Channel).
Abstract : This course will cover some of the fundamental gradient based (or first order) algorithms and results in optimization theory, with a focus on provably efficient methods. The choice of gradient based methods is due to their wide usage in machine learning as well as other applications. We will use some classical machine learning problems as running examples through out the course to illustrate the performance of various algorithms presented.
Background: Basic knowledge of linear algebra (matrix norms, singular values etc.) and functional analysis (derivatives, norms and dual norms). Everything else that is required will be covered in the course.
Appendix A in Boyd and Vandenberghe's book, (see here), covers these basics.
| Time | June 25 | June 26 | June 27 | June 28 |
|---|---|---|---|---|
| 08:00 - 08:45 | Registration | |||
| 08:45 - 09:00 | Opening Remarks | |||
| 09:00 - 10:00 | Network Information Theory, Lecture 1 Notes | Network Information Theory, Lecture 3 Notes | Network Information Theory, Lecture 5 Notes | Network Information Theory, Lecture 7 |
| 10:20 - 11:20 | Network Information Theory, Lecture 2 Notes | Network Information Theory, Lecture 4 Notes | Network Information Theory, Lecture 6 Notes | Network Information Theory, Lecture 8 |
| 11:30 - 12:00 | Open Discussion | Open Discussion | Open Discussion | Open Discussion |
| 12:00 - 12:30 | Invited Talk : Dr. Arpan Chattopadhyay | Invited Talk : Dr. Pooja Vyavahare (Slides) | Invited Talk : Dr. Lakshmi Narasimhan Theagarajan | Invited Talk : Dr. Abhishek Sinha (Slides) |
| 12:30 - 14:00 | Student Poster Session and Lunch | Student Poster Session and Lunch | Lunch | Lunch |
| 14:00 - 15:00 | Optimization, Lecture 1 Notes | Optimization, Lecture 3 Notes | Optimization, Lecture 5 Notes | Optimization, Lecture 7 Notes |
| 15:20 - 16:20 | Optimization, Lecture 2 Notes | Optimization, Lecture 4 Notes | Optimization, Lecture 6 Notes | Optimization, Lecture 8 Notes |
| 16:30 - 17:00 | Open Discussion | Open Discussion | Open Discussion | Open Discussion |
June 25: Dr. Arpan Chattopadhyay (IIT Delhi)
Title:
Security against false data injection attack in cyber-physical
systems
Abstract:
Secure, remote estimation of a linear Gaussian process via
observations at multiple sensors is
considered. Such a framework is relevant to many cyberphysical systems
and internet-of-things applications. Sensors make sequential
measurements that are shared with a fusion center; the fusion center
applies a filtering algorithm to make its estimates. The challenge is
the presence of a few unknown malicious sensors which can inject
anomalous observations to skew the estimates at the fusion center. The
set of malicious sensors may be time-varying. The problems of malicious
sensor detection and secure estimation are considered. First, an
algorithm for secure estimation is proposed. The proposed estimation
scheme uses a novel filtering and learning algorithm, where an optimal
filter is learnt over time by using the sensor observations in order to
filter out malicious sensor observations while retaining other sensor
measurements. Next, a novel detector to detect injection attacks on an
unknown sensor subset is developed. Numerical results demonstrate up to
3 dB gain in the mean squared error and up to 75% higher attack
detection probability under a small false alarm rate constraint, against
a competing algorithm that requires additional side information.
June 26: Dr. Pooja Vyavahare (IIT Tirupati)
Title:
Distributed learning over dynamic networks with adversarial agents
Abstract:
In this talk, I will discuss the non-Bayesian learning in multi-agent networks when some agents are adversarial (or faulty). In the first half of the talk, I will present a distributed algorithm for non-Bayesian learning in this setting. Existing algorithms require that all non-faulty agents in the network should be able to achieve consensus in order to learn the true state. In the second half of the talk, I will present an outline of the analysis of the algorithm that works under the relaxed network condition which does not require network to achieve consensus. In the end, extension to time-varying networks will be discussed.
June 27: Dr. Lakshmi Narasimhan Theagarajan (IIT Palakkad)
Title:
Lossy index coding
Abstract:
Index coding is a distributed source coding problem where multiple terminals, with some side information, request certain subsets of data from a server. The goal of index code is to minimize the transmission rate in the downlink while ensuring that the terminals can recover the requested data using the encoded broadcast message from the server. We consider a generalized version of this problem by allowing side information and requests to be coded, and lossy recovery. Such a setup often arises in content-cached networks. The achievable rates in this setup can be analyzed using rate-distortion theory. We shall discuss the relation between this problem and the rate distortion problem studied by Chris Heegard and Toby Berger. Based on the techniques developed by Heegard and Berger, we shall obtain bounds on the minimum transmission rate in the index coding problem. Finally, we shall discuss an algorithm based on alternating projections to design a lossy index code with certain recovery guarantees.
June 28: Dr. Abhishek Sinha (IIT Madras)
Title:
Optimal Scheduling Algorithms for Minimizing the Age-of-Information for the Multi-User Erasure Channel
Abstract:
Age-of-Information (AoI) is a recently proposed metric for quantifying the freshness of information from the UE's perspective in a communication network. AoI denotes the time elapsed since a UE received its most recent packet. In the first half of the talk, we consider the problem of minimizing the average-AoI for a single-hop downlink scheduling problem with erasure channels. We derive a universal lower-bound and a 2-approximation scheduling policy for optimizing the average-AoI metric. For delay-sensitive applications, including real-time control of a cyber-physical system, or scheduling URLLC traffic in 5G, it is essential to have a more stringent uniform control on AoI across all devices. In the second half of the talk, we discuss an exactly optimal scheduling policy minimizing the peak-AoI of the UEs in the same setup. Our proof of optimality involves an explicit solution to the associated countable-state average-cost Bellman Equation, which might be of independent theoretical interest. We establish that the resulting age-process is positive recurrent under the optimal policy, and has an exponentially light tail, with the optimal large-deviation exponent. We also extend these two policies with UE-specific throughput constraints.
Part of the work appeared in INFOCOM 2018 (Best paper award) and RAWNET 2019. Joint work with Igor Kadota (MIT), Eytan Modiano (MIT), Arunabh Srivastava (IIT Madras) and Krishna Jagannathan (IIT Madras).