| Invited Talk

Name of the Speaker: Dr. Vijay Subramanian
Name of the Organizer: Prof. Srikrishna Bhashyam
Venue: ESB-244 (Seminar Hall)
Date/Time: 3rd January 2024 (Wednesday), 2:30 PM
Title: Cooperative Multi-Agent Constrained POMDPs: Strong Duality and Primal-Dual Reinforcement Learning with Approximate Information States

Abstract

We study the problem of decentralized constrained POMDPs in a team-setting where the multiple non-strategic agents have asymmetric information. Strong duality is established for the setting of infinite-horizon expected total discounted costs when the observations lie in a countable space, the actions are chosen from a finite space, and the immediate cost functions are bounded. Following this, connections with the common-information and approximate information-state approaches are established. The approximate information-states are characterized independent of the Lagrange-multipliers vector (under certain assumptions) so that adaptations of the multiplier (during learning) will not necessitate new representations. Finally, a primal-dual multi-agent reinforcement learning (MARL) framework based on centralized training distributed execution (CTDE) and three time-scale stochastic approximation is developed with the aid of recurrent and feedforward neural-networks for function-approximation. As a part of this talk, some broader context on decentralized teams will also be provided. This is joint work with Nouman Khan at the University of Michigan, Ann Arbor (appeared in part in the proceedings of IEEE CDC 2023), and Hsu Kao when he was a Ph.D. student at the University of Michigan, Ann Arbor (appeared in the proceedings of AISTATS 2022).

Bio: Vijay Subramanian received the Ph.D. degree in electrical engineering from the University of Illinois at Urbana-Champaign, Champaign, IL, USA, in 1999. He worked at Motorola Inc., and at the Hamilton Institute, Maynooth, Ireland, for many years, and also in the EECS Department, Northwestern University, Evanston, IL, USA. In Fall 2014, he started in his current position as an Associate Professor with the EECS Department at the University of Michigan, Ann Arbor. For the academic year 2022-2023, he was an Adjunct Research Associate Professor in CSL and ECE at UIUC, and he continues to hold this position in academic year 2023-2024 as well. His research interests are in stochastic analysis, random graphs, multi-agent systems, and game theory (mechanism and information design) with applications to social, economic and technological networks.