*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************
Title: Cooperation in Multi-Agent Reinforcement Learning
Date: November 30th, Tuesday, 2021
Time: 7:00-9:00 PM Eastern Time (4:00-6:00 PM Pacific Time)
Location: https://bluejeans.com/773753749/7843
Jiachen Yang
Machine Learning PhD Candidate
School of Computational Science and Engineering
Georgia Institute of Technology
Committee
1. Dr. Hongyuan Zha (Advisor), School of Computational Science and Engineering, Georgia Institute of Technology | Executive Dean of School of Data Science, Chinese University of Hong Kong, Shenzhen
2. Dr. Tuo Zhao (Co-Advisor), School of Industrial and Systems Engineering, Georgia Institute of Technology
3. Dr. Charles Isbell, Dean of College of Computing, School of Interactive Computing, Georgia Institute of Technology
4. Dr. Matthew Gombolay, School of Interactive Computing, Georgia Institute of Technology
5. Dr. Daniel Faissol, Computational Engineering Division, Lawrence Livermore National Laboratory
Abstract
As progress in reinforcement learning (RL) gives rise to increasingly general and powerful artificial intelligence, society needs to anticipate a possible future in which multiple RL agents learn and interact in a shared multi-agent environment. When a single principal has oversight of the multi-agent system, how should agents learn to cooperate via centralized training to achieve individual and global objectives? When agents belong to self-interested principals with imperfectly aligned objectives, how can cooperation emerge from fully-decentralized learning? This dissertation addresses both questions by proposing novel methods for multi-agent reinforcement learning (MARL) and demonstrating the empirical effectiveness of these methods in high-dimensional simulated environments.
To address the first case, we propose new algorithms for fully-cooperative MARL in the paradigm of centralized training with decentralized execution. Firstly, we propose a method based on multi-agent curriculum learning and multi-agent credit assignment to address the setting where global optimality is defined as the attainment of all individual goals. Secondly, we propose a hierarchical MARL algorithm to discover and learn interpretable and useful skills for a multi-agent team to optimize a single team objective. Extensive experiments with ablations show the strengths of our approaches over state-of-the-art baselines.
To address the second case, we propose learning algorithms to attain cooperation within a population of self-interested RL agents. We propose the design of a new agent who is equipped with the new ability to incentivize other RL agents and explicitly account for the other agents' learning process. This agent overcomes the challenging limitation of fully-decentralized training and generates emergent cooperation in difficult social dilemmas. Then, we extend and apply this technique to the problem of incentive design, where a central incentive designer explicitly optimizes a global objective only by intervening on the rewards of a population of independent RL agents. Experiments on the problem of optimal taxation in a simulated market economy demonstrate the effectiveness of this approach.
