Paper-Conference

The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Models

Jan 1, 2024
Laser Learning Environment: A new environment for coordination-critical multi-agent tasks

We introduce the Laser Learning Environment (LLE), acollaborative multi-agent reinforcement learning environment in whichcoordination is central. In LLE, agents depend on each other to makeprogress (interdependence), must jointly take specific sequences of actionsto succeed (perfect coordination), and accomplishing those joint actionsdoes not yield any intermediate reward (zero-incentive dynamics). Thechallenge of such problems lies in the difficulty of escaping state spacebottlenecks caused by interdependence steps since escaping those bot-tlenecks is not rewarded. We test multiple state-of-the-art value-basedMARL algorithms against LLE and show that they consistently fail atthe collaborative task because of their inability to escape state spacebottlenecks, even though they successfully achieve perfect coordination.We show that Q-learning extensions such as prioritised experience replayand n-steps return hinder exploration in environments with zero-incentivedynamics, and find that intrinsic curiosity with random network distil-lation is not sufficient to escape those bottlenecks. We demonstrate theneed for novel methods to solve this problem and the relevance of LLEas cooperative MARL benchmark.

Jan 1, 2023
Toward Evolutionary Autocurricula: Emergent Sociality from Inclusive Rewards

Jan 1, 2022
Local Advantage Networks for Cooperative Multi-Agent Reinforcement Learning

Jan 1, 2022