Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md

InfoMARL

Here we use information regularization to promote cooperation / competition via intention signalling / hiding in a multi-agent RL problem. The environment is a simple, two-goal grid world built in OpenAI Gym based on the example here. The first agent, Alice, has access to the goal, is parameterized with a tabular policy and value function, and is trained using REINFORCE, based on an implementation here. Alice's policy is regularized with the mutual information between goal and action (given state), I(goal; action | state). Depending on the sign of the information weighting, this regularization encourages her to either signal or hide her private information about the goal. The second agent, Bob, does not have access to the goal, but instead must infer it purely from observing the behavior of Alice. Thus, information regularization of Alice directly affects the success of Bob. In summary, information regularization allows Alice to train alone, but to be prepared for cooperation / competition with a friend / foe (Bob) introduced later. More detailed notes can be found here.

TODOS:

  • make richer episode visualization
  • use I(t) = sum of info up until t, so that agent prefers revealing info later
  • find a lossy case
  • learn friend/foe policies and optimize mixture parameter
  • learn pi(beta) and optimize beta
  • try discounting kl / entropy into future (like Distral paper); for high enough beta, Alice should try not to terminate episodes
  • under what conditions might Alice "overshoot" to signal?
  • under what conditions are I(traj;goal) and I(action;goal|state) approx equal?

About

using information theory to encourage agents to cooperate and compete

Resources

Releases

No releases published

Packages

No packages published

Languages

You can’t perform that action at this time.