Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Example] Multiagent examples: MAPPO-IPPO-MADDPG-IDDPG-IQL-QMIX-VDN #1027

Merged
merged 177 commits into from
Jul 25, 2023
Merged

[Example] Multiagent examples: MAPPO-IPPO-MADDPG-IDDPG-IQL-QMIX-VDN #1027

merged 177 commits into from
Jul 25, 2023

Conversation

matteobettini
Copy link
Contributor

@matteobettini matteobettini commented Apr 6, 2023 •
edited
Loading

Example of multiagent training algos in vmas

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 6, 2023
@matteobettini matteobettini marked this pull request as draft April 6, 2023 08:26
@matteobettini matteobettini changed the title [Example] mapp ippo example [Example] MAPPO IPPO example Apr 6, 2023
@matteobettini matteobettini mentioned this pull request Apr 6, 2023
Merged
@matteobettini matteobettini changed the title [Example] MAPPO IPPO example [Example] Multiagent examples: MAPPO-IPPO-MADDPG-IDDPG Apr 17, 2023
This was referenced Apr 17, 2023
Closed
Closed
@matteobettini matteobettini mentioned this pull request Apr 19, 2023
Closed
@matteobettini
init
ee77a11 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
Copy link
Contributor Author

matteobettini commented Jul 11, 2023 •
edited
Loading

I have addressed the comments in the examples, when you can have a look.

I moved the models and loss to #1378 so we can merge them first

@matteobettini
tests
42e6bb2 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
docs
2df23a4 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
Merge branch 'main' into marl_losses_modules
e4773ad
@matteobettini
import
010bccf 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
amend
e678547 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
Merge branch 'marl_losses_modules' into mappo_ippo
dd0732a 
# Conflicts:
#	torchrl/objectives/__init__.py
#	torchrl/objectives/multiagent/qmixer.py
@matteobettini
docs mlp
c4561a2 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
docs
3725198 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
docs
f6047bf 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
docs
b0b4d22 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
Merge branch 'marl_losses_modules' into mappo_ippo
298a15b
@matteobettini
Merge branch 'main' into mappo_ippo
ea3ed53 
# Conflicts:
#	test/test_cost.py
#	torchrl/modules/models/multiagent.py
#	torchrl/objectives/dqn.py
#	torchrl/objectives/multiagent/qmixer.py
@matteobettini matteobettini changed the title [Example] Multiagent losses, models, and examples: MAPPO-IPPO-MADDPG-IDDPG-IQL-QMIX-VDN [Example] Multiagent examples: MAPPO-IPPO-MADDPG-IDDPG-IQL-QMIX-VDN Jul 14, 2023
@matteobettini
amend
c1f487f 
Signed-off-by: Matteo Bettini <matbet@meta.com>
matteobettini
matteobettini commented Jul 14, 2023
View reviewed changes
examples/media/marl_vmas.png Outdated
Copy link
Contributor Author

@matteobettini matteobettini Jul 14, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend not uploading media to this repo.

The repo size is already huge and this can impact user with limited disk space.

I would recommend creating a secundary repo/s3 bucket/ some other hosting service to host the media and just embed links to them

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, this isn't scalable on the long term.
We need to make sure that whatever we link will remain on the long term
pytorch/tutorials and pytorch/pytorch (as well as vision I think) are pretty big though
Let me see how we're supposed to handle things

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i dunno if pytorch has any hosting service or repo, but those would be cool.

It is your call anyway, I was just pointing out that git will not only save the media, but also all their previous versions, and it will all be downloaded upon clone

@matteobettini
amend
f9430b3 
Signed-off-by: Matteo Bettini <matbet@meta.com>
vmoens
vmoens reviewed Jul 14, 2023
View reviewed changes
Copy link
Contributor

@vmoens vmoens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we must add these examples to the example CI
They should not take more than a minute each to run (ie very small nets and very few iterations). We just want to make sure they run ok
I'll what we can do regarding the images and s3 bucket

examples/media/marl_vmas.png Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, since this file is likely to be modified in the future, to not make the git history too big we should host it on our s3 bucket

examples/multiagent/iql.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine by me :)

@matteobettini
Merge branch 'main' into mappo_ippo
a651317
@matteobettini
examples ci
2d35754 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
Copy link
Contributor Author

The examples are now in the CI, if we sort out the s3 we are ready to go

@matteobettini
typo
b2828ca 
Signed-off-by: Matteo Bettini <matbet@meta.com>
vmoens
vmoens approved these changes Jul 25, 2023
View reviewed changes
Copy link
Contributor

@vmoens vmoens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
The figure can be found here

@matteobettini
media
c5d9c0d 
Signed-off-by: Matteo Bettini <matbet@meta.com>
@matteobettini
Copy link
Contributor Author

LGTM The figure can be found here

updated

@vmoens vmoens merged commit 40a0929 into pytorch:main Jul 25, 2023
@matteobettini matteobettini deleted the mappo_ippo branch July 27, 2023 08:04
vmoens pushed a commit to hyerra/rl that referenced this pull request Oct 10, 2023
@matteobettini @vmoens
[Example] Multiagent examples: MAPPO-IPPO-MADDPG-IDDPG-IQL-QMIX-VDN (p…
4149579 
…ytorch#1027)

Signed-off-by: Matteo Bettini <matbet@meta.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vmoens vmoens vmoens approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@matteobettini @vmoens @facebook-github-bot