-
Notifications
You must be signed in to change notification settings - Fork 599
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add PPO + Transformer-XL #459
base: master
Are you sure you want to change the base?
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
pre-commitpre-commit fails because of two "obsolet" imports: memory_gym and PoMEnv. Without those imports, the environments are not registered inside gymnasium. enjoy.pyI added a script to load a trained model and then watch an episode. ProofofMemory-v0 and MiniGrid-MemoryS9-v0These environments require memory and converge pretty fast. That's why I included those initially. MemoryGym environments take in more time and resources (especially GPU memory due to the cached hidden states of Transformer-XL). TODOI still have to run the benchmarks and write documentation. Besides that, the single file implementation is basically done. I tried to stay close to ppo_atari_lstm.py |
…initial to final value.
…nused imports, however these imports are necessary for the used environments to be registered
…yperparameters for succesfull MMG training
Hey! This looks pretty impressive! Just curious, what is the state of this PR? |
Hi @roger-creus |
Nice! Looking forward to the results |
It reproduces the results of my paper: and this is the original implementation: |
I'm curious about how it performs in other environments (e.g. atari?) |
Description
Implementation of PPO with Transformer-XL as episodic memory.
Based on this repo and paper.
Types of changes
Checklist:
pre-commit run --all-files
passes (required).mkdocs serve
.If you need to run benchmark experiments for a performance-impacting changes:
--capture_video
.python -m openrlbenchmark.rlops
.python -m openrlbenchmark.rlops
utility to the documentation.python -m openrlbenchmark.rlops ....your_args... --report
, to the documentation.