Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: UniZero: Generalized and Efficient Planning with Scalable World Models #216

Open
wants to merge 119 commits into
base: main
Choose a base branch
from
Open

WIP: UniZero: Generalized and Efficient Planning with Scalable World Models #216

wants to merge 119 commits into from

Conversation

puyuan1996
Copy link
Collaborator

@puyuan1996
feature(pu): add init version of gpt-based muzero
36f69d0
@puyuan1996
sync code
344f9f6
@puyuan1996
sync code
f922cfb
@puyuan1996
sync code
6e436c5
@puyuan1996
fix(pu): fix reward/value/policy kl loss
e0c2b95
@puyuan1996
fix(pu): fix kv_cache used in MCTS search method
fc913c9
@puyuan1996
Merge branch 'dev-transformer' of https://github.com/opendilab/LightZero
8779039 
 into dev-transformer
@puyuan1996
feature(pu): add muzero_gpt for atari, polish world_model.past_keys_v…
a20deba 
…alues_cache
@puyuan1996
polish(pu): polish slicer
641680c
@puyuan1996
poliah(pu): polish compute_slice
36adc43
@puyuan1996
fix(pu): fix init latent state, fix past_keys_values_cache in mcts se…
e39d421 
…arch
@puyuan1996
fix(pu): fix past_keys_values_cache in mcts search
80f2239
@puyuan1996
polish(pu): polish world_model for batch processing, work in progress
3cad2f2
@puyuan1996
feature(pu): dd test_slicer and test_slicer_time
12d4174
@puyuan1996
fix(pu): fix reward/value/policy loss in muzero_gpt
03dc66f
@puyuan1996
fix(pu): fix state_action_history bug in mcts_ctree for muzero_gpt
26b6255
@puyuan1996
fix(pu): fix self.past_keys_values_cache bug in mcts_ctree for muzero…
fd85f4d 
…_gpt
@puyuan1996
polish(pu): use precomputed slices
30e7882
@puyuan1996
feature(pu): add target_policy_entropy log
ff6419f
@puyuan1996
feature(pu): add train tokenizer related loss
19c673f
@puyuan1996
polish(pu): polish obs_token_loss_weight
91d2e63
@puyuan1996
polish(pu): use different sample data for training transformer and to…
18a5e7b 
…kenizer
@puyuan1996
fix(pu): use argmax to select the most likely obs_token in recurrent_…
8a6da1d 
…inference
@puyuan1996
sync code
20f7c4c
@puyuan1996
fix(pu): tokenizer adam optimizer use weight_decay=0, set lr=1e-4 for…
8a1c418 
… all, use batchsize 256 for tokenizer
@puyuan1996
fix(pu): calculate target value in init_inference using unrolling 5 s…
7be1c32 
…teps
@puyuan1996
fix(pu): fix init_inference in mcts root state
fa2263e
@puyuan1996
plish(pu): polish muzero_gpt config
1c1c745
@puyuan1996
polish(pu): tokenizer delay update
a22cc88
@puyuan1996
polish(pu): polish configs
6c21c0f
puyuan1996 and others added 21 commits March 22, 2024 14:45
@puyuan1996
polish(pu): polish memory_env eval config
c3031b0
@puyuan1996
ppolish(pu): polish memory_env config
950639a
@puyuan1996
polish(pu): polish xzero config
ec7fa19
@puyuan1996
fix(pu): memory_env use rgb_img observation (3,5,5), use cnn encoder …
aabaa0d 
…in unizero
@puyuan1996 @jiayilee65
sync code
37b9a76
@jiayilee65
Merge branch 'dev-xzero-memory-debug' of https://github.com/opendilab…
9ecd46a 
…/LightZero into dev-xzero
@puyuan1996
fix(pu): use full episode to train world model in memory_env
29fe1e0
@jiayilee65
Merge branch 'dev-xzero-memory-debug' of https://github.com/opendilab…
2bd1f80 
…/LightZero into dev-xzero
@jiayilee65
feature(pu): add temporal gamma discount option
ea59840
@jiayilee65
fix(pu): use self.past_keys_values_cache_init_infer and self.past_key…
cb824c0 
…s_values_cache_recurrent_infer for init_infer and recurrent_infer
@jiayilee65
polish(pu): polish world_model
9644bdf
@jiayilee65
fix(pu): use different context length for init_infer and recurrent_infer
d500c2c
@jiayilee65
fix(pu): self.past_keys_values_cache_recurrent_infer.clear() per search
906856b
@jiayilee65
feature(pu): add depth_in_search_path utils
4255702
@jiayilee65
fix(pu): always save latest kv_cache for latent_state to tackle POMDP…
f5a928e 
… state alias problem
@jiayilee65
fix(pu): fix pos_embedding in kv_cache
5e3bd98
@jiayilee65
polish(pu): polish world_model
c0ff6df
@jiayilee65
polish(pu):polish configs
a343458
@jiayilee65
fix(pu): fix loss_value bug
74356a0
@jiayilee65
polish(pu): polish memory env settings and config
5fa006c
@jiayilee65
fix(pu): use polished and bigeer encoder/decoder net for memory env
b7cd384
@puyuan1996 puyuan1996 added algorithm New algorithm research Research work in progress labels Apr 14, 2024
@puyuan1996 puyuan1996 mentioned this pull request Apr 14, 2024
Closed
@puyuan1996 puyuan1996 added discussion Discussion of a typical issue or concept enhancement New feature or request labels Apr 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
algorithm New algorithm discussion Discussion of a typical issue or concept enhancement New feature or request research Research work in progress
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@puyuan1996 @jiayilee65