Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Define serialization format for behavioral cloning datasets #5

Closed
cswinter opened this issue Nov 23, 2021 · 2 comments · Fixed by #175
Closed

Define serialization format for behavioral cloning datasets #5

cswinter opened this issue Nov 23, 2021 · 2 comments · Fixed by #175

Comments

@cswinter
Copy link
Collaborator

Define a serialization format for behavioral cloning datasets of entity-gym observation/action pairs and add helper methods to serialize/deserialize datasets. Probably using MessagePack.
@cswinter cswinter assigned cswinter and unassigned cswinter Nov 26, 2021
@cswinter
Copy link
Collaborator Author

cswinter commented Nov 26, 2021 •
edited

Partial progress in #29
Still missing:

  • serialize ActionSpace
  • serialize ObsSpace
  • Record full logprobs, not just logprob of selected action
  • Record chosen action
  • Capture returns rather than instantaneous rewards

@cswinter
Copy link
Collaborator Author

Some more progress in #147, we now record the full logits and chosen actions.

@cswinter cswinter mentioned this issue Feb 21, 2022
Merged
cswinter added a commit that referenced this issue Feb 22, 2022
@cswinter
Behavioral cloning (#175)
77765f7 
Adds a new `supervised.py` script to enn-ppo which trains a model from samples recorded by another policy. Also makes various improvements to the sample recorder:
- add `--eval-capture-samples`/`--eval-capture-logits` options to record samples/logits during eval to a file
- add `--eval-on-step-0` arg to enable/disable running eval on the first step
- add `--codecraft-only-opponent` to run an eval with only a loaded eval policy against itself (this is slightly hacky, I'm planning to remove all the CodeCraft-specific options later)
- include action and observation spaces when recording samples
- fix `RaggedBufferBool` getting deserialized to `None`
- misc fixes to the `SampleRecorder` and `Trace`

Resolves #5, #6, and #8.
cswinter added a commit to entity-neural-network/entity-gym that referenced this issue May 11, 2022
@cswinter
Behavioral cloning (#175)
844f9d8 
Adds a new `supervised.py` script to enn-ppo which trains a model from samples recorded by another policy. Also makes various improvements to the sample recorder:
- add `--eval-capture-samples`/`--eval-capture-logits` options to record samples/logits during eval to a file
- add `--eval-on-step-0` arg to enable/disable running eval on the first step
- add `--codecraft-only-opponent` to run an eval with only a loaded eval policy against itself (this is slightly hacky, I'm planning to remove all the CodeCraft-specific options later)
- include action and observation spaces when recording samples
- fix `RaggedBufferBool` getting deserialized to `None`
- misc fixes to the `SampleRecorder` and `Trace`

Resolves entity-neural-network/incubator#5, entity-neural-network/incubator#6, and entity-neural-network/incubator#8.
cswinter added a commit to entity-neural-network/rogue-net that referenced this issue May 11, 2022
@cswinter
Behavioral cloning (#175)
4cb89e8 
Adds a new `supervised.py` script to enn-ppo which trains a model from samples recorded by another policy. Also makes various improvements to the sample recorder:
- add `--eval-capture-samples`/`--eval-capture-logits` options to record samples/logits during eval to a file
- add `--eval-on-step-0` arg to enable/disable running eval on the first step
- add `--codecraft-only-opponent` to run an eval with only a loaded eval policy against itself (this is slightly hacky, I'm planning to remove all the CodeCraft-specific options later)
- include action and observation spaces when recording samples
- fix `RaggedBufferBool` getting deserialized to `None`
- misc fixes to the `SampleRecorder` and `Trace`

Resolves entity-neural-network/incubator#5, entity-neural-network/incubator#6, and entity-neural-network/incubator#8.
cswinter added a commit to entity-neural-network/enn-trainer that referenced this issue May 12, 2022
@cswinter
Behavioral cloning (#175)
253592e 
Adds a new `supervised.py` script to enn-ppo which trains a model from samples recorded by another policy. Also makes various improvements to the sample recorder:
- add `--eval-capture-samples`/`--eval-capture-logits` options to record samples/logits during eval to a file
- add `--eval-on-step-0` arg to enable/disable running eval on the first step
- add `--codecraft-only-opponent` to run an eval with only a loaded eval policy against itself (this is slightly hacky, I'm planning to remove all the CodeCraft-specific options later)
- include action and observation spaces when recording samples
- fix `RaggedBufferBool` getting deserialized to `None`
- misc fixes to the `SampleRecorder` and `Trace`

Resolves entity-neural-network/incubator#5, entity-neural-network/incubator#6, and entity-neural-network/incubator#8.
cswinter added a commit to entity-neural-network/enn-zoo that referenced this issue May 14, 2022
@cswinter
Behavioral cloning (#175)
6a5b24c 
Adds a new `supervised.py` script to enn-ppo which trains a model from samples recorded by another policy. Also makes various improvements to the sample recorder:
- add `--eval-capture-samples`/`--eval-capture-logits` options to record samples/logits during eval to a file
- add `--eval-on-step-0` arg to enable/disable running eval on the first step
- add `--codecraft-only-opponent` to run an eval with only a loaded eval policy against itself (this is slightly hacky, I'm planning to remove all the CodeCraft-specific options later)
- include action and observation spaces when recording samples
- fix `RaggedBufferBool` getting deserialized to `None`
- misc fixes to the `SampleRecorder` and `Trace`

Resolves entity-neural-network/incubator#5, entity-neural-network/incubator#6, and entity-neural-network/incubator#8.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Behavioral cloning entity-neural-network/incubator
1 participant
@cswinter