[RLlib] TF2 Bandit Agent #22838
Merged
[RLlib] TF2 Bandit Agent #22838
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
March 5, 2022 13:08
sven1977
reviewed
Mar 15, 2022
| tf1, tf, tfv = try_import_tf() | ||
|
|
||
|
|
||
| class OnlineLinearRegression(tf.Module): |
There was a problem hiding this comment.
tf.Module if tf is not None else object
(in case tf is not installed)
There was a problem hiding this comment.
if tf is not installed, I feel like we should just error whenever possible?
we need this module to be tf.Module so the weights of all the arms can be checkpointed actually.
sven1977
reviewed
Mar 15, 2022
| @@ -32,7 +32,12 @@ def plot_model_weights(means, covs): | |||
| if __name__ == "__main__": | |||
| num_iter = 10 | |||
| print("Running training for %s time steps" % num_iter) | |||
| trainer = BanditLinTSTrainer(env=WheelBanditEnv) | |||
| config = { | |||
There was a problem hiding this comment.
Could we make this a command line arg, like in most other example scripts?
--framework=[tf|tf2|torch] and --eager-tracing.
You can copy paste this from any other example script, e.g. rllib/examples/attention_nets.py
sven1977
approved these changes
Mar 15, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
TF2 version of Bandit.
I also think these Bandit models need to save value outputs after applying UCB updates. Otherwise the UCB exploration won't really work.
Please help double check.
Related issue number
Checks
scripts/format.shto lint the changes in this PR.