[RLlib] Create a set of performance benchmark tests to run nightly. #19945
Conversation
gjoliver
force-pushed
the
rllib-nightly
branch
from
355721b
to
16aa3ad
Compare
| actor_learning_rate: 0.0003 | ||
| critic_learning_rate: 0.0003 | ||
| entropy_learning_rate: 0.0003 | ||
| num_workers: 0 |
There was a problem hiding this comment.
why do we not use any workers for SAC halfcheetah?
|
Hi Sven, I'd love your opinion about the set of tests I want to run nightly. |
gjoliver
force-pushed
the
rllib-nightly
branch
3 times, most recently
from
0c0ca1c
to
b51e5fa
Compare
gjoliver
changed the title
| @@ -0,0 +1,131 @@ | |||
| apex-breakoutnoframeskip-v4: | |||
There was a problem hiding this comment.
Could we sort these by alphabet?
| desired_throughput = None | ||
| # TODO(Jun): Stop checking throughput for now. | ||
| # desired_throughput = checks[experiment]["min_throughput"] | ||
| desired_throughput = checks[experiment]["min_throughput"] |
There was a problem hiding this comment.
Maybe for a later PR: Should we measure env-throughput and learner-throughput separately?
rllib/utils/test_utils.py
Outdated
| else: | ||
| keys.append(re.sub("^(\\w+)-", "\\1-torch-", k)) | ||
| experiments[keys[0]] = e | ||
| # Generate `checks` dict for all experiments (tf, tf2 and/or torch). |
There was a problem hiding this comment.
Just a question: This means we'll include tf2 in our weekly learning tests as well as our nightly multi-gpu tests, correct? I think the multi-gpu one would fail since RLlib+tf2 does not support multi-gpu yet. Could you check this? We should probably have a way to specify, which frameworks to test in the yaml files.
Like:
...
config:
frameworks: [tf, torch] # instead of "framework": expands `experiments_to_run`, then removes the "frameworks" key from the struct
0675d20
to
c0aeb70
Compare
|
all done. thanks for the thoughtful review. |
Why are these changes needed?
Track performances of the most important algorithms nightly.
The way these tests are set up is that we run them for a fixed amount of time every day without passing criteria.
We will then record the track the average reward achieved and throughput over time.
Run all of the RLlib nightly and weekly tests in tf2 framework too.
Write performance metrics in the output json file.
Related issue number
Checks
scripts/format.shto lint the changes in this PR.