Feature/algo eval

[maxhuettenrauch]

Changes

Dependencies

• New extra "eval"

Api Extension

• Experiment and ExperimentConfig now have a name, that can however be overridden when Experiment.run() is called
• When building an Experiment from an ExperimentConfig, the user has the option to add info about seeds to the name.
• New method in ExperimentConfig called build_default_seeded_experiments
• SamplingConfig has an explicit training seed, test_seed is inferred.
• New evaluation package for repeating the same experiment with multiple seeds and aggregating the results (important extension!). Currently in alpha state.
• Loggers can now restore the logged data into python by using the new restore_logged_data

Breaking Changes

• AtariEnvFactory (in examples) now receives explicit train and test seeds
• EnvFactoryRegistered now requires an explicit test_seed
• BaseLogger.prepare_dict_for_logging is now abstract

[maxhuettenrauch]

@MischaPanch @bordeauxred please have a look

[MischaPanch]

Preliminary review, we can have a closer look together later,

One thing to do already is to separate saving plots from plotting, and/or give the user the possibility to configure how plots should be saved

[maxhuettenrauch]

I had to use contextlib.suppress in order to make the docs build, but at least tests are passing now. I also linked our fork of rliable in the project dependencies as long as the original does not update the dependency on arch.

[MischaPanch]

I also linked our fork of rliable in the project dependencies as long as the original does not update the dependency on arch.

Thanks, that works! Seems like rliable is no longer maintained, my PR bumping the arch version got no attention. Let's see for how long we don't need to touch it again. If it creates further problems, might be worth to move the functionality over to tianshou or somewhere else

[MischaPanch]

@maxhuettenrauch as to the contextlib suppression - I don't think that's the best solution. Instead you could adjust the command in the CI, which is currently poetry install --with dev to poetry install --with dev -E evaluation (or something like that).

Note that you can run github actions locally with this, maybe it makes sense to have a poe command for it

[maxhuettenrauch]

Thanks for the recommendation, unfortunately I haven't managed to get act to work (getting ::error::Not Found but no idea what is actually not found)

[MischaPanch]

Seems like this is ready, right? Gonna run a few tests tomorrow and then merge it

[MischaPanch]

I couldn't make parallel execution work with joblib. It also might not make too much sense on a single machine, so we can consider using ray for parallelization instead. For now the limitations are documented, so I'd merge this.

In my last commits I improved the plots a bit (axis labels were cut off), added more docs, fixed some installation problems and did minor enhancements to the interfaces