[RLlib] Restored experiments params.json incorrectly serialized

[ahmedammar]

What happened + What you expected to happen

When restoring an experiment using tune.Experiment() the created params.json contains the non-serialized AlgoConfig, for example:

❯ cat params.json
"<ray.rllib.algorithms.ppo.ppo.PPOConfig object at 0xfffed410ed70>"

Seems like this object is not being serialized to json, instead the object reference is being dumped to params.json file.

params.pkl is fine though, but just reporting this as I noticed it.

Versions / Dependencies

Ray: 2.8.0
Python: 3.10.13
OS: Linux

Reproduction script

experiment_spec = tune.Experiment(
    name=None,
    run="PPO",
    checkpoint_config=ray.air.CheckpointConfig(
        checkpoint_frequency=10,
        checkpoint_score_attribute="episode_reward_mean",
        num_to_keep=20,
    ),
    config=config,
    restore=$SOME_PREV_CHECKPOINTG
)

results = tune.run_experiments(
    experiments=experiment_spec,
    verbose=1,
) 

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[simonsays1980]

@ahmedammar thanks for posting this. I have created a PR that should fix this issue. The reason was that our AlgorithmConfig is not a dicitonary and therefore its method to_dict() has to be called in the JsonLogger.