[tune] Better Serialization for Server

[richardliaw]

Addresses #3488 by using cloudpickle serialization for configuraiton.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10650/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10692/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/10697/
Test PASSed.

[richardliaw]

Test failures unrelated.

[waldroje]

So I am still seeing a problem similar to #3488 ... using the example in the docs, and just changing agent to DQN, because that is what I'm familiar with, and adding checkpoints...

def on_episode_start(info):
    print(info.keys())  # -> "env", 'episode"
    episode = info["episode"]
    print("episode {} started".format(episode.episode_id))
    episode.user_data["pole_angles"] = []

def on_episode_step(info):
    episode = info["episode"]
    pole_angle = abs(episode.last_observation_for()[2])
    episode.user_data["pole_angles"].append(pole_angle)

def on_episode_end(info):
    episode = info["episode"]
    pole_angle = np.mean(episode.user_data["pole_angles"])
    print("episode {} ended with length {} and pole angles {}".format(
        episode.episode_id, episode.length, pole_angle))
    episode.custom_metrics["pole_angle"] = pole_angle

def on_train_result(info):
    pass

ray.init()
trials = tune.run_experiments({
    "test": {
        "env": "CartPole-v0",
        "run": "DQN",
        "stop": {
            "timesteps_total": 50000
        },        
        "config": {
            "callbacks": {
                "on_episode_start": tune.function(on_episode_start),
                "on_episode_step": tune.function(on_episode_step),
                "on_episode_end": tune.function(on_episode_end),
                "on_train_result": tune.function(on_train_result),
            },
        },
        "checkpoint_freq": 10,
    }
})

I am still getting errors....

2019-01-10 13:56:23,398 ERROR trial_runner.py:227 -- Trial Runner checkpointing failed.
Traceback (most recent call last):
  File "/home/svc-tai-dev/Python/ray/python/ray/tune/trial_runner.py", line 225, in step
    self.checkpoint()
  File "/home/svc-tai-dev/Python/ray/python/ray/tune/trial_runner.py", line 120, in checkpoint
    json.dump(runner_state, f, indent=2)
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/__init__.py", line 178, in dump
    for chunk in iterable:
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/encoder.py", line 429, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict
    yield from chunks
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/encoder.py", line 324, in _iterencode_list
    yield from chunks
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict
    yield from chunks
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict
    yield from chunks
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict
    yield from chunks
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict
    yield from chunks
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/encoder.py", line 436, in _iterencode
    o = _default(o)
  File "/home/svc-tai-dev/miniconda3/envs/algo_new/lib/python3.5/json/encoder.py", line 179, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <ray.tune.suggest.variant_generator.function object at 0x7f2e1216c160> is not JSON serializable

[richardliaw]

#3743 should fix this.