_allow_variable_length_episodes is not an Attribute for CircuittrainingPPOLearner

[mrinal18]

Hello,

I am trying to run the run Quick Start commands, and after setting everything up on gcloud, when i try running the training script, i am getting the following error:

Attribute errror

I0418 03:03:54.281682 140093922326336 environment.py:240] ***Num node to place***:133
I0418 03:03:54.305619 140093922326336 train_ppo_lib.py:104] Using GRL agent networks.
I0418 03:04:32.939774 140093922326336 common.py:1007] No checkpoint available at ./logs/run_00/111/train/checkpoints
Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/dragonballz180796/circuit-training/circuit_training/learning/train_ppo.py", line 130, in <module>
    multiprocessing.handle_main(functools.partial(app.run, main))
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tf_agents/system/default/multiprocessing_core.py", line 78, in handle_main
    return app.run(parent_main_fn, *args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.9/dist-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "/usr/local/lib/python3.9/dist-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.9/dist-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "/home/dragonballz180796/circuit-training/circuit_training/learning/train_ppo.py", line 108, in main
    train_ppo_lib.train(
  File "/home/dragonballz180796/circuit-training/circuit_training/learning/train_ppo_lib.py", line 194, in train
    learner = learner_lib.CircuittrainingPPOLearner(
  File "/home/dragonballz180796/circuit-training/circuit_training/learning/learner.py", line 154, in __init__
    self._create_datasets(strategy)
  File "/home/dragonballz180796/circuit-training/circuit_training/learning/learner.py", line 236, in _create_datasets
    self._train_dataset = make_dataset(0)
  File "/home/dragonballz180796/circuit-training/circuit_training/learning/learner.py", line 228, in make_dataset
    return tf.data.experimental.Counter().flat_map(_make_dataset)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2092, in flat_map
    return FlatMapDataset(self, map_func, name=name)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 5327, in __init__
    self._map_func = structured_function.StructuredFunctionWrapper(
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/structured_function.py", line 271, in __init__
    self._function = fn_factory()
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 2567, in get_concrete_function
    graph_function = self._get_concrete_function_garbage_collected(
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 2533, in _get_concrete_function_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 2711, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 2627, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py", line 1141, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/structured_function.py", line 248, in wrapped_fn
    ret = wrapper_helper(*args)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/structured_function.py", line 177, in wrapper_helper
    ret = autograph.tf_convert(self._func, ag_ctx)(*nested_args)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 692, in wrapper
    raise e.ag_error_metadata.to_exception(e)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 689, in wrapper
    return converted_call(f, args, kwargs, options=options)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call
    result = converted_f(*effective_args, **kwargs)
  File "/tmp/__autograph_generated_file2z08qsdk.py", line 13, in tf___make_dataset
    train_dataset = ag__.converted_call(ag__.ld(train_dataset).filter, (ag__.ld(_filter_invalid_episodes),), None, fscope)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 377, in converted_call
    return _call_unconverted(f, args, kwargs, options)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 459, in _call_unconverted
    return f(*args)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 3946, in filter
    return DatasetV1Adapter(super(DatasetV1, self).filter(predicate, name=name))
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2246, in filter
    return FilterDataset(self, predicate, name=name)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 5477, in __init__
    wrapped_func = structured_function.StructuredFunctionWrapper(
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/structured_function.py", line 271, in __init__
    self._function = fn_factory()
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 2567, in get_concrete_function
    graph_function = self._get_concrete_function_garbage_collected(
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 2533, in _get_concrete_function_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 2711, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 2627, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py", line 1141, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/structured_function.py", line 248, in wrapped_fn
    ret = wrapper_helper(*args)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/data/ops/structured_function.py", line 177, in wrapper_helper
    ret = autograph.tf_convert(self._func, ag_ctx)(*nested_args)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 692, in wrapper
    raise e.ag_error_metadata.to_exception(e)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 689, in wrapper
    return converted_call(f, args, kwargs, options=options)
  File "/home/dragonballz180796/.local/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call
    result = converted_f(*effective_args, **kwargs)
  File "/tmp/__autograph_generated_filetax59fu6.py", line 40, in tf___filter_invalid_episodes
    ag__.if_stmt(ag__.ld(self)._allow_variable_length_episodes, if_body, else_body, get_state, set_state, ('do_return', 'retval_'), 2)
AttributeError: in user code:

    File "/home/dragonballz180796/circuit-training/circuit_training/learning/learner.py", line 184, in _make_dataset  *
        train_dataset = train_dataset.filter(_filter_invalid_episodes)
    File "/home/dragonballz180796/circuit-training/circuit_training/learning/learner.py", line 168, in _filter_invalid_episodes  *
        if self._allow_variable_length_episodes:

    AttributeError: 'CircuittrainingPPOLearner' object has no attribute '_allow_variable_length_episodes'

I can see that the code is not able to fetch information within this internal function : https://github.com/google-research/circuit_training/blob/main/circuit_training/learning/learner.py#L168

Command used:

python3 -m circuit_training.learning.train_ppo --root_dir=${ROOT_DIR} --replay_buffer_server_address=${REVERB_SERVER} --variable_container_server_address=${REVERB_SERVER} --num_episodes_per_iteration=16 --global_batch_size=64 --netlist_file=${NETLIST_FILE} --init_placement=${INIT_PLACEMENT}

I will try to debug from myside, but i wanted to ask if there's a quick fix too.

Thanks,
Mrinal

[braamBeresford]

Can confirm this bug as well. Commenting the relevant 'if' statements containing '_allow_variable_length_episodes' is a quick fix.