Unable to evaluate with GPU in colab

[CrackerHax]

I am training dcgan with TPU on colab but when I try to evaluate with GPU I get this
TypeError: '<=' not supported between instances of 'int' and 'str'

This happens with both a fake and real dataset.

Command used:
!python compare_gan/main.py --gin_config example_configs/dcgan_celeba64.gin --data_fake_dataset true --model_dir 'gs://***/models' --tfds_data_dir 'gs://***/' --schedule=continuous_eval --eval_every_steps=0

Here's the tail of the log:
2019-06-03 05:03:32.856155: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0 2019-06-03 05:03:32.856235: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix: 2019-06-03 05:03:32.856252: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990] 0 2019-06-03 05:03:32.856264: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0: N 2019-06-03 05:03:32.856460: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14115 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5) I0603 05:03:49.514711 140045070399360 fid_score.py:54] Frechet Inception Distance: 433.464. I0603 05:03:49.515237 140045070399360 eval_gan_lib.py:209] Computed results for task <compare_gan.metrics.fid_score.FIDScoreTask object at 0x7f5e5e6a1eb8>: {'fid_score_mean': 433.68958, 'fid_score_std': 0.2974496, 'fid_score_list': '434.10986_433.4947_433.46417'} I0603 05:03:49.515923 140045070399360 runner_lib.py:276] Evaluation result for checkpoint gs://***/models/model.ckpt-0: {'inception_score_mean': 1.0062603, 'inception_score_std': 0.00025255934, 'inception_score_list': '1.0059446_1.0065628_1.0062735', 'fid_score_mean': 433.68958, 'fid_score_std': 0.2974496, 'fid_score_list': '434.10986_433.4947_433.46417'} (default value: -1.0) Traceback (most recent call last): File "compare_gan/main.py", line 133, in <module> app.run(main) File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 300, in run _run_main(main, args) File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 251, in _run_main sys.exit(main(argv)) File "compare_gan/main.py", line 127, in main eval_every_steps=FLAGS.eval_every_steps) File "/content/gdrive/My Drive/compare_gan/compare_gan/runner_lib.py", line 354, in run_with_schedule num_averaging_runs=num_eval_averaging_runs) File "/content/gdrive/My Drive/compare_gan/compare_gan/runner_lib.py", line 277, in _run_eval task_manager.add_eval_result(checkpoint_path, result_dict, default_value) File "/content/gdrive/My Drive/compare_gan/compare_gan/runner_lib.py", line 209, in add_eval_result config = self._get_config_for_step(step) File "/content/gdrive/My Drive/compare_gan/compare_gan/runner_lib.py", line 202, in _get_config_for_step last_config_step = sorted([s for s in config_steps if s <= step])[-1] File "/content/gdrive/My Drive/compare_gan/compare_gan/runner_lib.py", line 202, in <listcomp> last_config_step = sorted([s for s in config_steps if s <= step])[-1] TypeError: '<=' not supported between instances of 'int' and 'str'

[anth2o]

I faced the same problem when running on my GPU. I just fixed it by adding the following code just where the last error is raised.

if not isinstance(step, int):
    step = int(step)
last_config_step = sorted([s for s in config_steps if s <= step])[-1]

[CrackerHax]

Thanks, I got it to work by changing 's' to 'int(s)' but your way is better I think.

[Marvin182]

Sorry for this bug. The next release will include a fix.