Error when continuing training

[tgeijten]

When an earlier run is found, I get the following error message:

Found earlier run, continuing training.
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\Development\depRL\deprl\main.py", line 151, in <module>
    main()
  File "D:\Development\depRL\deprl\main.py", line 147, in main
    train(config)
  File "D:\Development\depRL\deprl\main.py", line 77, in train
    logger.initialize(
  File "D:\Development\depRL\deprl\vendor\tonic\utils\logger.py", line 272, in initialize
    current_logger = Logger(*args, **kwargs)
  File "D:\Development\depRL\deprl\vendor\tonic\utils\logger.py", line 79, in __init__
    create_resumed_results_path(config, env)
  File "D:\Development\depRL\deprl\vendor\tonic\utils\path_utils.py", line 10, in wrapper
    result = func(*args, **kwargs)
  File "D:\Development\depRL\deprl\vendor\tonic\utils\logger.py", line 56, in create_resumed_results_path
    folder = get_sorted_folders(folders[0][1])[-1]
  File "D:\Development\depRL\deprl\vendor\tonic\utils\logger.py", line 27, in get_sorted_folders
    sorted_folders = sorted(folders, key=get_datetime_key)
  File "D:\Development\depRL\deprl\vendor\tonic\utils\logger.py", line 18, in get_datetime_key
    date_time_str = s.split(".")[0] + s.split(".")[1]
IndexError: list index out of range

[P-Schumacher]

Hi tgeijten, what exactly did you run and when did this error occur?

[tgeijten]

Hi Pierre, here's an example of steps to reproduce the issue (Windows 10):

1. Run: python -m deprl.main scone_run_h0918.yaml
2. Wait until some checkpoints are generated
3. Cancel the optimization
4. Run again: python -m deprl.main scone_run_h0918.yaml