CLI crashes when running with --insights flag

[afritzler]

Hi Marc,

Trying to train an old model of mine. Everything works fine, however, when I try to turn on the insights flag I get the following dump:

Running

aetros start --local USERNAME/MODEL --insights

Trace

Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/aetros/__main__.py", line 5, in <module>
    sys.exit(aetros.main())
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/aetros/__init__.py", line 121, in main
    code = command.main(cmd_args)
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/aetros/commands/StartSimpleCommand.py", line 45, in main
    start_keras(self.logger, job_backend)
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/aetros/starter.py", line 702, in start_keras
    keras_model_utils.job_start(job_backend, trainer, keras_logger)
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/aetros/keras_model_utils.py", line 93, in job_start
    model_provider.train(trainer, model, data_train, data_validation)
  File "model.py", line 132, in train
    callbacks=trainer.callbacks
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/keras/engine/training.py", line 2141, in fit_generator
    callbacks.on_train_begin()
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/keras/callbacks.py", line 130, in on_train_begin
    callback.on_train_begin(logs)
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/aetros/KerasCallback.py", line 195, in on_train_begin
    images = self.build_insight_images()
  File "/Users/i820491/.virtualenvs/aetros/lib/python2.7/site-packages/aetros/KerasCallback.py", line 375, in build_insight_images
    Y = result[0]
IndexError: list index out of range

Cheers

[marcj]

That happens on the newest Keras version. Please maker sure to install Keras >2.0,<=2.1.2 and try again.

[afritzler]

Thanks, that did the trick!