No such file or directory: ...aux.json

[aymeric75]

Hello,

I re-installed latplan from the begining and did not modify any file.

When I execute this command:

./train_kltune.py learn_summary_plot_dump puzzle mnist 3 3 40000 CubeSpaceAE_AMA4Plus

Here is the error that appears:

WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
Using TensorFlow backend.
0:00:00 0 | Estimated finish time:  ----/--/-- --:--:-- ------
0:00:00 1 | 2022-05-24 17:18:29 [t] elbo 4.27e+05  kl_a_z0     1.05  kl_a_z1     1.05  kl_z0     60.8  kl_z0z3      107  kl_z1     60.9  kl_z1z2      107  loss 9.62e+05  pdiff_z0z1   0.0832  pdiff_z0z2    0.312  pdiff_z0z3    0.338  pdiff_z1z2    0.337  tau        5  x0y0 2.12e+05  x0y3 2.16e+05  x1y1 2.12e+05  x1y2 2.16e+05
Fancy Traceback (most recent call last):
  File latplan/main/common.py line 115 function main : task(args)
              parameters = {'test_noise': False, 'test_hard': True, 'train_noise': True, 'train_hard': False, 'dropout_z': False, 'noise': 0.2, 'dropout': 0.2, 'optimizer': 'radam', 'min_temperature': 0.5, 'epoch': 2000, 'gs_annealing_start': 0, 'gs_annealing_end': 1000, 'clipnorm': 0.1, 'batch_size': [400], 'lr': [0.001], 'N': [50, 100, 300], 'zerosuppress': 0.1, 'densify': False, 'max_temperature': [5.0], 'conv_channel': [32], 'conv_channel_increment': [1], 'conv_kernel': [5], 'conv_pooling': [1], 'conv_per_pooling': [1], 'conv_depth': [3], 'fc_width': [100], 'fc_depth': [2], 'A': [6000], 'aae_activation': ['relu'], 'aae_width': [1000], 'aae_depth': [2], 'eff_regularizer': [None], 'beta_d': [1, 10, 100, '...<2 more>'], 'beta_z': [1, 10], 'output': 'GaussianOutput(sigma=0.1)'}

  File latplan/main/puzzle.py line 45 function puzzle : ae = run(os.path.join("samples",common.sae_path), transitions)
                    args = Namespace(aeclass='CubeSpaceAE_AMA4Plus', comment='', height=3, mode='learn_summary_plot_dump', num_examples=40000, type='mnist', width=3)
             transitions = '<numpy.ndarray float32  (40000, 2, 48, 48, 1)>'
                  states = '<numpy.ndarray float32  (80000, 48, 48, 1)>'

  File latplan/main/common.py line 228 function run : simple_genetic_search(
                   extra = None
                    path = 'samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus'
                    test = '<numpy.ndarray float32  (2000, 2, 48, 48, 1)>'
                   train = '<numpy.ndarray float32  (36000, 2, 48, 48, 1)>'
             transitions = '<numpy.ndarray float32  (40000, 2, 48, 48, 1)>'
                     val = '<numpy.ndarray float32  (2000, 2, 48, 48, 1)>'

  File latplan/util/tuning.py line 531 function simple_genetic_search : open_list, close_list = _iter(config)
              parameters = {'test_noise': [False], 'test_hard': [True], 'train_noise': [True], 'train_hard': [False], 'dropout_z': [False], 'noise': [0.2], 'dropout': [0.2], 'optimizer': ['radam'], 'min_temperature': [0.5], 'epoch': [2000], 'gs_annealing_start': [0], 'gs_annealing_end': [1000], 'clipnorm': [0.1], 'batch_size': [400], 'lr': [0.001], 'N': [50, 100, 300], 'zerosuppress': [0.1], 'densify': [False], 'max_temperature': [5.0], 'conv_channel': [32], 'conv_channel_increment': [1], 'conv_kernel': [5], 'conv_pooling': [1], 'conv_per_pooling': [1], 'conv_depth': [3], 'fc_width': [100], 'fc_depth': [2], 'A': [6000], 'aae_activation': ['relu'], 'aae_width': [1000], 'aae_depth': [2], 'eff_regularizer': [None], 'beta_d': [1, 10, 100, '...<2 more>'], 'beta_z': [1, 10], 'output': ['GaussianOutput(sigma=0.1)'], 'mode': ['learn_summary_plot_dump'], 'type': ['mnist'], 'width': [3], 'height': [3], 'num_examples': [40000], 'aeclass': ['CubeSpaceAE_AMA4Plus'], 'comment': [''], 'generator': ['
      initial_population = 100
              population = 100
               open_list = []
              close_list = {}
              max_trials = 100
              gen_config = <generator object _random_configs at 0x154442ea9f20>
                    done = False
                       _ = 0
                  config = {'test_noise': False, 'test_hard': True, 'train_noise': True, 'train_hard': False, 'dropout_z': False, 'noise': 0.2, 'dropout': 0.2, 'optimizer': 'radam', 'min_temperature': 0.5, 'epoch': 2000, 'gs_annealing_start': 0, 'gs_annealing_end': 1000, 'clipnorm': 0.1, 'batch_size': 400, 'lr': 0.001, 'N': 100, 'zerosuppress': 0.1, 'densify': False, 'max_temperature': 5.0, 'conv_channel': 32, 'conv_channel_increment': 1, 'conv_kernel': 5, 'conv_pooling': 1, 'conv_per_pooling': 1, 'conv_depth': 3, 'fc_width': 100, 'fc_depth': 2, 'A': 6000, 'aae_activation': 'relu', 'aae_width': 1000, 'aae_depth': 2, 'eff_regularizer': None, 'beta_d': 10000, 'beta_z': 1, 'output': 'GaussianOutput(sigma=0.1)', 'mode': 'learn_summary_plot_dump', 'type': 'mnist', 'width': 3, 'height': 3, 'num_examples': 40000, 'aeclass': 'CubeSpaceAE_AMA4Plus', 'comment': '', 'generator': 'latplan.puzzles.puzzle_mnist', 'picsize': [48, 48], 'mean': [[[0.0], [0.0], [0.0], '...<45 more>'], [[0.0], [0.0], [0.
                   limit = 100
                    path = 'samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus'
             report_best = None

  File latplan/util/tuning.py line 513 function _iter : artifact, eval = task(config)
              time_start = datetime.datetime(2022, 5, 24, 17, 18, 10, 209150)
                  config = {'test_noise': False, 'test_hard': True, 'train_noise': True, 'train_hard': False, 'dropout_z': False, 'noise': 0.2, 'dropout': 0.2, 'optimizer': 'radam', 'min_temperature': 0.5, 'epoch': 2000, 'gs_annealing_start': 0, 'gs_annealing_end': 1000, 'clipnorm': 0.1, 'batch_size': 400, 'lr': 0.001, 'N': 100, 'zerosuppress': 0.1, 'densify': False, 'max_temperature': 5.0, 'conv_channel': 32, 'conv_channel_increment': 1, 'conv_kernel': 5, 'conv_pooling': 1, 'conv_per_pooling': 1, 'conv_depth': 3, 'fc_width': 100, 'fc_depth': 2, 'A': 6000, 'aae_activation': 'relu', 'aae_width': 1000, 'aae_depth': 2, 'eff_regularizer': None, 'beta_d': 10000, 'beta_z': 1, 'output': 'GaussianOutput(sigma=0.1)', 'mode': 'learn_summary_plot_dump', 'type': 'mnist', 'width': 3, 'height': 3, 'num_examples': 40000, 'aeclass': 'CubeSpaceAE_AMA4Plus', 'comment': '', 'generator': 'latplan.puzzles.puzzle_mnist', 'picsize': [48, 48], 'mean': [[[0.0], [0.0], [0.0], '...<45 more>'], [[0.0], [0.0], [0.
                   limit = 100
                    path = 'samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus'
             report_best = None

  File latplan/util/util.py line 2 function <lambda> : return lambda *args,**kwargs: fn(*args1,*args,**{**kwargs1,**kwargs})
                    args = ({'test_noise': False, 'test_hard': True, 'train_noise': True, 'train_hard': False, 'dropout_z': False, 'noise': 0.2, 'dropout': 0.2, 'optimizer': 'radam', 'min_temperature': 0.5, 'epoch': 2000, 'gs_annealing_start': 0, 'gs_annealing_end': 1000, 'clipnorm': 0.1, 'batch_size': 400, 'lr': 0.001, 'N': 100, 'zerosuppress': 0.1, 'densify': False, 'max_temperature': 5.0, 'conv_channel': 32, 'conv_channel_increment': 1, 'conv_kernel': 5, 'conv_pooling': 1, 'conv_per_pooling': 1, 'conv_depth': 3, 'fc_width': 100, 'fc_depth': 2, 'A': 6000, 'aae_activation': 'relu', 'aae_width': 1000, 'aae_depth': 2, 'eff_regularizer': None, 'beta_d': 10000, 'beta_z': 1, 'output': 'GaussianOutput(sigma=0.1)', 'mode': 'learn_summary_plot_dump', 'type': 'mnist', 'width': 3, 'height': 3, 'num_examples': 40000, 'aeclass': 'CubeSpaceAE_AMA4Plus', 'comment': '', 'generator': 'latplan.puzzles.puzzle_mnist', 'picsize': [48, 48], 'mean': [[[0.0], [0.0], [0.0], '...<45 more>'], [[0.0], [0.0], [0
                  kwargs = {}
                   args1 = ((<class 'latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus'>, 0), ('samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus', 1), ('<numpy.ndarray float32  (36000, 2, 48, 48, 1)>', 2), ('...<3 more>', None))
                 kwargs1 = {}

  File latplan/util/tuning.py line 142 function nn_task : net.train(train_in,
                    path = 'samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus'
                train_in = '<numpy.ndarray float32  (36000, 2, 48, 48, 1)>'
               train_out = '<numpy.ndarray float32  (36000, 2, 48, 48, 1)>'
                  val_in = '<numpy.ndarray float32  (2000, 2, 48, 48, 1)>'
                 val_out = '<numpy.ndarray float32  (2000, 2, 48, 48, 1)>'
              parameters = {'test_noise': False, 'test_hard': True, 'train_noise': True, 'train_hard': False, 'dropout_z': False, 'noise': 0.2, 'dropout': 0.2, 'optimizer': 'radam', 'min_temperature': 0.5, 'epoch': 2000, 'gs_annealing_start': 0, 'gs_annealing_end': 1000, 'clipnorm': 0.1, 'batch_size': 400, 'lr': 0.001, 'N': 100, 'zerosuppress': 0.1, 'densify': False, 'max_temperature': 5.0, 'conv_channel': 32, 'conv_channel_increment': 1, 'conv_kernel': 5, 'conv_pooling': 1, 'conv_per_pooling': 1, 'conv_depth': 3, 'fc_width': 100, 'fc_depth': 2, 'A': 6000, 'aae_activation': 'relu', 'aae_width': 1000, 'aae_depth': 2, 'eff_regularizer': None, 'beta_d': 10000, 'beta_z': 1, 'output': 'GaussianOutput(sigma=0.1)', 'mode': 'learn_summary_plot_dump', 'type': 'mnist', 'width': 3, 'height': 3, 'num_examples': 40000, 'aeclass': 'CubeSpaceAE_AMA4Plus', 'comment': '', 'generator': 'latplan.puzzles.puzzle_mnist', 'picsize': [48, 48], 'mean': [[[0.0], [0.0], [0.0], '...<45 more>'], [[0.0], [0.0], [0.
                  resume = False
                     net = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>

  File latplan/network.py line 448 function train : clist.on_epoch_end(self.epoch,logs)
                        val_data = ['<numpy.ndarray float32  (2000, 2, 48, 48, 1)>']
                     val_data_to = ['<numpy.ndarray float32  (2000, 2, 48, 48, 1)>']
                          resume = False
                          kwargs = {'test_noise': False, 'test_hard': True, 'train_noise': True, 'train_hard': False, 'dropout_z': False, 'noise': 0.2, 'dropout': 0.2, 'optimizer': 'radam', 'min_temperature': 0.5, 'epoch': 2000, 'gs_annealing_start': 0, 'gs_annealing_end': 1000, 'clipnorm': 0.1, 'batch_size': 400, 'lr': 0.001, 'N': 100, 'zerosuppress': 0.1, 'densify': False, 'max_temperature': 5.0, 'conv_channel': 32, 'conv_channel_increment': 1, 'conv_kernel': 5, 'conv_pooling': 1, 'conv_per_pooling': 1, 'conv_depth': 3, 'fc_width': 100, 'fc_depth': 2, 'A': 6000, 'aae_activation': 'relu', 'aae_width': 1000, 'aae_depth': 2, 'eff_regularizer': None, 'beta_d': 10000, 'beta_z': 1, 'output': 'GaussianOutput(sigma=0.1)', 'mode': 'learn_summary_plot_dump', 'type': 'mnist', 'width': 3, 'height': 3, 'num_examples': 40000, 'aeclass': 'CubeSpaceAE_AMA4Plus', 'comment': '', 'generator': 'latplan.puzzles.puzzle_mnist', 'picsize': [48, 48], 'mean': [[[0.0], [0.0], [0.0], '...<45 more>'], [[0.0], [0
                           epoch = 2000
                     input_shape = ((2, 0), (48, 1), (48, 2), ('...<1 more>', None))
                             net = <keras.engine.training.Model object at 0x1544133b69d0>
                     index_array = '<numpy.ndarray int64    (36000,)>'
                           clist = <keras.callbacks.CallbackList object at 0x154413026460>
                         aborted = True
                            logs = {'t_loss': 961513.807638889, 't_tau': 5.0, 't_pdiff_z1z2': 0.33723170194360946, 't_pdiff_z0z3': 0.33751753667990364, 't_pdiff_z0z1': 0.08319545677966542, 't_pdiff_z0z2': 0.31239486038684844, 't_kl_z0': 60.83786286248101, 't_kl_z1': 60.90224732293023, 't_kl_a_z0': 1.053026196691725, 't_kl_a_z1': 1.0494115193684896, 't_kl_z1z2': 106.98173166910807, 't_kl_z0z3': 107.01499065823025, 't_x0y0': 212392.99565972222, 't_x1y1': 212254.3015625, 't_x0y3': 216092.028125, 't_x1y2': 215806.39965277776, 't_elbo': 426575.5024305555, 'v_loss': 963629.525, 'v_tau': 5.0, 'v_pdiff_z1z2': 0.33807403445243833, 'v_pdiff_z0z3': 0.3383484840393066, 'v_pdiff_z0z1': 0.0833947628736496, 'v_pdiff_z0z2': 0.3129697561264038, 'v_kl_z0': 60.833403778076175, 'v_kl_z1': 60.89156723022461, 'v_kl_a_z0': 1.040119481086731, 'v_kl_a_z1': 1.0365202665328979, 'v_kl_z1z2': 107.33934173583984, 'v_kl_z0z3': 107.41255035400391, 'v_x0y0': 212206.8, 'v_x1y1': 212671.03125, 'v_x0y3': 216667.725, 'v_x
                   indices_cache = ['<numpy.ndarray int64    (400,)>', '<numpy.ndarray int64    (400,)>', '<numpy.ndarray int64    (400,)>', '...<87 more>']
                train_data_cache = [['<numpy.ndarray float32  (400, 2, 48, 48, 1)>'], ['<numpy.ndarray float32  (400, 2, 48, 48, 1)>'], ['<numpy.ndarray float32  (400, 2, 48, 48, 1)>'], '...<87 more>']
             train_data_to_cache = [['<numpy.ndarray float32  (400, 2, 48, 48, 1)>'], ['<numpy.ndarray float32  (400, 2, 48, 48, 1)>'], ['<numpy.ndarray float32  (400, 2, 48, 48, 1)>'], '...<87 more>']
             train_subdata_cache = ['<numpy.ndarray float32  (400, 2, 48, 48, 1)>']
          train_subdata_to_cache = ['<numpy.ndarray float32  (400, 2, 48, 48, 1)>']
       train_subdata_batch_cache = '<numpy.ndarray float32  (400, 2, 48, 48, 1)>'
    train_subdata_to_batch_cache = '<numpy.ndarray float32  (400, 2, 48, 48, 1)>'
                               k = 'elbo'
                               v = 426803.48125
                      batch_size = 400
                        clipnorm = 0.1
                              lr = 0.001
                       optimizer = 'radam'
                   plot_val_data = '<numpy.ndarray float32  (1, 2, 48, 48, 1)>'
                            self = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>
                      train_data = ['<numpy.ndarray float32  (36000, 2, 48, 48, 1)>']
                   train_data_to = ['<numpy.ndarray float32  (36000, 2, 48, 48, 1)>']

  File ../../../g100/home/userexternal/abarbin0/.conda/envs/latplan/lib/python3.8/site-packages/keras/callbacks.py line 152 function on_epoch_end : callback.on_epoch_end(epoch, logs)
                    self = <keras.callbacks.CallbackList object at 0x154413026460>
                   epoch = 0
                    logs = {'t_loss': 961513.807638889, 't_tau': 5.0, 't_pdiff_z1z2': 0.33723170194360946, 't_pdiff_z0z3': 0.33751753667990364, 't_pdiff_z0z1': 0.08319545677966542, 't_pdiff_z0z2': 0.31239486038684844, 't_kl_z0': 60.83786286248101, 't_kl_z1': 60.90224732293023, 't_kl_a_z0': 1.053026196691725, 't_kl_a_z1': 1.0494115193684896, 't_kl_z1z2': 106.98173166910807, 't_kl_z0z3': 107.01499065823025, 't_x0y0': 212392.99565972222, 't_x1y1': 212254.3015625, 't_x0y3': 216092.028125, 't_x1y2': 215806.39965277776, 't_elbo': 426575.5024305555, 'v_loss': 963629.525, 'v_tau': 5.0, 'v_pdiff_z1z2': 0.33807403445243833, 'v_pdiff_z0z3': 0.3383484840393066, 'v_pdiff_z0z1': 0.0833947628736496, 'v_pdiff_z0z2': 0.3129697561264038, 'v_kl_z0': 60.833403778076175, 'v_kl_z1': 60.89156723022461, 'v_kl_a_z0': 1.040119481086731, 'v_kl_a_z1': 1.0365202665328979, 'v_kl_z1z2': 107.33934173583984, 'v_kl_z0z3': 107.41255035400391, 'v_x0y0': 212206.8, 'v_x1y1': 212671.03125, 'v_x0y3': 216667.725, 'v_x1y2': 21
                callback = <keras.callbacks.LambdaCallback object at 0x1544130263d0>

  File latplan/network.py line 370 function <lambda> : self.plot_transitions(
                   epoch = 0
                    logs = {'t_loss': 961513.807638889, 't_tau': 5.0, 't_pdiff_z1z2': 0.33723170194360946, 't_pdiff_z0z3': 0.33751753667990364, 't_pdiff_z0z1': 0.08319545677966542, 't_pdiff_z0z2': 0.31239486038684844, 't_kl_z0': 60.83786286248101, 't_kl_z1': 60.90224732293023, 't_kl_a_z0': 1.053026196691725, 't_kl_a_z1': 1.0494115193684896, 't_kl_z1z2': 106.98173166910807, 't_kl_z0z3': 107.01499065823025, 't_x0y0': 212392.99565972222, 't_x1y1': 212254.3015625, 't_x0y3': 216092.028125, 't_x1y2': 215806.39965277776, 't_elbo': 426575.5024305555, 'v_loss': 963629.525, 'v_tau': 5.0, 'v_pdiff_z1z2': 0.33807403445243833, 'v_pdiff_z0z3': 0.3383484840393066, 'v_pdiff_z0z1': 0.0833947628736496, 'v_pdiff_z0z2': 0.3129697561264038, 'v_kl_z0': 60.833403778076175, 'v_kl_z1': 60.89156723022461, 'v_kl_a_z0': 1.040119481086731, 'v_kl_a_z1': 1.0365202665328979, 'v_kl_z1z2': 107.33934173583984, 'v_kl_z0z3': 107.41255035400391, 'v_x0y0': 212206.8, 'v_x1y1': 212671.03125, 'v_x0y3': 216667.725, 'v_x1y2': 21
           plot_val_data = '<numpy.ndarray float32  (1, 2, 48, 48, 1)>'
                    self = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>

  File latplan/model.py line 1117 function plot_transitions : z = self.encode(x)
                    self = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>
                    data = '<numpy.ndarray float32  (1, 2, 48, 48, 1)>'
                    path = 'samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus/logs/15e7c133e5909c7240fdef903911e195/'
                 verbose = False
                   epoch = 0
                basename = 'samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus/logs/15e7c133e5909c7240fdef903911e195/'
                     ext = ''
                pre_path = 'samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus/logs/15e7c133e5909c7240fdef903911e195/_pre'
                suc_path = 'samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus/logs/15e7c133e5909c7240fdef903911e195/_suc'
                       x = '<numpy.ndarray float32  (1, 2, 48, 48, 1)>'

  File latplan/model.py line 487 function encode : return self.adaptively(super().encode, data, *args, **kwargs)
                    self = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>
                    data = '<numpy.ndarray float32  (1, 2, 48, 48, 1)>'
                    args = ()
                  kwargs = {}

  File latplan/model.py line 481 function adaptively : return fn(data,*args,**kwargs)
                    self = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>
                      fn = <bound method AE.encode of <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>>
                    data = '<numpy.ndarray float32  (1, 2, 48, 48, 1)>'
                    args = ()
                  kwargs = {}

  File latplan/model.py line 119 function encode : self.load()
                    self = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>
                    data = '<numpy.ndarray float32  (1, 2, 48, 48, 1)>'
                  kwargs = {}

  File latplan/network.py line 202 function load : self._load(path)
                    self = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>
           allow_failure = False
                    path = ''

  File latplan/model.py line 1410 function _load : super()._load(path)
                    self = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>
                    path = ''

  File latplan/network.py line 213 function _load : with open(self.local(os.path.join(path,"aux.json")), "r") as f:
                    self = <latplan.model.ConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x154442e85b80>
                    path = ''


FileNotFoundError: [Errno 2] No such file or directory: 'samples/puzzle_mnist_3_3_40000_CubeSpaceAE_AMA4Plus/logs/15e7c133e5909c7240fdef903911e195/aux.json'

Any idea why it's creating this error ?

Thank you

[guicho271828]

As communicated in the email, also documented in the readme, please use a released tag 5.0.0 which is stable.