Save, load and save again

[rm-rf0]

Hi,

Thanks for this useful library!

I wonder how to do save - load - save - load multiple times?
To achieve this, I use the code below:

if os.path.exists(log_path):
    load_logs(optimizer, logs=[log_path])
    print('Load {} records from disk.'.format(len(optimizer.space)))

logger = JSONLogger(path=log_path)
optimizer.subscribe(Events.OPTMIZATION_STEP, logger)

But it seems that this line

logger = JSONLogger(path=log_path)

will delete the orginal json file and previous records are lost.

Why do we need to do os.remove(self._path) in JSONLogger.__init__()?

[fmfn]

Good point, maybe we don't. Let me take a look and try to remember if I had a good reason to add it. If not I can remove it. In the meantime, feel free to submit a PR.

[bionicles]

this messes it up for us, because it deletes our logs!

@fmfn it might be good to check if the bounds changed, and only replace the logs if they did, and also critically to rename the logs file instead of deleting it, because that data is valuable, and it's just getting tossed out

[bionicles]

@fmfn
more thoughts ... I think this package is gonna accelerate deep learning research greatly, but I would request one change to the logs system:

Instead of rewriting the logs file and passing a filepath, is it too hard to pass a logs folder? Then we could save timestamped logs which might be useful for reproducability and publications. I think if the first line of the logs file contained the "Bounds" information, then bayesopt can easily compare the current bounds, to the bounds of the previous logs in the logs folder, and just pick the most recent logs with the same bounds. So if you pass new bounds, it starts a new log file, but if you use bounds from before, it picks where it left off. Is this hard to implement?

[KaleabTessera]

Any feedback on this? I am still unsure as how to save - load - save again. Once I load the logs and save, it clears the logs and all history of past probes. Won't this affect the underlying GP?

[FrancoisData]

Hi,
While it's not save you can just add the following lines in your code:

class newJSONLogger(JSONLogger) :

      def __init__(self, path):
            self._path=None
            super(JSONLogger, self).__init__()
            self._path = path if path[-5:] == ".json" else path + ".json"

[yurkovak]

For those who are wondering, the reset option from @AlbertAlonso isn't available in v1.2.0 yet, so it doesn't come with the installation from pip. In the meanwhile, thanks for this workaround!

Hi,
While it's not save you can just add the following lines in your code:

class newJSONLogger(JSONLogger) :

      def __init__(self, path):
            self._path=None
            super(JSONLogger, self).__init__()
            self._path = path if path[-5:] == ".json" else path + ".json"