RuntimeError in trying BasinHopping method

[JNapoli]

Hi @leeping, I hope all is well! I want to try out the BasinHopping optimization routine in FB with the lammpsio.py interface I have added. I receive the following error when I try to run it:

"""
indicate.log should not exist yet in this directory: /home/jnapoli/test_reax_fb/regularization_tests/L2_1.0_anneal/opt_reax.tmp/cluster-01/iter_0000
"""

This is triggered in target.py (https://github.com/leeping/forcebalance/blob/master/src/target.py#L508). I think this means that the program should not be in the iter_0000 directory, but I am not very sure. I have also tried BasinHopping with an AbInitio_OPENMM target and receive the same error.

-Joe

[leeping]

Hi Joe,

It's good to hear from you Joe! :) I hope everything is well in the Markland lab. Things are getting underway at UC Davis, and being a faculty member means lots of different responsibilities to allocate time into. I'm still actively developing ForceBalance but it's a challenge to save time for doing research.

I think what is happening is this:

indicate.log is a file that contains summary information on the current iteration. For example, when matching energies and forces, indicate.log will contain the percentage errors in the energy and force. I think the basin-hopping algorithm is somehow evaluating the target twice in the same "iter" folder, which is not the intended design of the code - different evaluations of the force field should live in different folders. Thus, it throws the error.

I think that something in the code should be changed to update the iteration number when basin hopping is used. Alternatively, if the optimization is structured differently (i.e. it is not a linear sequence of iterations) then perhaps ForceBalance should support different directory trees in optimize.tmp. For now, I think some small hack to increment ITERATION_NUMBER, perhaps in optimizer.py, should fix the problem. Let me know if this makes sense.

Thanks,

• Lee-Ping

[JNapoli]

Hi @leeping ,

I'm glad to hear things are off to a good start at Davis! :)

Thanks for the input. I think that makes sense. I can say more specifically now that I think the problem has to do with the basinhopping method's evaluating the objective function multiple times throughout its own 'microiterations'. As a result, the callback within the function wrapping the objective function (https://github.com/leeping/forcebalance/blob/master/src/optimizer.py#L938) was triggering the error. For now it seems like providing callback=False to the wrapper is taking care of the issue.

On a separate note, the Xstream cluster is up and running now at Stanford, which I'd like to be able to use for my PIMD water model optimizations. They are not supporting ssh from the compute nodes to the front end, so I haven't been able to get workers to connect to a master process on the front end. I seem to be able to ssh from a compute node to another cluster (e.g. the Markland cluster), though. Have you dealt with something similar on other HPC resources? It'd be disappointing not to be able to use WorkQueue on this cluster.

Thank you!

Joe

[rmcgibbo]

They are not supporting ssh from the compute nodes to the front end.

This works for me. You have to add your public key to .ssh/authorized_keys.

[JNapoli]

Thanks @rmcgibbo , that's working for me too.

[leeping]

Sounds good, Joe. Go forth and use those resources. :)