Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stable minimization #1267

Closed
wants to merge 3 commits into from
Closed

Stable minimization #1267

wants to merge 3 commits into from

Conversation

jfennick
Copy link

@jfennick jfennick commented Mar 9, 2022

I have been experiencing some instabilities which, after much difficulty, I was able to track down to the minimizer. The observed behavior is that the minimizer will finish 'successfully', but some time later, during equilibration and/or propagating replicas, the simulation may or may not randomly crash with NaNs or even throw CUDA_ERROR_ILLEGAL_ADDRESS (700).

Nearly identical behavior can be found here openmm/openmm#3414. In that ticket, the issue was that the box vectors were not being updated correctly. For stability, during minimization the box vectors should not be allowed to change at all. Subsequent equilibration at NPT should be used to adjust the box vectors.

In my case(s), the box vectors were changing only at the fourth significant figure, and yet that was sufficient to (usually) crash the simulations. Inserting Gradient Descent before FIRE seems to greatly ameliorate but not completely eliminate the instabilities. Replacing FIRE with L-BFGS (with or without Gradient Descent) also seems to fix the instabilities.

This PR adds three changes to improve the stability of minimization:

  1. First and foremost, it temporarily disables the barostat during minimization to keep the box vectors fixed. This is probably the only change that is absolutely necessary to fix the instabilities.
  2. It inserts a very short (24 timestep) Gradient Descent minimization before the main FIRE minimization. Gradient Descent is not asymptotically fast, but it is excellent for preconditioning a 'better' minimizer.
  3. I have replaced FIRE with L-BFGS. Unlike FIRE, L-BFGS does not modify the box vectors (even when the pressure is not None) and it's a fine minimizer anyway.

@jfennick
Copy link
Author

jfennick commented Mar 9, 2022

See also choderalab/openmmtools#557

@jfennick
Copy link
Author

jfennick commented Mar 9, 2022

FYI I just realized that I accidentally named this branch stable_equilibration instead of stable_minimization.

@jfennick jfennick changed the title Stable equilibration Stable minimization Mar 10, 2022
@jfennick jfennick closed this by deleting the head repository Aug 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant