Floating Point Exceptions with Sevennet

[rohan335]

When using Sevennet to run MD and then relax structures, I'm getting a floating point error:
/var/spool/slurmd/job1010021/slurm_script: line 49: 2660161 Floating point exception(core dumped) python evals/torch_sim_test.py

Sometimes the code get stuck and doesn't make it to this point (just hanging forever), but when it seems to not hang, there's a floating point exception. I'm also converting cif strings to atoms objects here, but that part of the pipeline seemed to be working fine with other libraries.

Here's my code that has this issue:

  cp = sevenn.util.load_checkpoint("7net-mf-ompa")
  backend = "e3nn"
  model_loaded = cp.build_model(backend)
  model_loaded.set_is_batch_data(True)
  model_loaded = model_loaded.to(device)

  sevenn_model = SevenNetModel(
      model=model_loaded,
      modal="mpa",
      device=device
  )

  atoms_list = []
  for cif in cif_strings:
    crystal = cif_str_to_crystal(cif)

    if crystal is None or not crystal.valid:
        continue

    structure = Structure.from_str(cif, fmt="cif")
    if len(structure) == 1:
        continue

    cif_buffer = io.StringIO(cif)
    atoms = read(cif_buffer, format="cif")
    atoms_list.append(atoms)
    
  final_state = ts.integrate(
      system=atoms_list,
      model=sevenn_model,
      n_steps=50,
      timestep=0.002,
      temperature=1000,
      integrator=ts.nvt_langevin,
      autobatcher=True,
  )

  relaxed_state = ts.optimize(
      system=final_state,
      model=sevenn_model,
      optimizer=ts.frechet_cell_fire,
      max_steps=1500,
      autobatcher=True,
  )

I would love some thoughts or ideas on how to solve this! Thanks :)

[orionarcher]

Hi @rohan335! Thanks for raising an issue!

I've successfully run this snippet on a set of Si atoms instead of the CIFs you mentioned. Could you share some of the CIFs so that I can try to reproduce the error more directly? Could you also share what device you are using? Running this on CPU will fail because autobatching is not supported on CPU, though if that's the issue it should throw an informative error message.

In the meantime, try running this snippet and seeing if it works for you. That should help isolate whether it's an issue with the system or with your environment configuration.

device = torch.device("cuda")
import torch_sim as ts
cp = sevenn.util.load_checkpoint("7net-mf-ompa")
backend = "e3nn"
model_loaded = cp.build_model(backend)
model_loaded.set_is_batch_data(True)
model_loaded = model_loaded.to(device)

sevenn_model = SevenNetModel(
    model=model_loaded,
    modal="mpa",
    device=device
)

si_atoms = bulk("Si", "diamond", a=5.43, cubic=True)

atoms_list = [si_atoms] * 4

final_state = ts.integrate(
    system=atoms_list,
    model=sevenn_model,
    n_steps=10,
    timestep=0.002,
    temperature=1000,
    integrator=ts.nvt_langevin,
    autobatcher=True,
)

relaxed_state = ts.optimize(
    system=final_state,
    model=sevenn_model,
    optimizer=ts.frechet_cell_fire,
    max_steps=10,
    autobatcher=True,
)

[CompRhys]

Closing this issue as it wasn't able to be reproduced. Feel free to re-open if it can be reproduced/traced to a particular issue.