libontology wrongly keeps models that are not present

[amjjbonvin]

I am running the homotrimer test example locally on my laptop. In step 4 (flexref) one model fails (i.e. no PDB is written to file). Since the tolerance is set yo 20% this does not stop the workflow. The next module caprieval however tries to read that failed structure, causing an error:

Traceback (most recent call last):
  File "/Users/abonvin/haddock_git/haddock3/src/haddock/libs/libutil.py", line 320, in log_error_and_exit
    yield
  File "/Users/abonvin/haddock_git/haddock3/src/haddock/clis/cli.py", line 148, in main
    workflow.run()
  File "/Users/abonvin/haddock_git/haddock3/src/haddock/libs/libworkflow.py", line 27, in run
    step.execute()
  File "/Users/abonvin/haddock_git/haddock3/src/haddock/libs/libworkflow.py", line 100, in execute
    module.run(**self.config)
  File "/Users/abonvin/haddock_git/haddock3/src/haddock/modules/__init__.py", line 159, in run
    self._run()
  File "/Users/abonvin/haddock_git/haddock3/src/haddock/modules/analysis/caprieval/__init__.py", line 48, in _run
    capri = CAPRI(
  File "/Users/abonvin/haddock_git/haddock3/src/haddock/modules/analysis/caprieval/capri.py", line 92, in __init__
    self.atoms = get_atoms(model_list + [reference])
  File "/Users/abonvin/haddock_git/haddock3/src/haddock/modules/analysis/caprieval/capri.py", line 591, in get_atoms
    with open(pdb) as fh:
FileNotFoundError: [Errno 2] No such file or directory: '../04_flexref/flexref_4.pdb'
[2022-02-07 16:13:05,564 libutil ERROR] [Errno 2] No such file or directory: '../04_flexref/flexref_4.pdb'

[rvhonorato]

This happens because libontology.check_faulty does not remove the models that are not present, it simply counts them. Same error will happen for other modules.

Since this is a python-shell related issue and I'm not sure what is the "correct" way of solving, @joaomcteixeira have any guidelines? My initial idea is to simply remove them from the ontology object if they are not present using del or remove - is this acceptable or do I need to use a different method?

[joaomcteixeira]

I will have a look.

[joaomcteixeira]

Adding that functionality to the ModuleIO is a good strategy. We could add it to another method, but for now, and simplicity you can add it directly to the check_faulty, if the file does not exist, remove it from the self.output, yes.

[rvhonorato]

We could add it to another method

good idea, so since its all python I think you are more qualified to handle this 🥂

[amjjbonvin]

This one is needed to ensure full functionality / usability