Update pybio (bioimageio) submodules

[FynnBe]

.. and remove unused cache_path from eval_model_zip

[FynnBe]

.. and refactor package name pybio -> bioimageio

problem: torch script weight formats seems to become invalid due to this refactor

Are any bioimageio models currently on the website already use torchscript or can we just go ahead and recreate these test weights and ignore this problem otherwise? @constantinpape , I think you created the torch script weights, what's your take?

[k-dominik]

problem: torch script weight formats seems to become invalid due to this refactor

out of curiosity - do you know why this happens?

[FynnBe]

problem: torch script weight formats seems to become invalid due to this refactor

out of curiosity - do you know why this happens?

no, but the same happens with pickle. So maybe this is related

[k-dominik]

ahh @FynnBe, you have good instincts. I have checked the old weights, and those reference the model python file in pybio/torch/models/unet2d.py... So I would assume this is only related to this testdata.

[FynnBe]

So I would assume this is only related to this testdata.

good. The refactor really shouldn't break anything else

[FynnBe]

@k-dominik any idea why the CI doesn't run here?

[k-dominik]

lgtm in general. Test failure seems to be related to torchscript archive. Not sure how this was generated, but given that the size is almost twice as before, there seems to be something odd.

[k-dominik]

to me it looks like the tests/data/unet2d_torchscript/weights.pt file is corrupt. I tried unzipping it and got the following error:

caution:  zipfile comment truncated
error [.../tiktorch/tests/data/unet2d_torchscript/weights.pt]:  missing 9131070 bytes in zipfile
  (attempting to process anyway)
error [.../tiktorch/tests/data/unet2d_torchscript/weights.pt]:  attempt to seek before beginning of zipfile

@constantinpape , @FynnBe how was this weight file generated? Would it be possible to regenerate it? (By the way, I found it interesting that this information could not be inferred from the model.yaml... - how are users supposed to know how it was trained?)

[constantinpape]

to me it looks like the tests/data/unet2d_torchscript/weights.pt file is corrupt.

Yes indeed, looks like something with the torchscript weights is wrong.

@constantinpape , @FynnBe how was this weight file generated? Would it be possible to regenerate it?

The whole model was copied from here https://github.com/bioimage-io/pytorch-bioimage-io/blob/master/specs/models/unet2d_nuclei_broad/UNet2DNucleiBroad.model.yaml#L63-L76 and the torchscript weigths are just generated from the normal pytorch model.

(By the way, I found it interesting that this information could not be inferred from the model.yaml... - how are users supposed to know how it was trained?)

Yes, that's indeed a bit unfortunate. The reason is that we had a formalized description of the training procedure in the first iteration of the bioimage format. However, it turned out that this was to complex, so we know rely on self-documentation of the training procedure and providing the training script or linking to the relevant notebook. Unfortunately for the model at hand this got lost over time because it was copied around so much ...
I can re-add this part, but for this it would be good if we use a single version of this model (preferably the one at https://github.com/bioimage-io/pytorch-bioimage-io/blob/master/specs/models/unet2d_nuclei_broad/UNet2DNucleiBroad.model.yaml) and don't keep separate versions that need to be kept in sync.

[k-dominik]

thank you for the update @constantinpape !

I can re-add this part, but for this it would be good if we use a single version of this model (preferably the one at https://github.com/bioimage-io/pytorch-bioimage-io/blob/master/specs/models/unet2d_nuclei_broad/UNet2DNucleiBroad.model.yaml) and don't keep separate versions that need to be kept in sync.

let's re-add the weights here and tackle the single model change in a separate PR. I've opened an issue (#165) about using a single model for the test case.

[constantinpape]

I think the torchscript weights were generated with this script:
https://github.com/bioimage-io/pytorch-bioimage-io/blob/master/pybio/torch/converters/torchscript.py

[k-dominik]

lgtm,

the only thing I really do not like is that this PR adds even more pip dependencies.

Ideally, we would build the whole tiktorch-server package without pip. I have started the process on making some of the packages available via conda-forge (conda-forge/staged-recipes#15250)

[FynnBe]

@k-dominik should we continue this PR a bit more do switch to conda-packages and further update the bioimageio dependencies?

[k-dominik]

oh I would prefer to merge this now. With conda packages you never know :)