Spec issues for zero cost models #195
Comments
|
2d-unet: So it looks like the |
|
3d-unet: |
|
@esgomezm I can load the 2d model now. It does not work in ilastik yet, but that's because neither keras nor tensorflow are working in our current release yet. |
|
This is good news! If you need to create any additional file to set up a python environment, just let me know |
|
@esgomezm we are making some progress with the python model runner, but now I am running into issues when trying to load the zero-cost models. Any ideas? |
|
Hi! I know, I think. Your model is zipped as defined in the bioimage specs. You need to unzip first to find the saved_model.pb. In these cases, it might be easier for you to load the Keras model (.h5). Here is the code I used to export the models, in case it helps: |
Ok, I will try to unzip and see what happens.
The Keras model fails with a different error: |
|
@esgomezm with unzipped weigths I get the following error when trying to load into tf: |
|
Ok, I guess the problem is that I am using tensorflow 2: Which version should I use for these models? |
the 2D unet has tensorflow_version: 1.15 set (the 3d one does not specify) makes me think we should make those fields mandatory and also add a mandatory pytorch_version for the pytorch weights... |
|
All the models are TF 1.15 as we cannot load TF 2 in ImageJ.
Yes, for Python environments at least seems unavoidable |
|
@esgomezm tensorflow 1.15 is pretty annoying as dependency, as it does not exist for most python versions, see Do you think it would also work with 14? Edit: also looks like conda does not have 1.15 at all: https://anaconda.org/conda-forge/tensorflow/files |
|
@constantinpape does TF 1.15.2 work? It should work with tf 1.14 but it seems weird because we take the latest 1.x version and it's always 1.1.5: |
It seems like 1.15 was not properly released, at least it's not really available via conda or pip. |
I have updated the 3D UNet model.yaml and rdf |
|
Ok, I have set up an environment with 1.14 now, but this yields a different set of errors: Keras Model: tensorflow model: |
|
@esgomezm do you have a |
|
It's a problem with the Keras version and TF 1. Try Sorry, I don't have the requirements list as everything is written down in the notebooks: https://github.com/HenriquesLab/ZeroCostDL4Mic/tree/master/Colab_notebooks/BioImage.io%20notebooks Also, keras backend was imported from tensorflow rather than importing keras directly. |
|
I updated the h5py version, now I get the following error for the keras model: tensorflow model still has the same error as before |
|
@esgomezm we have the same issues for the frunet model (https://raw.githubusercontent.com/deepimagej/models/master/fru-net_sev_segmentation/model.yaml). Do you have an environment file for that one? Without a reproducible env where we know that the models should run debugging this is pretty hopeless. (This also means that we need an env export for zerocost, otherwise the models will not be portable). |
|
Keras: my fault. I need to save the model without compiling it. Otherwise it complains about the custom loss function. For the tensorflow model, I manage to load it and run a prediction with the following code in a clean colab session using !pip install h5py==2.10
%tensorflow_version 1.x
import tensorflow as tf
import numpy as np
input_key = 'input' # name from the model.yaml
output_key = 'output' # name from the model.yaml
tag = tf.saved_model.tag_constants.SERVING # it's usually this one but otherwise it's given in config/deepimagej field in model.yaml.
graph2 = tf.Graph()
with graph2.as_default():
with tf.Session(graph=graph2) as sess:
# Restore saved values
print('\nRestoring...')
tf.saved_model.loader.load(
sess,
[tag],
"/content/tensorflow_saved_model_bundle"
)
# Get model input and output tensor names in the graph
x_tensor_name = signature[signature_key].inputs[input_key].name
y_tensor_name = signature[signature_key].outputs[output_key].name
# Get restored model input and output
input_tensor = graph2.get_tensor_by_name(x_tensor_name) # name of the input tensor in the model.yaml
output_tensor = graph2.get_tensor_by_name(y_tensor_name) # name of the input tensor in the model.yaml
# Random input dataset
input_data = np.random.rand(1, 256, 256, 8, 1)
input_data[:,150:200, 150:200] = 1
# Run prediction
output_array = sess.run(output_tensor, {input_tensor:input_data})The error you got is weird because it's looking for a keras model rather than the protobuffer ( For the FRU-Net it's also given in the notebook or the documentation (but you can avoid all except tensorflow): I hope this helps |
|
@esgomezm Ok, I understand the issue now: In tf1 we can't use the keras API to load (and predict with) the model saved as model bundle. (This is possible in tf2 and the preferred way when you read the doc). With the code snippet you posted above it should work, but the definitions of
Can you please update the code so that it's correct? |
|
Please, check this one: !pip install h5py==2.10
%tensorflow_version 1.x
import tensorflow as tf
import numpy as np
input_key = 'input' # name from the model.yaml
output_key = 'output' # name from the model.yaml
export_path = "/content/tensorflow_saved_model_bundle"
tag = tf.saved_model.tag_constants.SERVING # it's usually this one but otherwise it's given in config/deepimagej field in model.yaml.
signature_key = tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY # it's usually this one but otherwise it's given in config/deepimagej field in model.yaml.
graph = tf.Graph()
with graph.as_default():
with tf.Session() as sess:
# load the model
meta_graph_def = tf.saved_model.loader.load(
sess,
[tag],
export_path,
)
# Get the input-output dictionary
signature = meta_graph_def.signature_def
# Get model input and output tensor names in the graph
x_tensor_name = signature[signature_key].inputs[input_key].name
y_tensor_name = signature[signature_key].outputs[output_key].name
# Get restored model input and output
input_tensor = graph.get_tensor_by_name(x_tensor_name) # name of the input tensor in the model.yaml
output_tensor = graph.get_tensor_by_name(y_tensor_name) # name of the input tensor in the model.yaml
# Random input dataset
input_data = np.random.rand(1, 256, 256, 8, 1)
input_data[:,150:200, 150:200] = 1
# Run prediction
output_array = sess.run(output_tensor, {input_tensor:input_data}) |
|
Thanks, @esgomezm. I think everything is set up to work in principle now, however there are several issues with the models (both zerocost models and the models we use for testing):
Frunet keras backtrace: zerocost-keras: |
|
It's nice to hear that some are working :)
Yes, this model has a fixed shape, but I didn't know that the test tensor should be of that exact shape. If there's some halo, you may need to do tiling anyways.
Loading it using keras worked for me, so the problem is due to the dependencies most probably. I let you some code: from keras.models import load_model
model = load_model('fully_residual_dropout_segmentation.h5')The zerocost notebooks have all, except StarDist, the same problem I guess. I'll update all of them tomorrow and upload again. |
It would be better to avoid tiling for the test, because that adds more complexity (because there can be different tiling strategies). Thanks for the keras code and updating the zero cost models, I will give it another try tomorrow. |
This keras version is so old that it does not work with python 3.7+ on conda. Is there any chance to use a 2.X version? |
|
Ok, I can run the keras model now (but it fails with the usual size error); I am not sure if we can set up a reproducible environment for this though... |
|
@esgomezm I created a version of the zero-cost 3d unet trained on arabidopsis (https://bioimage.io/#/?id=10.5072%2Fzenodo.849110) now that partially works. The current state is: tensorflow prediction works, keras fails with: I assume that this is again some issue with the keras version or how the model was compiled. Anyway, I think that the prediction part of the python library works, now we need to fix the models and then make a new ilastik release that supports this. Most of the ilastik team will be gone for the rest of August. You can set up a development environment with the python library, if you want to work on this in the meantime: |
|
Hi @constantinpape! I'm playing with the python code. I got the following issue with the last version of the repository when I try to set up the development environment When trying to install tensorflow bioimageio.core back-end, I get the following message: I tried to test 3D UNet model for Arabidopsis dataset, using the development environment and I get the following message which I'm not sure whether it refers to errors in the model.yaml or the way I'm trying to test it: |
I see, this is a recent issue that is fixed already, but not yet merged on master. Can you try with the branch
Yes,
Yes, that issue is known, see #228. We will fix it. In the meantime you can circumvent it by passing a relative path instead of an absolute path to the model. |
|
@esgomezm I ran the validation for the tf zero cost model now and it still fails: I have mad a fixed rdf and will post it in gitter, in case it helps you to debug this. |
|
I installed the development environment from The same applies for |
for these entry points to work you need to install this bioimageio.spec package as well (with |
|
I installed it with pip and still doesn't work. It says |
|
No. Just note that I'm using the |
|
@esgomezm the correct command is (Edit and also @FynnBe I would like to not advertise the |
good point. I just didn't notice the additional dash and was puzzled that it didn't work for her...
this is the expected output though... but try |
We currently have two models exported from zero-cost in the model zoo:
Both do not pass validation with the current spec, see next comments.
The text was updated successfully, but these errors were encountered: