Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not read abnormally long HDF5 attribute (0.9.2-SNAPSHOT) #4744

Closed
blackredscarf opened this issue Mar 1, 2018 · 11 comments
Closed

Could not read abnormally long HDF5 attribute (0.9.2-SNAPSHOT) #4744

blackredscarf opened this issue Mar 1, 2018 · 11 comments
Labels
Bug Bugs and problems DL4J Keras Issues related to Keras import

Comments

@blackredscarf
Copy link

when I import a keras model,

val net: ComputationGraph = KerasModelImport
  .importKerasModelAndWeights("update.h5")

throw this error:

Exception in thread "main" org.deeplearning4j.nn.modelimport.keras.exceptions.UnsupportedKerasConfigurationException: Could not read abnormally long HDF5 attribute. Please file an issue at http://github.com/deeplearning4j/deeplearning4j/issues.
	at org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.readAttributeAsJson(Hdf5Archive.java:303)
	at org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.readAttributeAsJson(Hdf5Archive.java:107)
	at org.deeplearning4j.nn.modelimport.keras.utils.KerasModelBuilder.modelHdf5Filename(KerasModelBuilder.java:90)
	at org.deeplearning4j.nn.modelimport.keras.KerasModelImport.importKerasModelAndWeights(KerasModelImport.java:123)
	at pers.zjw.dl4j.snapshot.keras2.InceptionRest$.main(InceptionRest.scala:19)
	at pers.zjw.dl4j.snapshot.keras2.InceptionRest.main(InceptionRest.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
@raver119
Copy link
Contributor

raver119 commented Mar 1, 2018

Could you please add your .h5 model file as well? Without it it's impossible to tell/do something here.

@blackredscarf
Copy link
Author

blackredscarf commented Mar 2, 2018

@raver119
my model: model

@blackredscarf
Copy link
Author

@raver119
Do you find the reason?

@raver119
Copy link
Contributor

cc @saudet @maxpumperla

@maxpumperla
Copy link
Contributor

thanks @raver119, @blackredscarf I know what part throws this error and can have a look starting next week.

@maxpumperla maxpumperla self-assigned this Mar 17, 2018
@maxpumperla maxpumperla added the DL4J Keras Issues related to Keras import label Mar 17, 2018
@maxpumperla maxpumperla added the Bug Bugs and problems label Mar 29, 2018
@maxpumperla
Copy link
Contributor

Summoning @saudet :) this comes from our Hdf5Archive here:

https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-modelimport/src/main/java/org/deeplearning4j/nn/modelimport/keras/Hdf5Archive.java#L321-L323

Do you recall why we chose 100 here. It seems @blackredscarf has the problem or luxury of dealing with a very large HDF5 group. @saudet Any intuition on what a good new upper bound could be?

@maxpumperla
Copy link
Contributor

or better yet: an idea how to handle this a little more reliably (or even elegantly)?!

@saudet
Copy link
Contributor

saudet commented Mar 29, 2018

Actually that's from @turambar. I guess we can try to increase that to a 1000 maybe? :)

@maxpumperla
Copy link
Contributor

haha, good number, good number. I'll test it

@maxpumperla
Copy link
Contributor

@blackredscarf I've increased the HDF5 buffer limit, your problem is gone.

but: your model now complains that Lambda layers aren't supported (which is correct, see: #4704 )

@lock
Copy link

lock bot commented Sep 23, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Sep 23, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Bug Bugs and problems DL4J Keras Issues related to Keras import
Projects
None yet
Development

No branches or pull requests

4 participants