Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saving custom tensor fails #34908

Closed
awav opened this issue Dec 6, 2019 · 15 comments
Closed

Saving custom tensor fails #34908

awav opened this issue Dec 6, 2019 · 15 comments
Assignees
Labels
stale This label marks the issue/pr stale - to be closed automatically if no activity stat:awaiting response Status - Awaiting response from author

Comments

@awav
Copy link

awav commented Dec 6, 2019

Hello everyone,

There is a bug with saving and loading of custom tensors (variables) in TF. It was discovered at GPflow/GPflow#1127. I have created the MWE in colab here. There are two errors when tf.save_model applies to custom tensors:

  • The loading for custom tensor (variable) restores broken tf.Variable object
  • Saving of tf.Module with custom tensors (variables) fails

Details in colab notebook

PS: referencing @alextp and @dynamicwebpaige, because they know what I mean by custom tensor.

Thanks,
Artem

@awav awav changed the title Custom tensor saving fails Saving custom tensor fails Dec 6, 2019
@alextp
Copy link
Contributor

alextp commented Dec 6, 2019

@k-w-w as far as I can tell those module-visible variables should be correctly handled by savedmodel. Do you know what is going on here?

@awav
Copy link
Author

awav commented Dec 12, 2019

@k-w-w, @alextp ping ^^^

@k-w-w
Copy link
Contributor

k-w-w commented Dec 12, 2019

This is an interesting use case. SavedModel should be correctly handling this, but we hadn't considered this case yet. I'll see what changes are required to resolve the issue.

@awav
Copy link
Author

awav commented Dec 19, 2019

@k-w-w, @alextp any progress on the issue?

@awav
Copy link
Author

awav commented Jan 6, 2020

@k-w-w, @alextp ping

@awav
Copy link
Author

awav commented Feb 18, 2020

@k-w-w, @alextp Hello! Because of this issue, in GPflow we can't save models. My suspicion that it also affects TensorFlow Probability project. Can you make an update about it? Can you propose any workaround? Can you advise on how we can contribute to TensorFlow to make this work?
Thanks!

@awav
Copy link
Author

awav commented Mar 26, 2020

hello @k-w-w, @alextp! any updates?

@marcoadurno
Copy link

Hello @k-w-w @alextp , any updates on this?

@alextp
Copy link
Contributor

alextp commented Jul 1, 2020

Pinging @rohan100jain for FYI on something that might need a priority bump

@awav
Copy link
Author

awav commented Jul 9, 2020

@alextp thanks!

@nathanhack
Copy link

@alextp any idea when this might be fixed?

@alextp
Copy link
Contributor

alextp commented Jul 22, 2020

@ccrusius FYI

@Venkat6871
Copy link
Contributor

Hi,

Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.

The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.

Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.

@Venkat6871 Venkat6871 added the stat:awaiting response Status - Awaiting response from author label Jul 1, 2024
Copy link

github-actions bot commented Jul 9, 2024

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

@github-actions github-actions bot added the stale This label marks the issue/pr stale - to be closed automatically if no activity label Jul 9, 2024
Copy link

This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale This label marks the issue/pr stale - to be closed automatically if no activity stat:awaiting response Status - Awaiting response from author
Projects
None yet
Development

No branches or pull requests

6 participants