Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SameDiff: Serialization (/loading flatbuffers in Java) #5759

AlexDBlack opened this issue Jun 30, 2018 · 3 comments


Copy link

commented Jun 30, 2018

Use case: I want to save/serialize a SameDiff instance graph/op structure - without the arrays - and in a format that won't break on every version upgrade.

(Why: this would allow what I would call "true" lambda layers in DL4J: a neat but non-critical feature... defining no-params layers as a lambda class should be possible now (pending a PR merge) but the JSON serialization clearly won't work here. However, if I can write a custom JSON serializer that handles the SameDiff instance serialization, even in a binary format - that would be fine).

Currently, the only serialization format we support is FlatBuffers - via SameDiff.asFlatBuffers.
However, this is inadequate for the above use case for 2 reasons:

  1. Arrays are always included
  2. As far as I can see: there is no way to create a new SameDiff instance from a serialized FlatBuffers version.

@AlexDBlack AlexDBlack added this to the SameDiff Ops and Next Steps milestone Jun 30, 2018


This comment has been minimized.

Copy link
Contributor Author

commented Jun 30, 2018

Discussed in dev channel: adding support to load the FlatBuffers format we already have seems like a reasonable way forward here.

Adding support to not serialize the arrays should also be easy.

@AlexDBlack AlexDBlack changed the title SameDiff: Serialization SameDiff: Serialization (/loading flatbuffers in Java) Jun 30, 2018


This comment has been minimized.

Copy link

commented Jun 30, 2018

Arrays are always included mostly due to const existence. Arrays, which are not outputs of existing ops must be preserved.

@raver119 raver119 modified the milestones: SameDiff Ops and Next Steps, DL4J 1.0.0 Release Aug 1, 2018

@raver119 raver119 added the Java label Aug 1, 2018

@AlexDBlack AlexDBlack self-assigned this Sep 24, 2018


This comment has been minimized.

Copy link

commented Nov 2, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Nov 2, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet
2 participants
You can’t perform that action at this time.