Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SameDiff: add TF import support for (user defined functions?) in pb #7178

Closed
AlexDBlack opened this issue Feb 16, 2019 · 2 comments

Comments

@AlexDBlack
Copy link
Contributor

commented Feb 16, 2019

Here's an odd section from GPT-2 117M parameter model protobuf (text format):

library {
  function {
    signature {
      name: "convert_gradient_to_tensor_8rf1PQaJCP4"
      input_arg {
        name: "x_0"
        type: DT_FLOAT
      }
      output_arg {
        name: "x"
        type: DT_FLOAT
      }
      description: "Force gradient to be a dense tensor.\n\n    It\'s often faster to do dense embedding gradient on GPU than sparse on CPU.\n    "
    }
    ret {
      key: "x"
      value: "x_0"
    }
  }
}

Model is from https://github.com/openai/gpt-2 - download_model.sh and then manually frozen.

I'm not exactly sure what's going on here, but as far as I can tell from looking at the model, it's basically an identity operation other than the fact that it presumably changes array location from CPU to GPU?

If that's the case, then we can probably just add a mechanism like we have for Keras import in DL4J: the user specifies how the UDF should be handled (what it should be mapped to).

@treo

This comment has been minimized.

@lock

This comment has been minimized.

Copy link

commented Mar 20, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 20, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
2 participants
You can’t perform that action at this time.