Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

syntaxnet: Enable building Parsey McParseface in one model to be served in TF serving #250

Closed
wants to merge 16 commits into from
Closed

syntaxnet: Enable building Parsey McParseface in one model to be served in TF serving #250

wants to merge 16 commits into from

Commits on Jul 22, 2016

  1. Add ability to feed (pre-parsed) Sentence protos to SentenceBatch 

    SentenceBatch accepts a vector of Sentence protos to use instead
of reading from the configured corpus. When using a feed, the corpus
will be ignored and only the fed sentences will be used. Supports
"rewind" on the feed.
    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    65d4b2a View commit details
    Browse the repository at this point in the history

  2. Add an input Tensor to BeamParseReader to provide serialised sentence… 

    … protos to parse

Add an input Tensor to BeamParseReader to provide serialised sentence protos to parse.
Documents may come from the input tensor (e.g. read by a DocumentSource op)
or read directly from the corpus as defined in the task context. The decision
of the approach should is explicit based on attributes during graph configuration.
    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    ab7c5f9 View commit details
    Browse the repository at this point in the history

  3. Add ability to read text from a vector of strings in TextReader. 

    If constructed with a vector of strings, TextReader will use a
new "RandomAccessFile" implementation, VectorIn (cloned from the
StdIn implementation) to read the strings from the vector. The
format of the strings is still determined by the corpus configured
in the task context.
    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    3675f62 View commit details
    Browse the repository at this point in the history

  4. Add the ability to pass text in a vector to DocumentSource

    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    4222147 View commit details
    Browse the repository at this point in the history

  5. Add an (unused) input value to the DocumentSource op as required 

    The DocumentSource requires a text input tensor to be provided, even when
the text is to be read from the corpus, because tensors cannot be
optional. The value of the tensor is unused.
    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    5ee1ede View commit details
    Browse the repository at this point in the history

  6. Be explicit that input documents come from tensor

    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    ce7e726 View commit details
    Browse the repository at this point in the history

  7. Allow the syntaxnet ops to be used by other workspaces

    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    249faaf View commit details
    Browse the repository at this point in the history

  8. Modify graph builder to take optional document_source at build time 

    If the document source is provided, it will be used and the parser will
not read from the corpus. If it is provided, the document protos will be
consumed from the provided op and the configured corpus will be ignored
    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    ebd85d3 View commit details
    Browse the repository at this point in the history

  9. Remove some extra debugging

    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    1793109 View commit details
    Browse the repository at this point in the history

  10. Script which replaces demo.sh for running Parsey McParseface model 

    This new script replaces demo.sh for running Parsey McParseface model.
It takes a different approach from demo.sh which is that it builds the
entire operation chain (tagging AND parsing) in a single model, complete
with a document source and document sink. The script also exports the model
in various ways, including using the TF Serving exporter, such that the
model can be served using TF Serving
    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    7405212 View commit details
    Browse the repository at this point in the history

  11. Add parsey_mcparseface.py to BUILD

    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    0efc132 View commit details
    Browse the repository at this point in the history

  12. Add cmdline option to export (only) the PMP model

    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    cdf0560 View commit details
    Browse the repository at this point in the history

  13. Avoid specifying a corpus that isn't used/read

    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    7d59fe7 View commit details
    Browse the repository at this point in the history

  14. Improve comment

    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    1c37544 View commit details
    Browse the repository at this point in the history

  15. Fix two tests that use document_source

    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    f1804e3 View commit details
    Browse the repository at this point in the history

  16. Include the "assets" (required files for loading the model) in the ex… 

    …port
    @dmansfield
    dmansfield committed Jul 22, 2016
    Configuration menu
    Copy the full SHA
    2a7f1c3 View commit details
    Browse the repository at this point in the history