Latest commit e5e91b4 Aug 17, 2017 @tensorflower-gardener tensorflower-gardener committed with tensorflower-gardener BUILD cleanup in tensorflow/python/...
PiperOrigin-RevId: 165530192
Permalink
..
Failed to load latest commit information.
BUILD BUILD cleanup in tensorflow/python/... Aug 17, 2017
README.md fix path to saved model constant in docs. Jul 7, 2017
builder.py Expose saved_model via tf.saved_model. Jan 12, 2017
builder_impl.py Comment update for main-op in the SavedModel builder APIs. Aug 9, 2017
constants.py SavedModel updates for TensorInfo validation and exposing existing AP… Jan 31, 2017
loader.py Expose saved_model via tf.saved_model. Jan 12, 2017
loader_impl.py Include link to SavedModel CLI in the Py loader when tag-set is not f… Jun 20, 2017
main_op.py SavedModel updates for TensorInfo validation and exposing existing AP… Jan 31, 2017
main_op_impl.py Organize the lookup table ops into it's own lookup_ops.cc file instea… May 4, 2017
saved_model.py SavedModel updates for TensorInfo validation and exposing existing AP… Jan 31, 2017
saved_model_test.py Add a tag constant, gpu, to present graph with GPU support. Jul 7, 2017
signature_constants.py Expose saved_model via tf.saved_model. Jan 12, 2017
signature_def_utils.py Expose saved_model via tf.saved_model. Jan 12, 2017
signature_def_utils_impl.py Bugfix: Don't ignore logical names in single tensor predict signature… Jun 9, 2017
signature_def_utils_test.py Remove hourglass imports from even more tests Dec 17, 2016
tag_constants.py Add a tag constant, gpu, to present graph with GPU support. Jul 7, 2017
utils.py Support SparseTensors transparently in saved_model.utils: Jul 7, 2017
utils_impl.py Support SparseTensors transparently in saved_model.utils: Jul 7, 2017
utils_test.py Simplify saved_model/utils_test by using tf.sparse_placeholder() as a… Aug 15, 2017

README.md

TensorFlow SavedModel

[TOC]

Overview

This document describes SavedModel, the universal serialization format for TensorFlow models.

SavedModel provides a language-neutral format to save machine-learned models that is recoverable and hermetic. It enables higher-level systems and tools to produce, consume and transform TensorFlow models.

Features

The following is a summary of the features in SavedModel:

  • Multiple graphs sharing a single set of variables and assets can be added to a single SavedModel. Each graph is associated with a specific set of tags to allow identification during a load or restore operation.
  • Support for SignatureDefs
    • Graphs that are used for inference tasks typically have a set of inputs and outputs. This is called a Signature.
    • SavedModel uses SignatureDefs to allow generic support for signatures that may need to be saved with the graphs.
    • For commonly used SignatureDefs in the context of TensorFlow Serving, please see documentation here.
  • Support for Assets.
    • For cases where ops depend on external files for initialization, such as vocabularies, SavedModel supports this via assets.
    • Assets are copied to the SavedModel location and can be read when loading a specific meta graph def.
  • Support to clear devices before generating the SavedModel.

The following is a summary of features that are NOT supported in SavedModel. Higher-level frameworks and tools that use SavedModel may provide these.

  • Implicit versioning.
  • Garbage collection.
  • Atomic writes to the SavedModel location.

Background

SavedModel manages and builds upon existing TensorFlow primitives such as TensorFlow Saver and MetaGraphDef. Specifically, SavedModel wraps a TensorFlow Saver. The Saver is primarily used to generate the variable checkpoints. SavedModel will replace the existing TensorFlow Inference Model Format as the canonical way to export TensorFlow graphs for serving.

Components

A SavedModel directory has the following structure:

assets/
assets.extra/
variables/
    variables.data-?????-of-?????
    variables.index
saved_model.pb
  • SavedModel protocol buffer
    • saved_model.pb or saved_model.pbtxt
    • Includes the graph definitions as MetaGraphDef protocol buffers.
  • Assets
    • Subfolder called assets.
    • Contains auxiliary files such as vocabularies, etc.
  • Extra assets
    • Subfolder where higher-level libraries and users can add their own assets that co-exist with the model, but are not loaded by the graph.
    • This subfolder is not managed by the SavedModel libraries.
  • Variables
    • Subfolder called variables.
    • Includes output from the TensorFlow Saver.
      • variables.data-?????-of-?????
      • variables.index

APIs

The APIs for building and loading a SavedModel are described in this section.

Builder

The SavedModel builder is implemented in Python.

The SavedModelBuilder class provides functionality to save multiple meta graph defs, associated variables and assets.

To build a SavedModel, the first meta graph must be saved with variables. Subsequent meta graphs will simply be saved with their graph definitions. If assets need to be saved and written or copied to disk, they can be provided when the meta graph def is added. If multiple meta graph defs are associated with an asset of the same name, only the first version is retained.

Tags

Each meta graph added to the SavedModel must be annotated with user specified tags. The tags provide a means to identify the specific meta graph to load and restore, along with the shared set of variables and assets. These tags typically annotate a MetaGraph with it's functionality (e.g. serving or training), and possibly hardware specific aspects such as GPU.

Usage

The typical usage of builder is as follows:

export_dir = ...
...
builder = tf.saved_model.builder.SavedModelBuilder(export_dir)
with tf.Session(graph=tf.Graph()) as sess:
  ...
  builder.add_meta_graph_and_variables(sess,
                                       [tf.saved_model.tag_constants.TRAINING],
                                       signature_def_map=foo_signatures,
                                       assets_collection=foo_assets)
...
with tf.Session(graph=tf.Graph()) as sess:
  ...
  builder.add_meta_graph(["bar-tag", "baz-tag"])
...
builder.save()

Loader

The SavedModel loader is implemented in C++ and Python.

Python

The Python version of the SavedModel loader provides load and restore capability for a SavedModel. The load operation requires the session in which to restore the graph definition and variables, the tags used to identify the meta graph def to load and the location of the SavedModel. Upon a load, the subset of variables and assets supplied as part of the specific meta graph def, will be restored into the supplied session.

export_dir = ...
...
with tf.Session(graph=tf.Graph()) as sess:
  tf.saved_model.loader.load(sess, [tag_constants.TRAINING], export_dir)
  ...

C++

The C++ version of the SavedModel loader provides an API to load a SavedModel from a path, while allowing SessionOptions and RunOptions. Similar to the Python version, the C++ version requires the tags associated with the graph to be loaded, to be specified. The loaded version of SavedModel is referred to as SavedModelBundle and contains the meta graph def and the session within which it is loaded.

const string export_dir = ...
SavedModelBundle bundle;
...
LoadSavedModel(session_options, run_options, export_dir, {kSavedModelTagTrain},
               &bundle);

Constants

SavedModel offers the flexibility to build and load TensorFlow graphs for a variety of use-cases. For the set of most common expected use-cases, SavedModel's APIs provide a set of constants in Python and C++ that are easy to reuse and share across tools consistently.

Tag constants

Sets of tags can be used to uniquely identify a MetaGraphDef saved in a SavedModel. A subset of commonly used tags is specified in:

Signature constants

SignatureDefs are used to define the signature of a computation supported in a TensorFlow graph. Commonly used input keys, output keys and method names are defined in: