Branch: master
Find file History
allenlavoie and tensorflower-gardener Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac…
…kable

No API changes in this CL. Just more refactoring for a future API change.

PiperOrigin-RevId: 234242335
Latest commit bd36b48 Feb 16, 2019
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
model_utils Unprotect SignatureDef default input names. Feb 14, 2019
BUILD Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
README.md Update documentation for SavedModel tags. Sep 12, 2018
builder.py Write AssetFileDef to Metagraph's asset_file_def field. Nov 19, 2018
builder_impl.py Support exporting tables from tf.saved_model.save Dec 3, 2018
constants.py Skeleton loading for SavedModels in 2.x Dec 3, 2018
function_deserialization.py Add an imported representation of init ops for 1.x-style SavedModels Feb 14, 2019
function_serialization.py Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
load.py Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
load_test.py Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
load_v1_in_v2.py Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
load_v1_in_v2_test.py Add an imported representation of init ops for 1.x-style SavedModels Feb 14, 2019
loader.py Removing remove_undocumented calls from tensorflow/python. Apr 25, 2018
loader_impl.py Feed asset paths in tf.save_model.load when loading 1.x SavedModels Feb 13, 2019
loader_test.py Annotate tests as @run_v1_only Dec 8, 2018
main_op.py Removing remove_undocumented calls from tensorflow/python. Apr 25, 2018
main_op_impl.py Remove SavedModel functions from the 2.0 API. Nov 1, 2018
nested_structure_coder.py Simplify TensorShape v1/v2 switch (and fix bugs in it). Note that thi… Feb 14, 2019
nested_structure_coder_test.py Add proto encoder/decoder for general nested structures. Dec 19, 2018
revived_types.py Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
revived_types_test.py Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
save.py Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
save_test.py Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
saved_model.py Add tf.saved_model.save to the 2.x API (tf.saved_model.experimental.s… Nov 6, 2018
saved_model_test.py Wrap global_variables_initializer with self.evaluate() Dec 12, 2018
saved_object_graph.proto Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
signature_constants.py Delete TODO to change the signature constants to match the Keras Mode… Feb 13, 2019
signature_def_utils.py Save main_op and train_op in SignatureDefs instead of collections. Nov 29, 2018
signature_def_utils_impl.py Deprecate public SignatureDef symbols. Feb 14, 2019
signature_def_utils_test.py Annotate tests as @run_v1_only Dec 8, 2018
signature_serialization.py Rename Checkpointable -> Trackable and AutoCheckpointable -> AutoTrac… Feb 16, 2019
simple_save.py Remove SavedModel functions from the 2.0 API. Nov 1, 2018
simple_save_test.py Add @run_deprecated_v1 annotation to tests failing in v2 Nov 29, 2018
struct.proto Add proto encoder/decoder for general nested structures. Dec 19, 2018
tag_constants.py Fix TRANING -> TRAINING typo in v2.0 API. Jan 29, 2019
utils.py Add build_op_info to support defun caller op for TF internal use cases. Nov 8, 2018
utils_impl.py Remove a deprecation warning from tf.saved_model.save Jan 9, 2019
utils_test.py Mark certain tests in export_output_test as v1 only as export_output.… Jan 9, 2019

README.md

TensorFlow SavedModel

[TOC]

Overview

This document describes SavedModel, the universal serialization format for TensorFlow models.

SavedModel provides a language-neutral format to save machine-learned models that is recoverable and hermetic. It enables higher-level systems and tools to produce, consume and transform TensorFlow models.

Features

The following is a summary of the features in SavedModel:

  • Multiple graphs sharing a single set of variables and assets can be added to a single SavedModel. Each graph is associated with a specific set of tags to allow identification during a load or restore operation.
  • Support for SignatureDefs
    • Graphs that are used for inference tasks typically have a set of inputs and outputs. This is called a Signature.
    • SavedModel uses SignatureDefs to allow generic support for signatures that may need to be saved with the graphs.
    • For commonly used SignatureDefs in the context of TensorFlow Serving, please see documentation here.
  • Support for Assets.
    • For cases where ops depend on external files for initialization, such as vocabularies, SavedModel supports this via assets.
    • Assets are copied to the SavedModel location and can be read when loading a specific meta graph def.
  • Support to clear devices before generating the SavedModel.

The following is a summary of features that are NOT supported in SavedModel. Higher-level frameworks and tools that use SavedModel may provide these.

  • Implicit versioning.
  • Garbage collection.
  • Atomic writes to the SavedModel location.

Background

SavedModel manages and builds upon existing TensorFlow primitives such as TensorFlow Saver and MetaGraphDef. Specifically, SavedModel wraps a TensorFlow Saver. The Saver is primarily used to generate the variable checkpoints. SavedModel will replace the existing TensorFlow Inference Model Format as the canonical way to export TensorFlow graphs for serving.

Components

A SavedModel directory has the following structure:

assets/
assets.extra/
variables/
    variables.data-?????-of-?????
    variables.index
saved_model.pb
  • SavedModel protocol buffer
    • saved_model.pb or saved_model.pbtxt
    • Includes the graph definitions as MetaGraphDef protocol buffers.
  • Assets
    • Subfolder called assets.
    • Contains auxiliary files such as vocabularies, etc.
  • Extra assets
    • Subfolder where higher-level libraries and users can add their own assets that co-exist with the model, but are not loaded by the graph.
    • This subfolder is not managed by the SavedModel libraries.
  • Variables
    • Subfolder called variables.
    • Includes output from the TensorFlow Saver.
      • variables.data-?????-of-?????
      • variables.index

APIs

The APIs for building and loading a SavedModel are described in this section.

Builder

The SavedModel builder is implemented in Python.

The SavedModelBuilder class provides functionality to save multiple meta graph defs, associated variables and assets.

To build a SavedModel, the first meta graph must be saved with variables. Subsequent meta graphs will simply be saved with their graph definitions. If assets need to be saved and written or copied to disk, they can be provided when the meta graph def is added. If multiple meta graph defs are associated with an asset of the same name, only the first version is retained.

Tags

Each meta graph added to the SavedModel must be annotated with user specified tags, which reflect the meta graph capabilities or use-cases. More specifically, these tags typically annotate a meta graph with its functionality (e.g. serving or training), and possibly hardware specific aspects such as GPU. In the SavedModel, the meta graph def whose tag-set exactly matches those specified in the loader API, will be the one loaded by the loader. If no meta graph def is found matching the specified tags, an error is returned. For example, a loader with a requirement to serve on GPU hardware would be able to load only meta graph annotated with tags='serve,gpu' by specifying this set of tags in tensorflow::LoadSavedModel(...).

Usage

The typical usage of builder is as follows:

export_dir = ...
...
builder = tf.saved_model.builder.SavedModelBuilder(export_dir)
with tf.Session(graph=tf.Graph()) as sess:
  ...
  builder.add_meta_graph_and_variables(sess,
                                       [tf.saved_model.tag_constants.TRAINING],
                                       signature_def_map=foo_signatures,
                                       assets_collection=foo_assets)
...
with tf.Session(graph=tf.Graph()) as sess:
  ...
  builder.add_meta_graph(["bar-tag", "baz-tag"])
...
builder.save()

Stripping Default valued attributes

The SavedModelBuilder class allows users to control whether default-valued attributes must be stripped from the NodeDefs while adding a meta graph to the SavedModel bundle. Both SavedModelBuilder.add_meta_graph_and_variables and SavedModelBuilder.add_meta_graph methods accept a Boolean flag strip_default_attrs that controls this behavior.

If strip_default_attrs is False, the exported MetaGraphDef will have the default valued attributes in all it's NodeDef instances. This can break forward compatibility with a sequence of events such as the following:

  • An existing Op (Foo) is updated to include a new attribute (T) with a default (bool) at version 101.
  • A model producer (such as a Trainer) binary picks up this change (version 101) to the OpDef and re-exports an existing model that uses Op Foo.
  • A model consumer (such as Tensorflow Serving) running an older binary (version 100) doesn't have attribute T for Op Foo, but tries to import this model. The model consumer doesn't recognize attribute T in a NodeDef that uses Op Foo and therefore fails to load the model.

By setting strip_default_attrs to True, the model producers can strip away any default valued attributes in the NodeDefs. This helps ensure that newly added attributes with defaults don't cause older model consumers to fail loading models regenerated with newer training binaries.

TIP: If you care about forward compatibility, then set strip_default_attrs to True while using SavedModelBuilder.add_meta_graph_and_variables and SavedModelBuilder.add_meta_graph.

Loader

The SavedModel loader is implemented in C++ and Python.

Python

The Python version of the SavedModel loader provides load and restore capability for a SavedModel. The load operation requires the session in which to restore the graph definition and variables, the tags used to identify the meta graph def to load and the location of the SavedModel. Upon a load, the subset of variables and assets supplied as part of the specific meta graph def, will be restored into the supplied session.

export_dir = ...
...
with tf.Session(graph=tf.Graph()) as sess:
  tf.saved_model.loader.load(sess, [tag_constants.TRAINING], export_dir)
  ...

C++

The C++ version of the SavedModel loader provides an API to load a SavedModel from a path, while allowing SessionOptions and RunOptions. Similar to the Python version, the C++ version requires the tags associated with the graph to be loaded, to be specified. The loaded version of SavedModel is referred to as SavedModelBundle and contains the meta graph def and the session within which it is loaded.

const string export_dir = ...
SavedModelBundle bundle;
...
LoadSavedModel(session_options, run_options, export_dir, {kSavedModelTagTrain},
               &bundle);

Constants

SavedModel offers the flexibility to build and load TensorFlow graphs for a variety of use-cases. For the set of most common expected use-cases, SavedModel's APIs provide a set of constants in Python and C++ that are easy to reuse and share across tools consistently.

Tag constants

Sets of tags can be used to uniquely identify a MetaGraphDef saved in a SavedModel. A subset of commonly used tags is specified in:

Signature constants

SignatureDefs are used to define the signature of a computation supported in a TensorFlow graph. Commonly used input keys, output keys and method names are defined in: