Explicitly track function input arg expansion into Placeholders,
and keep metadata to map between FunctionDef and GraphDef connectivity
formats.

PiperOrigin-RevId: 192462592

PiperOrigin-RevId: 192463583

PiperOrigin-RevId: 192463997

…les.get_or_create_global_step().

PiperOrigin-RevId: 192476077

PiperOrigin-RevId: 192479630

PiperOrigin-RevId: 192480452

PiperOrigin-RevId: 192481690

PiperOrigin-RevId: 192482242

…taset_op_test to "medium".

PiperOrigin-RevId: 192484895

PiperOrigin-RevId: 192491201

PiperOrigin-RevId: 192491335

PiperOrigin-RevId: 192498471

PiperOrigin-RevId: 192499401

…` in BN (not yet supported by TPU compilation).

PiperOrigin-RevId: 192502020

…. Choosing not to do this at static analysis because it exposes the scope to any node, making it easier to use by any specialization of a transformer.

PiperOrigin-RevId: 192502309

https://github.com/tensorflow/hub/tree/master/examples/image_retraining
has the same tool, upgraded to use TensorFlow Hub instead of raw graph defs.

PiperOrigin-RevId: 192502469

PiperOrigin-RevId: 192503204

PiperOrigin-RevId: 192504411

…_jacobian` methods to take event_ndims.

The class level event_ndims parameter is being deprecated in favor of passing it in
to the `log_det_jacobian` methods.

Specific changes:

  - `log_det_jacobian` signatures are now `log_det_jacobian(input, event_ndims)`

  - Constructors no long have event_ndims passed in (e.g. Affine() vs. Affine(event_ndims=0)).

  - All bijectors must specify a subset of [forward_min_event_ndims, inverse_min_event_ndims]. This is the minimal dimensionality the bijector operates on, with it being "broadcasted" to any passed in event_ndims (e.g. Exp has forward_min_event_ndims = 0. That means it operates on scalars. However, we can use the bijector on any event_ndims > 0 (i.e. we've broadcasted the transformation to work on any amount of event_ndims > 0), and jacobian reduction will work in those cases.

As a result of this change, all bijectors should "broadcast" (e.g. Sigmoid now works on any number of event_ndims).

Other changes (internal and documentation):
  - Added clarifications on Jacobian Determinant vs. Jacobian Matrix.
  - Added clarifications on min_event_ndims, and what the jacobian reduction is over.
  - Changed caching of ildj to be keyed on event_ndims.
  - Several bug fixes to bugs unearthed while writing this code (e.g. transformed distribution shape computation being incorrect)

PiperOrigin-RevId: 192504919

…n by mutating

NodeDef before creating the copied operation.

PiperOrigin-RevId: 192505209

PiperOrigin-RevId: 192511036

PiperOrigin-RevId: 192516190

…e updated checkpoints from the contents.

PiperOrigin-RevId: 192517819

…able in the metagraph.

PiperOrigin-RevId: 192518307

PiperOrigin-RevId: 192519639

PiperOrigin-RevId: 192520351

… trees estimators.

PiperOrigin-RevId: 192521398

data are saved with host prefix.

PiperOrigin-RevId: 192523668

Implement a vectorized way to compute the same thing instead.

PiperOrigin-RevId: 192524667

PiperOrigin-RevId: 192530231

…-TRAIN modes too.

PiperOrigin-RevId: 192532297

PiperOrigin-RevId: 192534931

PiperOrigin-RevId: 192536085

PiperOrigin-RevId: 192537874

This is a starting point for Dataset-compatible weighted sampling across a list of datasets.

PiperOrigin-RevId: 192540412

…d Keras variables(which assumes there exists a session). Otherwise create a session so that we can run get_config().

Actually fix #18193.

PiperOrigin-RevId: 192541442

…"medium".

PiperOrigin-RevId: 192542956

Also set the sharding to the instruction when created from proto.

PiperOrigin-RevId: 192543024

See:
https://source.cloud.google.com/results/invocations/31632a30-3728-4635-a456-f89b9e8b9dfe/log
PiperOrigin-RevId: 192544848

CreateFromProto requires that the config has proper entry_computation_layout, so give the config the program shape.

PiperOrigin-RevId: 192546316

PiperOrigin-RevId: 192546579

PiperOrigin-RevId: 192548367

PiperOrigin-RevId: 192548652

PiperOrigin-RevId: 192549109

PiperOrigin-RevId: 192550101

PiperOrigin-RevId: 192560111

…ses.

PiperOrigin-RevId: 192591402

…ows top-level name scopes to begin with underscores. Also use the transformer scope information to get to the enclosing function name.

PiperOrigin-RevId: 192600256

… the original and optimized graphs and check if the outputs are same.

PiperOrigin-RevId: 192616402

PiperOrigin-RevId: 192624191

BaseCollectiveExecutor and RingReducer.

This change is part of a series of changes introducing infrastructure
for collective ops and initial implementations of reduction and broadcast.

PiperOrigin-RevId: 192624521

…ing random data and outfeeding the data generated at each step.

The arguments of the computation can be either read from the session module, or randomly generated.
The tool uses the raw transfer manager API to infeed and outfeed the data.

PiperOrigin-RevId: 192628605

Also modify code to ensure aliased forwarding happens whenever
possible with DT_VARIANT objects in ResourceVariables and in the new op.

PiperOrigin-RevId: 192632202

…their dtype.

PiperOrigin-RevId: 192634133

…les.

PiperOrigin-RevId: 192637289

PiperOrigin-RevId: 192640621

PiperOrigin-RevId: 192643127

…sting no-speech clips

PiperOrigin-RevId: 192644704

PiperOrigin-RevId: 192644946

PiperOrigin-RevId: 192645769

PiperOrigin-RevId: 192646199

Doesn't add to the public API yet, just shifts code around. Changes:
  - A tiny bit of renaming (to avoid having _Checkpoint and Checkpoint in the same file)
  - Removed the garbage collection decorator from a few tests due to the uuid4() garbage issue (apparently core tests get run on Python 2.7.9?)
  - Renamed "Object" to "CheckpointableObject" in the proto, since core protos have Java bindings and apparently Java had something else in mind for the keyword "Object" :)
but otherwise this is a pure move.

After this CL I'll propose adding tf.train.Checkpoint to the API (currently tf.contrib.eager.Checkpoint), move the utilities that are still in contrib/eager to their own contrib directory (there will be a few more misc. utilities for inspecting checkpoints and managing dependencies), get tf.train.Saver to read object-based checkpoints for compatibility, and work on Model.save_weights/load_weights.

PiperOrigin-RevId: 192646890

PiperOrigin-RevId: 192647818

PiperOrigin-RevId: 192648596

PiperOrigin-RevId: 192649052

PiperOrigin-RevId: 192651583

element_length_func. Current code requires tf.int32 in order to compare with
buckets_min which is int32.

PiperOrigin-RevId: 192652917

NOTE: libjpeg-turbo 1.5.90 also exists, which adds AVX2 SIMD support. However
it also comes with a CMake build rewrite and 35 micro-architecture specialized
nasm files for x86_64 alone. We do not have the cycles to update jpeg.BUILD to
incorporate those changes at this time. If anyone wants to try, please note
we'd need predicates such as the following:

config_setting(
    name = "haswell_opt",
    values = {
        "cpu": "haswell",  # First Intel chip with AVX2
        "compilation_mode": "opt",
    },
    visibility = ["//visibility:public"],
)

config_setting(
    name = "excavator_opt",
    values = {
        "cpu": "excavator",  # First AMD chip with AVX2
        "compilation_mode": "opt",
    },
    visibility = ["//visibility:public"],
)

PiperOrigin-RevId: 192655533

… used in summary utils (which training depends on, thus causing circular dependency).

PiperOrigin-RevId: 192656997

PiperOrigin-RevId: 192659701

PiperOrigin-RevId: 192659759

…spose.

PiperOrigin-RevId: 192670991

As LayerCollections are required to instantiate KfacOptimizer and FisherEstimator, a deprecation warning is printed upon instantiating LayerCollection.

PiperOrigin-RevId: 192671370

In some versions of Python, the Session._session field may be cleared
(in `Session.__del__()`) before a callable that has a reference to
that Session is deleted. Add a defensive check in the
`Session._Callable.__del__()` method.

PiperOrigin-RevId: 192679796

Create spearate targets for each submodule, so that each test can depend on
the appropriate subset of Python files.

PiperOrigin-RevId: 192679856

PiperOrigin-RevId: 192683166

PiperOrigin-RevId: 192689747

PiperOrigin-RevId: 192691078

… enables eager execution in validate_args=True contexts.

PiperOrigin-RevId: 192693458

…ayout assignment.

PiperOrigin-RevId: 192693972

A custom-call-conv should have four arguments: lhs, rhs, algorithm, and
use-tensor-cores.  CudnnAlgorithmPicker did the right thing, and that
path is exercised 99% of the time.  But CudnnAlgorithmPicker can fail,
and if it does, we're stuck with whatever we had before.  What we had
before only had three of the four args, which is bad.

In addition to fixing it, added an e2e test that catches the bug.

PiperOrigin-RevId: 192694119

PiperOrigin-RevId: 192694244

Impact on time per optimizer on inception graph:

model_pruner:          590 ms -> 550 ms   (-7%)
function_optimizer:    130 ms -> 130 ms   (-0%)
constant_folding:     7600 ms -> 7550 ms  (-0.7%)
arithmetic_optimizer: 1860 ms -> 1550 ms  (-20%)
loop_optimizer:        320 ms -> 320 ms   (-0%)
dependency_optimizer: 1300 ms -> 720 ms   (-45%)
layout:               1400 ms -> 1400 ms  (-0%)
memory_optimizer:     4200 ms -> 3540 ms  (-16%)
PiperOrigin-RevId: 192694528

PiperOrigin-RevId: 192694794

…ompiler/tf2xla/kernels.

Enable type DT_STRING for AssertOp and ConstOp, in order to make dummy Assert compile with a const string (assert message) as its input.

PiperOrigin-RevId: 192695938

PiperOrigin-RevId: 192698931

PiperOrigin-RevId: 192699277

… fixes a regression introduced by cl/176147440.

PiperOrigin-RevId: 192702279

PiperOrigin-RevId: 192703924

PiperOrigin-RevId: 192704808

…since it instantiates a placeholder_with_default, which is not supported by TPU compilation.

PiperOrigin-RevId: 192705478

…eras backend.

PiperOrigin-RevId: 192707345

PiperOrigin-RevId: 192708480

PiperOrigin-RevId: 192711203

PiperOrigin-RevId: 192711501

PiperOrigin-RevId: 192718697

…r,Platform}.

These types live inside StreamExecutor's namespace, but they are
specific to XLA.  Therefore they either shouldn't live in SE's namespace
or should have "XLA" in the name.

Moving them out of SE's namespace is ugly, because almost every type
used inside of these headers then needs to be qualified.  So name-change
it is.

This patch was generated by a mechanical find/replace.

PiperOrigin-RevId: 192724238

-- Refactor utility functions into pruning_utils.py and add tests

PiperOrigin-RevId: 192727737

PiperOrigin-RevId: 192728635

PiperOrigin-RevId: 192751211

PiperOrigin-RevId: 192752798