Branch 188540944 #17603
Merged
Branch 188540944 #17603
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PiperOrigin-RevId: 188263046
PiperOrigin-RevId: 188263337
PiperOrigin-RevId: 188264080
…iltins_test, which somehow got left out. PiperOrigin-RevId: 188264644
This lets you use Dimension objects in numerical computations; e.g., it lets you evaluate expressions like 3 + my_tensor.shape[0] when executing eagerly. At time of writing, without this change, `matplotlib.pyplot.plt(my_tensor, my_other_tensor)` fails when executing eagerly, but it works with this change. PiperOrigin-RevId: 188265500
… consumers.. PiperOrigin-RevId: 188267649
…ngs. PiperOrigin-RevId: 188270727
PiperOrigin-RevId: 188272354
PiperOrigin-RevId: 188273641
PiperOrigin-RevId: 188284335
In libc++ std::map and std::multimap call the comparison functor from a const object, which requires the `operator()` to be a const method. PiperOrigin-RevId: 188285407
PiperOrigin-RevId: 188293315
…ix powers of the approximate Fisher
- Added multi-tower support to multi/RNN fully connected layers
- All op creation is now done inside functions that explicitly create ops, thus allowing fine control of their placement. One result of this is that we no longer need any colocation statements (and these have been removed)
- Multi-tower computations are now handled using ParitionedTensor class, which appears to be a single tensor to the FisherFactors but actually contains a list of tensors.
- To achieve the above damping values are passed around as special functions that are packaged along with "ids" that can be used to uniquely identify the computation they perform. Topohash might provide a better solution for this in the future.
- Variable creation in the factors is now done via special methods so we can have fine control over where these are placed
- FisherEstimator now has special functions to create ops and variables using different placement strategies (currently: no strategy, round-robin, and as thunks). By default this will use the round-robin strategy and manufacture the usual convenience properties ("inv_update_ops", etc). This default behavior is to preserve backwards compatibility but in the future we should deprecate this and require the user to ask for an explicit strategy.
- LossFunctions no longer make any ops in their constructors. The only make ops when evaluated. LayerCollection maintains a list of tensors/ops which we can colocate LossFunction computations with (typically their inputs)
- LossFunctions no longer support multi-tower/mini-batches directly. Instead LayerCollection maintains a list of these objects, one for each tower. This solution is better since now the loss function related computations can take place exclusively on the corresponding tower.
- All loss functions now support multiple towers/minibatches (via LayerCollection).
- tf.gradients is passed list of loss function values instead of their sum, which will prevent extraneous gradient ops being placed on arbitrary devices. Hopefully with this change and the above one for loss functions all ops associated with gradient computations (for computing stats) will occur completely on the device that defines that part of the graph. e.g. this will do the right thing for multiple towers
- I've also made sure that sensible colocation occurs for the extra ops needed by the curvature_propagation and exact estimation modes.
- Variables and ops made by FisherEstimator are now placed inside of name scopes (based on the name given to FisherEstimator)
- Restored old variable use count tracker implementation, thus fixing the issue with how generic registrations were handled by check_registration().
- Restored interface to FisherEstimator (which was changed in the previous CL).
- Fixed bug in LazyKFacOptimizer: optional/named arguments weren't being passed in properly
- Lots of other minor refactors/improvements
PiperOrigin-RevId: 188310846
…d repeated staging. PiperOrigin-RevId: 188316160
…sponding script. PiperOrigin-RevId: 188324090
This requires adding a special case to SourceIndexOfBitcast if the bitcast is a reshape. PiperOrigin-RevId: 188324197
PiperOrigin-RevId: 188327338
PiperOrigin-RevId: 188332858
PiperOrigin-RevId: 188335233
…s logic of launching an XLA computation. Also changes the resource variable container from a std::vector<OptionalTensor> to a std::map<int, OptionalTensor> in preparation for backends where the resource variables aren't ordered densely at the end of the argument list. PiperOrigin-RevId: 188335574
PiperOrigin-RevId: 188339438
In the common case of clean termination, we can avoid performing several atomic operations and allocations. PiperOrigin-RevId: 188339594
PiperOrigin-RevId: 188343238
PiperOrigin-RevId: 188353354
…orward compatibility around TF op attributes. PiperOrigin-RevId: 188359164
Adds initialization methods to Platform. Some platforms require initialization. Those that do not have trivial implementations of these methods. PiperOrigin-RevId: 188363315
…n all of the cases that are changed by this CL, a failure indicates a software bug, not a runtime condition that should be handled and continued beyond. Continuing to execute only promotes silently-ignored bugs. I also removed the useless call which attempts to set the HTTP protocol to HTTP/2, because this call always fails. I opened b/74351157 to track the possible feature of adding support for HTTP/2. Also simplified the code around constructing the error string when returning actual Status objects, by moving code into a lambda. PiperOrigin-RevId: 188363531
I need something like this for my Gather HLO->HLO lowering pass. PiperOrigin-RevId: 188365102
PiperOrigin-RevId: 188503184
…ency on eigen!) PiperOrigin-RevId: 188504172
Fusing GTE works, but it's slower than not fusing. (In some sense, GTE is *always* fused; it's just that our "implicit fusion" implementation is faster than our explicit fusion implementation.) PiperOrigin-RevId: 188509801
PiperOrigin-RevId: 188512706
…sts. PiperOrigin-RevId: 188513895
…h a runner to execute kernels with. In that case, it defaults to using the threadpool provided by the device. Also makes sure each device has a default threadpool to fall back on. PiperOrigin-RevId: 188520648
…to third_party/tensorflow/python/training (move WarmStartSettings definition to third_party/tensorflow/python/estimator/estimator.py), and make _warm_start() public under tf.train.warm_start(). WarmStartSettings and VocabInfo are both available under tf.estimator, and VocabInfo is also available under tf.train. PiperOrigin-RevId: 188522820
PiperOrigin-RevId: 188525171
PiperOrigin-RevId: 188525453
PiperOrigin-RevId: 188528771
PiperOrigin-RevId: 188529107
PiperOrigin-RevId: 188532135
… checkpoints. PiperOrigin-RevId: 188533156
PiperOrigin-RevId: 188534066
PiperOrigin-RevId: 188536576
PiperOrigin-RevId: 188536863
PiperOrigin-RevId: 188540513
PiperOrigin-RevId: 188540659
PiperOrigin-RevId: 188540944
yifeif
reviewed
Mar 9, 2018
| @@ -267,6 +267,7 @@ cuda_py_test( | |||
| "//tensorflow/python/keras", | |||
| ], | |||
| tags = ["no_windows"], # TODO: needs investigation on Windows | |||
| tags = ["notsan"], | |||
|
I disabled a flaky test |
yifeif
approved these changes
Mar 10, 2018
StanislawAntol
pushed a commit
to StanislawAntol/tensorflow
that referenced
this pull request
Mar 23, 2018
Branch 188540944
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.