Overlap XLA tensor sync with future XLA tensor work. #478
Conversation
dlibenzi
force-pushed
the
overlap_tensor_work
branch
3 times, most recently
from
ddbe09e to
1f761a6
Compare
|
|
||
| class DeviceLocker { | ||
| public: | ||
| explicit DeviceLocker(Device device) : device_(std::move(device)) {} |
There was a problem hiding this comment.
It doesn't look like device_ is used, why store it?
There was a problem hiding this comment.
It is handy to drop in debug statements on loc/unlock paths.
| cv_.notify_one(); | ||
| } | ||
|
|
||
| void Barrier() { |
There was a problem hiding this comment.
Since this is supposed to be per device, it's not much of a barrier. It's much more similar to a monitor / condition variable, I'd call it Wait() instead.
There was a problem hiding this comment.
It is a barrier. Every async operation in flight before the barrier, must complete in order for the operations after the barrier to commence.
third_party/xla_client/cache.h
Outdated
| Equaler>; | ||
|
|
||
| void MaybeLRU(typename ElementList::iterator it) { | ||
| if (it != element_list_.begin()) { |
There was a problem hiding this comment.
What is the harm in doing this unconditionally? I wouldn't expect a measurable performance hit from doing it unconditionally, plus you can then drop the Maybe. To be fair, you could drop the Maybe anyway, since it's already LRU on the no-op path.
There was a problem hiding this comment.
SG. Renamed DoLRU().
Prepare XRT session so that they do not fall into the dreaded one-more-node-in-graph TF case. Reserve XRT sessions just for allocations (sending data to device).
dlibenzi
force-pushed
the
overlap_tensor_work
branch
from
1f761a6 to
8fb92c8
Compare
No description provided.