Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Requested GPU:0, but only XLA_GPU:0 exsits, tf-gpu1.14.0 #30748

Closed
AmitMY opened this issue Jul 16, 2019 · 23 comments
Closed

Requested GPU:0, but only XLA_GPU:0 exsits, tf-gpu1.14.0 #30748

AmitMY opened this issue Jul 16, 2019 · 23 comments
Labels
comp:gpu GPU related issues TF 1.14 for issues seen with TF 1.14 type:support Support issues

Comments

@AmitMY
Copy link

AmitMY commented Jul 16, 2019 •
edited

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
  • TensorFlow installed from (source or binary): binary
  • TensorFlow version: 1.14.0
  • Python version: 3.6.8 / 3.7.3
  • Installed using virtualenv? pip? conda?: pip
  • CUDA/cuDNN version: 10/7.4
  • GPU model and memory:
    image

Describe the problem

  1. Installed tensorflow-gpu latest version by performing: pip install --upgrade --force-reinstall tensorflow-gpu
  2. Run tf.device('/gpu:0')

The error seems to say I have 4 gpus, but they don't match GPU, only XLA_GPU. I have no idea why, earlier versions of tensorflow do say I have GPU, but claim other bugs.

Error:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device for operation pooled_center_map/center_map/AvgPool: {{node pooled_center_map/center_map/AvgPool}}was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0, /job:localhost/replica:0/task:0/device:XLA_CPU:0, /job:localhost/replica:0/task:0/device:XLA_GPU:0, /job:localhost/replica:0/task:0/device:XLA_GPU:1, /job:localhost/replica:0/task:0/device:XLA_GPU:2, /job:localhost/replica:0/task:0/device:XLA_GPU:3 ]. Make sure the device specification refers to a valid device.

Even if I try:
tf.device('/job:localhost/replica:0/task:0/device:XLA_GPU:0')

I get:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device for operation sub_stages/sub_conv1/weights/Initializer/random_uniform/RandomUniform: Could not satisfy explicit device specification '' because the node {{colocation_node sub_stages/sub_conv1/weights/Initializer/random_uniform/RandomUniform}} was colocated with a group of nodes that required incompatible device '/job:localhost/replica:0/task:0/device:XLA_GPU:0'. All available devices [/job:localhost/replica:0/task:0/device:CPU:0, /job:localhost/replica:0/task:0/device:XLA_GPU:0, /job:localhost/replica:0/task:0/device:XLA_GPU:1, /job:localhost/replica:0/task:0/device:XLA_GPU:2, /job:localhost/replica:0/task:0/device:XLA_GPU:3, /job:localhost/replica:0/task:0/device:XLA_CPU:0].
@gadagashwini-zz gadagashwini-zz self-assigned this Jul 17, 2019
@gadagashwini-zz gadagashwini-zz added TF 1.14 for issues seen with TF 1.14 comp:gpu GPU related issues type:support Support issues labels Jul 17, 2019
@jvishnuvardhan
Copy link
Contributor

@AmitMY You could use allow_soft_placement=True as an argument to ConfigProto so that it uses any other existing and supported devices for that operation. Check the following example from TF website.

# Creates a graph.
with tf.device('/device:GPU:2'):
  a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
  b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
  c = tf.matmul(a, b)
# Creates a session with allow_soft_placement and log_device_placement set
# to True.
sess = tf.Session(config=tf.ConfigProto(
      allow_soft_placement=True, log_device_placement=True))
# Runs the op.
print(sess.run(c))

Please let me know how it progresses. Thanks!

@jvishnuvardhan jvishnuvardhan added the stat:awaiting response Status - Awaiting response from author label Jul 17, 2019
@AmitMY
Copy link
Author

AmitMY commented Jul 17, 2019

Thanks @jvishnuvardhan
Using this config, it now does not error.
However, with the log_device_placement I see:

Ignoring device specification /device:GPU:0 for node 'Assign_4' because the input edge from 'sub_stages/sub_conv3/weights' is a reference connection and already has a device field set to /job:localhost/replica:0/task:0/device:CPU:0
Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [/job:localhost/replica:0/task:0/device:CPU:0].

Although I only create the computation graph under tf.device('gpu:0')

I also see:

Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

(To make sure I didn't change anything except for this config, when I remove the config I am getting that error again, still seeing my GPUs as XLA_GPU)

@jvishnuvardhan jvishnuvardhan added stat:awaiting tensorflower Status - Awaiting response from tensorflower and removed stat:awaiting response Status - Awaiting response from author labels Jul 17, 2019
@aaroey
Copy link
Member

aaroey commented Jul 18, 2019

@sanjoy would you please help to take a look?

@aaroey aaroey removed their assignment Jul 18, 2019
@tensorflowbutler tensorflowbutler removed the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Jul 18, 2019
@sanjoy
Copy link
Contributor

sanjoy commented Jul 18, 2019

Can you please attach the full log?

XLA creates an XLA_GPU device for every present on the system whereas TF creates a GPU device only for GPUs suitable for compute (i.e. ignores slower GPUs) so that could explain what you're seeing. TF logs out "Ignoring visible gpu device" when it does this enhanced filtering so you should see it in your logs.

@AmitMY
Copy link
Author

AmitMY commented Jul 19, 2019

I wouldn't say 1080Ti is slow...
I also tried on a server with 4 Titan X Pascals.

I can't attach the full log including log_device_placement, as it is at least 100 times the size of my console buffer.
Without that flag:

Titan X log output:

WARNING: Logging before flag parsing goes to stderr.
W0719 10:34:50.315408 139938591782720 deprecation_wrapper.py:119] From /home/nlp/amit/project-fingering-extraction/components/hand_pose/detect_hand_pose.py:45: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0719 10:34:50.318552 139938591782720 deprecation_wrapper.py:119] From /home/nlp/amit/project-fingering-extraction/components/hand_pose/cpm_hand_slim.py:24: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0719 10:34:51.952562 139938591782720 deprecation_wrapper.py:119] From /home/nlp/amit/project-fingering-extraction/components/hand_pose/detect_hand_pose.py:57: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

W0719 10:34:52.075365 139938591782720 deprecation_wrapper.py:119] From /home/nlp/amit/project-fingering-extraction/components/hand_pose/detect_hand_pose.py:61: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

W0719 10:34:52.075703 139938591782720 deprecation_wrapper.py:119] From /home/nlp/amit/project-fingering-extraction/components/hand_pose/detect_hand_pose.py:61: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

2019-07-19 10:34:52.076576: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-07-19 10:34:52.098554: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2019-07-19 10:34:52.788527: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5636a27cda20 executing computations on platform CUDA. Devices:
2019-07-19 10:34:52.788595: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): GeForce GTX TITAN X, Compute Capability 5.2
2019-07-19 10:34:52.788614: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (1): GeForce GTX TITAN X, Compute Capability 5.2
2019-07-19 10:34:52.788629: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (2): GeForce GTX TITAN X, Compute Capability 5.2
2019-07-19 10:34:52.788644: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (3): GeForce GTX TITAN X, Compute Capability 5.2
2019-07-19 10:34:52.795911: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2199900000 Hz
2019-07-19 10:34:52.801752: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5636a29364d0 executing computations on platform Host. Devices:
2019-07-19 10:34:52.801799: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): ,
2019-07-19 10:34:52.803912: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce GTX TITAN X major: 5 minor: 2 memoryClockRate(GHz): 1.076
pciBusID: 0000:02:00.0
2019-07-19 10:34:52.805028: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 1 with properties:
name: GeForce GTX TITAN X major: 5 minor: 2 memoryClockRate(GHz): 1.076
pciBusID: 0000:03:00.0
2019-07-19 10:34:52.806183: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 2 with properties:
name: GeForce GTX TITAN X major: 5 minor: 2 memoryClockRate(GHz): 1.076
pciBusID: 0000:81:00.0
2019-07-19 10:34:52.807128: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 3 with properties:
name: GeForce GTX TITAN X major: 5 minor: 2 memoryClockRate(GHz): 1.076
pciBusID: 0000:82:00.0
2019-07-19 10:34:52.807368: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807494: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807617: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807745: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807868: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807989: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.815678: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-07-19 10:34:52.815715: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2019-07-19 10:34:52.815868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-07-19 10:34:52.815894: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 1 2 3
2019-07-19 10:34:52.815916: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N Y N N
2019-07-19 10:34:52.815933: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 1: Y N N N
2019-07-19 10:34:52.815948: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 2: N N N Y
2019-07-19 10:34:52.815963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 3: N N Y N
W0719 10:34:52.821785 139938591782720 deprecation_wrapper.py:119] From /home/nlp/amit/project-fingering-extraction/components/hand_pose/detect_hand_pose.py:63: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

2019-07-19 10:34:52.867261: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv1/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv1/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv1/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv1/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv1/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv1/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv1/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv1/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv1/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv1/weights/read (Identity) /device:GPU:0
save/Assign_75 (Assign) /device:GPU:0

2019-07-19 10:34:52.867633: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv1/biases/Initializer/zeros (Const)
sub_stages/sub_conv1/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv1/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv1/biases/read (Identity) /device:GPU:0
save/Assign_74 (Assign) /device:GPU:0

2019-07-19 10:34:52.867914: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv2/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv2/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv2/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv2/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv2/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv2/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv2/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv2/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv2/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv2/weights/read (Identity) /device:GPU:0
save/Assign_87 (Assign) /device:GPU:0

2019-07-19 10:34:52.868169: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv2/biases/Initializer/zeros (Const)
sub_stages/sub_conv2/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv2/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv2/biases/read (Identity) /device:GPU:0
save/Assign_86 (Assign) /device:GPU:0

2019-07-19 10:34:52.868447: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv3/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv3/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv3/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv3/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv3/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv3/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv3/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv3/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv3/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv3/weights/read (Identity) /device:GPU:0
save/Assign_89 (Assign) /device:GPU:0

2019-07-19 10:34:52.868734: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv3/biases/Initializer/zeros (Const)
sub_stages/sub_conv3/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv3/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv3/biases/read (Identity) /device:GPU:0
save/Assign_88 (Assign) /device:GPU:0

2019-07-19 10:34:52.869076: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv4/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv4/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv4/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv4/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv4/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv4/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv4/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv4/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv4/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv4/weights/read (Identity) /device:GPU:0
save/Assign_91 (Assign) /device:GPU:0

2019-07-19 10:34:52.869361: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv4/biases/Initializer/zeros (Const)
sub_stages/sub_conv4/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv4/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv4/biases/read (Identity) /device:GPU:0
save/Assign_90 (Assign) /device:GPU:0

2019-07-19 10:34:52.869657: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv5/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv5/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv5/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv5/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv5/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv5/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv5/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv5/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv5/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv5/weights/read (Identity) /device:GPU:0
save/Assign_93 (Assign) /device:GPU:0

2019-07-19 10:34:52.869937: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv5/biases/Initializer/zeros (Const)
sub_stages/sub_conv5/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv5/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv5/biases/read (Identity) /device:GPU:0
save/Assign_92 (Assign) /device:GPU:0

2019-07-19 10:34:52.870231: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv6/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv6/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv6/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv6/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv6/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv6/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv6/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv6/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv6/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv6/weights/read (Identity) /device:GPU:0
save/Assign_95 (Assign) /device:GPU:0

2019-07-19 10:34:52.870506: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv6/biases/Initializer/zeros (Const)
sub_stages/sub_conv6/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv6/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv6/biases/read (Identity) /device:GPU:0
save/Assign_94 (Assign) /device:GPU:0

2019-07-19 10:34:52.870804: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv7/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv7/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv7/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv7/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv7/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv7/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv7/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv7/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv7/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv7/weights/read (Identity) /device:GPU:0
save/Assign_97 (Assign) /device:GPU:0

2019-07-19 10:34:52.871075: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv7/biases/Initializer/zeros (Const)
sub_stages/sub_conv7/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv7/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv7/biases/read (Identity) /device:GPU:0
save/Assign_96 (Assign) /device:GPU:0

2019-07-19 10:34:52.871368: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv8/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv8/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv8/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv8/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv8/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv8/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv8/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv8/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv8/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv8/weights/read (Identity) /device:GPU:0
save/Assign_99 (Assign) /device:GPU:0

2019-07-19 10:34:52.871645: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv8/biases/Initializer/zeros (Const)
sub_stages/sub_conv8/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv8/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv8/biases/read (Identity) /device:GPU:0
save/Assign_98 (Assign) /device:GPU:0

2019-07-19 10:34:52.871980: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv9/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv9/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv9/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv9/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv9/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv9/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv9/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv9/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv9/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv9/weights/read (Identity) /device:GPU:0
save/Assign_101 (Assign) /device:GPU:0

2019-07-19 10:34:52.872245: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv9/biases/Initializer/zeros (Const)
sub_stages/sub_conv9/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv9/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv9/biases/read (Identity) /device:GPU:0
save/Assign_100 (Assign) /device:GPU:0

2019-07-19 10:34:52.872517: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv10/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv10/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv10/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv10/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv10/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv10/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv10/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv10/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv10/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv10/weights/read (Identity) /device:GPU:0
save/Assign_77 (Assign) /device:GPU:0

2019-07-19 10:34:52.872785: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv10/biases/Initializer/zeros (Const)
sub_stages/sub_conv10/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv10/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv10/biases/read (Identity) /device:GPU:0
save/Assign_76 (Assign) /device:GPU:0

2019-07-19 10:34:52.873052: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv11/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv11/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv11/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv11/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv11/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv11/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv11/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv11/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv11/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv11/weights/read (Identity) /device:GPU:0
save/Assign_79 (Assign) /device:GPU:0

2019-07-19 10:34:52.873305: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv11/biases/Initializer/zeros (Const)
sub_stages/sub_conv11/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv11/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv11/biases/read (Identity) /device:GPU:0
save/Assign_78 (Assign) /device:GPU:0

2019-07-19 10:34:52.873576: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv12/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv12/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv12/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv12/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv12/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv12/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv12/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv12/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv12/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv12/weights/read (Identity) /device:GPU:0
save/Assign_81 (Assign) /device:GPU:0

2019-07-19 10:34:52.873833: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv12/biases/Initializer/zeros (Const)
sub_stages/sub_conv12/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv12/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv12/biases/read (Identity) /device:GPU:0
save/Assign_80 (Assign) /device:GPU:0

2019-07-19 10:34:52.874103: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv13/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv13/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv13/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv13/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv13/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv13/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv13/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv13/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv13/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv13/weights/read (Identity) /device:GPU:0
save/Assign_83 (Assign) /device:GPU:0

2019-07-19 10:34:52.874355: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv13/biases/Initializer/zeros (Const)
sub_stages/sub_conv13/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv13/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv13/biases/read (Identity) /device:GPU:0
save/Assign_82 (Assign) /device:GPU:0

2019-07-19 10:34:52.874618: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv14/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_conv14/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_conv14/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_conv14/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_conv14/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_conv14/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_conv14/weights/Initializer/random_uniform (Add)
sub_stages/sub_conv14/weights (VariableV2) /device:GPU:0
sub_stages/sub_conv14/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_conv14/weights/read (Identity) /device:GPU:0
save/Assign_85 (Assign) /device:GPU:0

2019-07-19 10:34:52.874873: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_conv14/biases/Initializer/zeros (Const)
sub_stages/sub_conv14/biases (VariableV2) /device:GPU:0
sub_stages/sub_conv14/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_conv14/biases/read (Identity) /device:GPU:0
save/Assign_84 (Assign) /device:GPU:0

2019-07-19 10:34:52.875141: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_stage_img_feature/weights/Initializer/random_uniform/shape (Const)
sub_stages/sub_stage_img_feature/weights/Initializer/random_uniform/min (Const)
sub_stages/sub_stage_img_feature/weights/Initializer/random_uniform/max (Const)
sub_stages/sub_stage_img_feature/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
sub_stages/sub_stage_img_feature/weights/Initializer/random_uniform/sub (Sub)
sub_stages/sub_stage_img_feature/weights/Initializer/random_uniform/mul (Mul)
sub_stages/sub_stage_img_feature/weights/Initializer/random_uniform (Add)
sub_stages/sub_stage_img_feature/weights (VariableV2) /device:GPU:0
sub_stages/sub_stage_img_feature/weights/Assign (Assign) /device:GPU:0
sub_stages/sub_stage_img_feature/weights/read (Identity) /device:GPU:0
save/Assign_103 (Assign) /device:GPU:0

2019-07-19 10:34:52.875392: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
sub_stages/sub_stage_img_feature/biases/Initializer/zeros (Const)
sub_stages/sub_stage_img_feature/biases (VariableV2) /device:GPU:0
sub_stages/sub_stage_img_feature/biases/Assign (Assign) /device:GPU:0
sub_stages/sub_stage_img_feature/biases/read (Identity) /device:GPU:0
save/Assign_102 (Assign) /device:GPU:0

2019-07-19 10:34:52.875655: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_1/conv1/weights/Initializer/random_uniform/shape (Const)
stage_1/conv1/weights/Initializer/random_uniform/min (Const)
stage_1/conv1/weights/Initializer/random_uniform/max (Const)
stage_1/conv1/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_1/conv1/weights/Initializer/random_uniform/sub (Sub)
stage_1/conv1/weights/Initializer/random_uniform/mul (Mul)
stage_1/conv1/weights/Initializer/random_uniform (Add)
stage_1/conv1/weights (VariableV2) /device:GPU:0
stage_1/conv1/weights/Assign (Assign) /device:GPU:0
stage_1/conv1/weights/read (Identity) /device:GPU:0
save/Assign_1 (Assign) /device:GPU:0

2019-07-19 10:34:52.875909: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_1/conv1/biases/Initializer/zeros (Const)
stage_1/conv1/biases (VariableV2) /device:GPU:0
stage_1/conv1/biases/Assign (Assign) /device:GPU:0
stage_1/conv1/biases/read (Identity) /device:GPU:0
save/Assign (Assign) /device:GPU:0

2019-07-19 10:34:52.876172: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_1/stage_heatmap/weights/Initializer/random_uniform/shape (Const)
stage_1/stage_heatmap/weights/Initializer/random_uniform/min (Const)
stage_1/stage_heatmap/weights/Initializer/random_uniform/max (Const)
stage_1/stage_heatmap/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_1/stage_heatmap/weights/Initializer/random_uniform/sub (Sub)
stage_1/stage_heatmap/weights/Initializer/random_uniform/mul (Mul)
stage_1/stage_heatmap/weights/Initializer/random_uniform (Add)
stage_1/stage_heatmap/weights (VariableV2) /device:GPU:0
stage_1/stage_heatmap/weights/Assign (Assign) /device:GPU:0
stage_1/stage_heatmap/weights/read (Identity) /device:GPU:0
save/Assign_3 (Assign) /device:GPU:0

2019-07-19 10:34:52.876422: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_1/stage_heatmap/biases/Initializer/zeros (Const)
stage_1/stage_heatmap/biases (VariableV2) /device:GPU:0
stage_1/stage_heatmap/biases/Assign (Assign) /device:GPU:0
stage_1/stage_heatmap/biases/read (Identity) /device:GPU:0
save/Assign_2 (Assign) /device:GPU:0

2019-07-19 10:34:52.876703: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv1/weights/Initializer/random_uniform/shape (Const)
stage_2/mid_conv1/weights/Initializer/random_uniform/min (Const)
stage_2/mid_conv1/weights/Initializer/random_uniform/max (Const)
stage_2/mid_conv1/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_2/mid_conv1/weights/Initializer/random_uniform/sub (Sub)
stage_2/mid_conv1/weights/Initializer/random_uniform/mul (Mul)
stage_2/mid_conv1/weights/Initializer/random_uniform (Add)
stage_2/mid_conv1/weights (VariableV2) /device:GPU:0
stage_2/mid_conv1/weights/Assign (Assign) /device:GPU:0
stage_2/mid_conv1/weights/read (Identity) /device:GPU:0
save/Assign_5 (Assign) /device:GPU:0

2019-07-19 10:34:52.876955: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv1/biases/Initializer/zeros (Const)
stage_2/mid_conv1/biases (VariableV2) /device:GPU:0
stage_2/mid_conv1/biases/Assign (Assign) /device:GPU:0
stage_2/mid_conv1/biases/read (Identity) /device:GPU:0
save/Assign_4 (Assign) /device:GPU:0

2019-07-19 10:34:52.877215: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv2/weights/Initializer/random_uniform/shape (Const)
stage_2/mid_conv2/weights/Initializer/random_uniform/min (Const)
stage_2/mid_conv2/weights/Initializer/random_uniform/max (Const)
stage_2/mid_conv2/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_2/mid_conv2/weights/Initializer/random_uniform/sub (Sub)
stage_2/mid_conv2/weights/Initializer/random_uniform/mul (Mul)
stage_2/mid_conv2/weights/Initializer/random_uniform (Add)
stage_2/mid_conv2/weights (VariableV2) /device:GPU:0
stage_2/mid_conv2/weights/Assign (Assign) /device:GPU:0
stage_2/mid_conv2/weights/read (Identity) /device:GPU:0
save/Assign_7 (Assign) /device:GPU:0

2019-07-19 10:34:52.877468: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv2/biases/Initializer/zeros (Const)
stage_2/mid_conv2/biases (VariableV2) /device:GPU:0
stage_2/mid_conv2/biases/Assign (Assign) /device:GPU:0
stage_2/mid_conv2/biases/read (Identity) /device:GPU:0
save/Assign_6 (Assign) /device:GPU:0

2019-07-19 10:34:52.877737: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv3/weights/Initializer/random_uniform/shape (Const)
stage_2/mid_conv3/weights/Initializer/random_uniform/min (Const)
stage_2/mid_conv3/weights/Initializer/random_uniform/max (Const)
stage_2/mid_conv3/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_2/mid_conv3/weights/Initializer/random_uniform/sub (Sub)
stage_2/mid_conv3/weights/Initializer/random_uniform/mul (Mul)
stage_2/mid_conv3/weights/Initializer/random_uniform (Add)
stage_2/mid_conv3/weights (VariableV2) /device:GPU:0
stage_2/mid_conv3/weights/Assign (Assign) /device:GPU:0
stage_2/mid_conv3/weights/read (Identity) /device:GPU:0
save/Assign_9 (Assign) /device:GPU:0

2019-07-19 10:34:52.877992: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv3/biases/Initializer/zeros (Const)
stage_2/mid_conv3/biases (VariableV2) /device:GPU:0
stage_2/mid_conv3/biases/Assign (Assign) /device:GPU:0
stage_2/mid_conv3/biases/read (Identity) /device:GPU:0
save/Assign_8 (Assign) /device:GPU:0

2019-07-19 10:34:52.878253: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv4/weights/Initializer/random_uniform/shape (Const)
stage_2/mid_conv4/weights/Initializer/random_uniform/min (Const)
stage_2/mid_conv4/weights/Initializer/random_uniform/max (Const)
stage_2/mid_conv4/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_2/mid_conv4/weights/Initializer/random_uniform/sub (Sub)
stage_2/mid_conv4/weights/Initializer/random_uniform/mul (Mul)
stage_2/mid_conv4/weights/Initializer/random_uniform (Add)
stage_2/mid_conv4/weights (VariableV2) /device:GPU:0
stage_2/mid_conv4/weights/Assign (Assign) /device:GPU:0
stage_2/mid_conv4/weights/read (Identity) /device:GPU:0
save/Assign_11 (Assign) /device:GPU:0

2019-07-19 10:34:52.878501: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv4/biases/Initializer/zeros (Const)
stage_2/mid_conv4/biases (VariableV2) /device:GPU:0
stage_2/mid_conv4/biases/Assign (Assign) /device:GPU:0
stage_2/mid_conv4/biases/read (Identity) /device:GPU:0
save/Assign_10 (Assign) /device:GPU:0

2019-07-19 10:34:52.878769: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv5/weights/Initializer/random_uniform/shape (Const)
stage_2/mid_conv5/weights/Initializer/random_uniform/min (Const)
stage_2/mid_conv5/weights/Initializer/random_uniform/max (Const)
stage_2/mid_conv5/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_2/mid_conv5/weights/Initializer/random_uniform/sub (Sub)
stage_2/mid_conv5/weights/Initializer/random_uniform/mul (Mul)
stage_2/mid_conv5/weights/Initializer/random_uniform (Add)
stage_2/mid_conv5/weights (VariableV2) /device:GPU:0
stage_2/mid_conv5/weights/Assign (Assign) /device:GPU:0
stage_2/mid_conv5/weights/read (Identity) /device:GPU:0
save/Assign_13 (Assign) /device:GPU:0

2019-07-19 10:34:52.879019: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv5/biases/Initializer/zeros (Const)
stage_2/mid_conv5/biases (VariableV2) /device:GPU:0
stage_2/mid_conv5/biases/Assign (Assign) /device:GPU:0
stage_2/mid_conv5/biases/read (Identity) /device:GPU:0
save/Assign_12 (Assign) /device:GPU:0

2019-07-19 10:34:52.879283: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv6/weights/Initializer/random_uniform/shape (Const)
stage_2/mid_conv6/weights/Initializer/random_uniform/min (Const)
stage_2/mid_conv6/weights/Initializer/random_uniform/max (Const)
stage_2/mid_conv6/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_2/mid_conv6/weights/Initializer/random_uniform/sub (Sub)
stage_2/mid_conv6/weights/Initializer/random_uniform/mul (Mul)
stage_2/mid_conv6/weights/Initializer/random_uniform (Add)
stage_2/mid_conv6/weights (VariableV2) /device:GPU:0
stage_2/mid_conv6/weights/Assign (Assign) /device:GPU:0
stage_2/mid_conv6/weights/read (Identity) /device:GPU:0
save/Assign_15 (Assign) /device:GPU:0

2019-07-19 10:34:52.879535: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv6/biases/Initializer/zeros (Const)
stage_2/mid_conv6/biases (VariableV2) /device:GPU:0
stage_2/mid_conv6/biases/Assign (Assign) /device:GPU:0
stage_2/mid_conv6/biases/read (Identity) /device:GPU:0
save/Assign_14 (Assign) /device:GPU:0

2019-07-19 10:34:52.879803: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv7/weights/Initializer/random_uniform/shape (Const)
stage_2/mid_conv7/weights/Initializer/random_uniform/min (Const)
stage_2/mid_conv7/weights/Initializer/random_uniform/max (Const)
stage_2/mid_conv7/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_2/mid_conv7/weights/Initializer/random_uniform/sub (Sub)
stage_2/mid_conv7/weights/Initializer/random_uniform/mul (Mul)
stage_2/mid_conv7/weights/Initializer/random_uniform (Add)
stage_2/mid_conv7/weights (VariableV2) /device:GPU:0
stage_2/mid_conv7/weights/Assign (Assign) /device:GPU:0
stage_2/mid_conv7/weights/read (Identity) /device:GPU:0
save/Assign_17 (Assign) /device:GPU:0

2019-07-19 10:34:52.880057: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_2/mid_conv7/biases/Initializer/zeros (Const)
stage_2/mid_conv7/biases (VariableV2) /device:GPU:0
stage_2/mid_conv7/biases/Assign (Assign) /device:GPU:0
stage_2/mid_conv7/biases/read (Identity) /device:GPU:0
save/Assign_16 (Assign) /device:GPU:0

2019-07-19 10:34:52.880329: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv1/weights/Initializer/random_uniform/shape (Const)
stage_3/mid_conv1/weights/Initializer/random_uniform/min (Const)
stage_3/mid_conv1/weights/Initializer/random_uniform/max (Const)
stage_3/mid_conv1/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_3/mid_conv1/weights/Initializer/random_uniform/sub (Sub)
stage_3/mid_conv1/weights/Initializer/random_uniform/mul (Mul)
stage_3/mid_conv1/weights/Initializer/random_uniform (Add)
stage_3/mid_conv1/weights (VariableV2) /device:GPU:0
stage_3/mid_conv1/weights/Assign (Assign) /device:GPU:0
stage_3/mid_conv1/weights/read (Identity) /device:GPU:0
save/Assign_19 (Assign) /device:GPU:0

2019-07-19 10:34:52.880584: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv1/biases/Initializer/zeros (Const)
stage_3/mid_conv1/biases (VariableV2) /device:GPU:0
stage_3/mid_conv1/biases/Assign (Assign) /device:GPU:0
stage_3/mid_conv1/biases/read (Identity) /device:GPU:0
save/Assign_18 (Assign) /device:GPU:0

2019-07-19 10:34:52.880855: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv2/weights/Initializer/random_uniform/shape (Const)
stage_3/mid_conv2/weights/Initializer/random_uniform/min (Const)
stage_3/mid_conv2/weights/Initializer/random_uniform/max (Const)
stage_3/mid_conv2/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_3/mid_conv2/weights/Initializer/random_uniform/sub (Sub)
stage_3/mid_conv2/weights/Initializer/random_uniform/mul (Mul)
stage_3/mid_conv2/weights/Initializer/random_uniform (Add)
stage_3/mid_conv2/weights (VariableV2) /device:GPU:0
stage_3/mid_conv2/weights/Assign (Assign) /device:GPU:0
stage_3/mid_conv2/weights/read (Identity) /device:GPU:0
save/Assign_21 (Assign) /device:GPU:0

2019-07-19 10:34:52.881106: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv2/biases/Initializer/zeros (Const)
stage_3/mid_conv2/biases (VariableV2) /device:GPU:0
stage_3/mid_conv2/biases/Assign (Assign) /device:GPU:0
stage_3/mid_conv2/biases/read (Identity) /device:GPU:0
save/Assign_20 (Assign) /device:GPU:0

2019-07-19 10:34:52.881368: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv3/weights/Initializer/random_uniform/shape (Const)
stage_3/mid_conv3/weights/Initializer/random_uniform/min (Const)
stage_3/mid_conv3/weights/Initializer/random_uniform/max (Const)
stage_3/mid_conv3/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_3/mid_conv3/weights/Initializer/random_uniform/sub (Sub)
stage_3/mid_conv3/weights/Initializer/random_uniform/mul (Mul)
stage_3/mid_conv3/weights/Initializer/random_uniform (Add)
stage_3/mid_conv3/weights (VariableV2) /device:GPU:0
stage_3/mid_conv3/weights/Assign (Assign) /device:GPU:0
stage_3/mid_conv3/weights/read (Identity) /device:GPU:0
save/Assign_23 (Assign) /device:GPU:0

2019-07-19 10:34:52.881616: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv3/biases/Initializer/zeros (Const)
stage_3/mid_conv3/biases (VariableV2) /device:GPU:0
stage_3/mid_conv3/biases/Assign (Assign) /device:GPU:0
stage_3/mid_conv3/biases/read (Identity) /device:GPU:0
save/Assign_22 (Assign) /device:GPU:0

2019-07-19 10:34:52.881887: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv4/weights/Initializer/random_uniform/shape (Const)
stage_3/mid_conv4/weights/Initializer/random_uniform/min (Const)
stage_3/mid_conv4/weights/Initializer/random_uniform/max (Const)
stage_3/mid_conv4/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_3/mid_conv4/weights/Initializer/random_uniform/sub (Sub)
stage_3/mid_conv4/weights/Initializer/random_uniform/mul (Mul)
stage_3/mid_conv4/weights/Initializer/random_uniform (Add)
stage_3/mid_conv4/weights (VariableV2) /device:GPU:0
stage_3/mid_conv4/weights/Assign (Assign) /device:GPU:0
stage_3/mid_conv4/weights/read (Identity) /device:GPU:0
save/Assign_25 (Assign) /device:GPU:0

2019-07-19 10:34:52.882138: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv4/biases/Initializer/zeros (Const)
stage_3/mid_conv4/biases (VariableV2) /device:GPU:0
stage_3/mid_conv4/biases/Assign (Assign) /device:GPU:0
stage_3/mid_conv4/biases/read (Identity) /device:GPU:0
save/Assign_24 (Assign) /device:GPU:0

2019-07-19 10:34:52.882405: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv5/weights/Initializer/random_uniform/shape (Const)
stage_3/mid_conv5/weights/Initializer/random_uniform/min (Const)
stage_3/mid_conv5/weights/Initializer/random_uniform/max (Const)
stage_3/mid_conv5/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_3/mid_conv5/weights/Initializer/random_uniform/sub (Sub)
stage_3/mid_conv5/weights/Initializer/random_uniform/mul (Mul)
stage_3/mid_conv5/weights/Initializer/random_uniform (Add)
stage_3/mid_conv5/weights (VariableV2) /device:GPU:0
stage_3/mid_conv5/weights/Assign (Assign) /device:GPU:0
stage_3/mid_conv5/weights/read (Identity) /device:GPU:0
save/Assign_27 (Assign) /device:GPU:0

2019-07-19 10:34:52.882654: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv5/biases/Initializer/zeros (Const)
stage_3/mid_conv5/biases (VariableV2) /device:GPU:0
stage_3/mid_conv5/biases/Assign (Assign) /device:GPU:0
stage_3/mid_conv5/biases/read (Identity) /device:GPU:0
save/Assign_26 (Assign) /device:GPU:0

2019-07-19 10:34:52.882925: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv6/weights/Initializer/random_uniform/shape (Const)
stage_3/mid_conv6/weights/Initializer/random_uniform/min (Const)
stage_3/mid_conv6/weights/Initializer/random_uniform/max (Const)
stage_3/mid_conv6/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_3/mid_conv6/weights/Initializer/random_uniform/sub (Sub)
stage_3/mid_conv6/weights/Initializer/random_uniform/mul (Mul)
stage_3/mid_conv6/weights/Initializer/random_uniform (Add)
stage_3/mid_conv6/weights (VariableV2) /device:GPU:0
stage_3/mid_conv6/weights/Assign (Assign) /device:GPU:0
stage_3/mid_conv6/weights/read (Identity) /device:GPU:0
save/Assign_29 (Assign) /device:GPU:0

2019-07-19 10:34:52.883174: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv6/biases/Initializer/zeros (Const)
stage_3/mid_conv6/biases (VariableV2) /device:GPU:0
stage_3/mid_conv6/biases/Assign (Assign) /device:GPU:0
stage_3/mid_conv6/biases/read (Identity) /device:GPU:0
save/Assign_28 (Assign) /device:GPU:0

2019-07-19 10:34:52.883436: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv7/weights/Initializer/random_uniform/shape (Const)
stage_3/mid_conv7/weights/Initializer/random_uniform/min (Const)
stage_3/mid_conv7/weights/Initializer/random_uniform/max (Const)
stage_3/mid_conv7/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_3/mid_conv7/weights/Initializer/random_uniform/sub (Sub)
stage_3/mid_conv7/weights/Initializer/random_uniform/mul (Mul)
stage_3/mid_conv7/weights/Initializer/random_uniform (Add)
stage_3/mid_conv7/weights (VariableV2) /device:GPU:0
stage_3/mid_conv7/weights/Assign (Assign) /device:GPU:0
stage_3/mid_conv7/weights/read (Identity) /device:GPU:0
save/Assign_31 (Assign) /device:GPU:0

2019-07-19 10:34:52.883699: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_3/mid_conv7/biases/Initializer/zeros (Const)
stage_3/mid_conv7/biases (VariableV2) /device:GPU:0
stage_3/mid_conv7/biases/Assign (Assign) /device:GPU:0
stage_3/mid_conv7/biases/read (Identity) /device:GPU:0
save/Assign_30 (Assign) /device:GPU:0

2019-07-19 10:34:52.883977: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv1/weights/Initializer/random_uniform/shape (Const)
stage_4/mid_conv1/weights/Initializer/random_uniform/min (Const)
stage_4/mid_conv1/weights/Initializer/random_uniform/max (Const)
stage_4/mid_conv1/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_4/mid_conv1/weights/Initializer/random_uniform/sub (Sub)
stage_4/mid_conv1/weights/Initializer/random_uniform/mul (Mul)
stage_4/mid_conv1/weights/Initializer/random_uniform (Add)
stage_4/mid_conv1/weights (VariableV2) /device:GPU:0
stage_4/mid_conv1/weights/Assign (Assign) /device:GPU:0
stage_4/mid_conv1/weights/read (Identity) /device:GPU:0
save/Assign_33 (Assign) /device:GPU:0

2019-07-19 10:34:52.884228: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv1/biases/Initializer/zeros (Const)
stage_4/mid_conv1/biases (VariableV2) /device:GPU:0
stage_4/mid_conv1/biases/Assign (Assign) /device:GPU:0
stage_4/mid_conv1/biases/read (Identity) /device:GPU:0
save/Assign_32 (Assign) /device:GPU:0

2019-07-19 10:34:52.884498: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv2/weights/Initializer/random_uniform/shape (Const)
stage_4/mid_conv2/weights/Initializer/random_uniform/min (Const)
stage_4/mid_conv2/weights/Initializer/random_uniform/max (Const)
stage_4/mid_conv2/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_4/mid_conv2/weights/Initializer/random_uniform/sub (Sub)
stage_4/mid_conv2/weights/Initializer/random_uniform/mul (Mul)
stage_4/mid_conv2/weights/Initializer/random_uniform (Add)
stage_4/mid_conv2/weights (VariableV2) /device:GPU:0
stage_4/mid_conv2/weights/Assign (Assign) /device:GPU:0
stage_4/mid_conv2/weights/read (Identity) /device:GPU:0
save/Assign_35 (Assign) /device:GPU:0

2019-07-19 10:34:52.884804: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv2/biases/Initializer/zeros (Const)
stage_4/mid_conv2/biases (VariableV2) /device:GPU:0
stage_4/mid_conv2/biases/Assign (Assign) /device:GPU:0
stage_4/mid_conv2/biases/read (Identity) /device:GPU:0
save/Assign_34 (Assign) /device:GPU:0

2019-07-19 10:34:52.885094: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv3/weights/Initializer/random_uniform/shape (Const)
stage_4/mid_conv3/weights/Initializer/random_uniform/min (Const)
stage_4/mid_conv3/weights/Initializer/random_uniform/max (Const)
stage_4/mid_conv3/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_4/mid_conv3/weights/Initializer/random_uniform/sub (Sub)
stage_4/mid_conv3/weights/Initializer/random_uniform/mul (Mul)
stage_4/mid_conv3/weights/Initializer/random_uniform (Add)
stage_4/mid_conv3/weights (VariableV2) /device:GPU:0
stage_4/mid_conv3/weights/Assign (Assign) /device:GPU:0
stage_4/mid_conv3/weights/read (Identity) /device:GPU:0
save/Assign_37 (Assign) /device:GPU:0

2019-07-19 10:34:52.885347: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv3/biases/Initializer/zeros (Const)
stage_4/mid_conv3/biases (VariableV2) /device:GPU:0
stage_4/mid_conv3/biases/Assign (Assign) /device:GPU:0
stage_4/mid_conv3/biases/read (Identity) /device:GPU:0
save/Assign_36 (Assign) /device:GPU:0

2019-07-19 10:34:52.885614: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv4/weights/Initializer/random_uniform/shape (Const)
stage_4/mid_conv4/weights/Initializer/random_uniform/min (Const)
stage_4/mid_conv4/weights/Initializer/random_uniform/max (Const)
stage_4/mid_conv4/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_4/mid_conv4/weights/Initializer/random_uniform/sub (Sub)
stage_4/mid_conv4/weights/Initializer/random_uniform/mul (Mul)
stage_4/mid_conv4/weights/Initializer/random_uniform (Add)
stage_4/mid_conv4/weights (VariableV2) /device:GPU:0
stage_4/mid_conv4/weights/Assign (Assign) /device:GPU:0
stage_4/mid_conv4/weights/read (Identity) /device:GPU:0
save/Assign_39 (Assign) /device:GPU:0

2019-07-19 10:34:52.885877: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv4/biases/Initializer/zeros (Const)
stage_4/mid_conv4/biases (VariableV2) /device:GPU:0
stage_4/mid_conv4/biases/Assign (Assign) /device:GPU:0
stage_4/mid_conv4/biases/read (Identity) /device:GPU:0
save/Assign_38 (Assign) /device:GPU:0

2019-07-19 10:34:52.886139: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv5/weights/Initializer/random_uniform/shape (Const)
stage_4/mid_conv5/weights/Initializer/random_uniform/min (Const)
stage_4/mid_conv5/weights/Initializer/random_uniform/max (Const)
stage_4/mid_conv5/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_4/mid_conv5/weights/Initializer/random_uniform/sub (Sub)
stage_4/mid_conv5/weights/Initializer/random_uniform/mul (Mul)
stage_4/mid_conv5/weights/Initializer/random_uniform (Add)
stage_4/mid_conv5/weights (VariableV2) /device:GPU:0
stage_4/mid_conv5/weights/Assign (Assign) /device:GPU:0
stage_4/mid_conv5/weights/read (Identity) /device:GPU:0
save/Assign_41 (Assign) /device:GPU:0

2019-07-19 10:34:52.886390: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv5/biases/Initializer/zeros (Const)
stage_4/mid_conv5/biases (VariableV2) /device:GPU:0
stage_4/mid_conv5/biases/Assign (Assign) /device:GPU:0
stage_4/mid_conv5/biases/read (Identity) /device:GPU:0
save/Assign_40 (Assign) /device:GPU:0

2019-07-19 10:34:52.886654: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv6/weights/Initializer/random_uniform/shape (Const)
stage_4/mid_conv6/weights/Initializer/random_uniform/min (Const)
stage_4/mid_conv6/weights/Initializer/random_uniform/max (Const)
stage_4/mid_conv6/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_4/mid_conv6/weights/Initializer/random_uniform/sub (Sub)
stage_4/mid_conv6/weights/Initializer/random_uniform/mul (Mul)
stage_4/mid_conv6/weights/Initializer/random_uniform (Add)
stage_4/mid_conv6/weights (VariableV2) /device:GPU:0
stage_4/mid_conv6/weights/Assign (Assign) /device:GPU:0
stage_4/mid_conv6/weights/read (Identity) /device:GPU:0
save/Assign_43 (Assign) /device:GPU:0

2019-07-19 10:34:52.886911: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv6/biases/Initializer/zeros (Const)
stage_4/mid_conv6/biases (VariableV2) /device:GPU:0
stage_4/mid_conv6/biases/Assign (Assign) /device:GPU:0
stage_4/mid_conv6/biases/read (Identity) /device:GPU:0
save/Assign_42 (Assign) /device:GPU:0

2019-07-19 10:34:52.887173: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv7/weights/Initializer/random_uniform/shape (Const)
stage_4/mid_conv7/weights/Initializer/random_uniform/min (Const)
stage_4/mid_conv7/weights/Initializer/random_uniform/max (Const)
stage_4/mid_conv7/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_4/mid_conv7/weights/Initializer/random_uniform/sub (Sub)
stage_4/mid_conv7/weights/Initializer/random_uniform/mul (Mul)
stage_4/mid_conv7/weights/Initializer/random_uniform (Add)
stage_4/mid_conv7/weights (VariableV2) /device:GPU:0
stage_4/mid_conv7/weights/Assign (Assign) /device:GPU:0
stage_4/mid_conv7/weights/read (Identity) /device:GPU:0
save/Assign_45 (Assign) /device:GPU:0

2019-07-19 10:34:52.887426: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_4/mid_conv7/biases/Initializer/zeros (Const)
stage_4/mid_conv7/biases (VariableV2) /device:GPU:0
stage_4/mid_conv7/biases/Assign (Assign) /device:GPU:0
stage_4/mid_conv7/biases/read (Identity) /device:GPU:0
save/Assign_44 (Assign) /device:GPU:0

2019-07-19 10:34:52.887708: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv1/weights/Initializer/random_uniform/shape (Const)
stage_5/mid_conv1/weights/Initializer/random_uniform/min (Const)
stage_5/mid_conv1/weights/Initializer/random_uniform/max (Const)
stage_5/mid_conv1/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_5/mid_conv1/weights/Initializer/random_uniform/sub (Sub)
stage_5/mid_conv1/weights/Initializer/random_uniform/mul (Mul)
stage_5/mid_conv1/weights/Initializer/random_uniform (Add)
stage_5/mid_conv1/weights (VariableV2) /device:GPU:0
stage_5/mid_conv1/weights/Assign (Assign) /device:GPU:0
stage_5/mid_conv1/weights/read (Identity) /device:GPU:0
save/Assign_47 (Assign) /device:GPU:0

2019-07-19 10:34:52.887958: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv1/biases/Initializer/zeros (Const)
stage_5/mid_conv1/biases (VariableV2) /device:GPU:0
stage_5/mid_conv1/biases/Assign (Assign) /device:GPU:0
stage_5/mid_conv1/biases/read (Identity) /device:GPU:0
save/Assign_46 (Assign) /device:GPU:0

2019-07-19 10:34:52.888220: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv2/weights/Initializer/random_uniform/shape (Const)
stage_5/mid_conv2/weights/Initializer/random_uniform/min (Const)
stage_5/mid_conv2/weights/Initializer/random_uniform/max (Const)
stage_5/mid_conv2/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_5/mid_conv2/weights/Initializer/random_uniform/sub (Sub)
stage_5/mid_conv2/weights/Initializer/random_uniform/mul (Mul)
stage_5/mid_conv2/weights/Initializer/random_uniform (Add)
stage_5/mid_conv2/weights (VariableV2) /device:GPU:0
stage_5/mid_conv2/weights/Assign (Assign) /device:GPU:0
stage_5/mid_conv2/weights/read (Identity) /device:GPU:0
save/Assign_49 (Assign) /device:GPU:0

2019-07-19 10:34:52.888468: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv2/biases/Initializer/zeros (Const)
stage_5/mid_conv2/biases (VariableV2) /device:GPU:0
stage_5/mid_conv2/biases/Assign (Assign) /device:GPU:0
stage_5/mid_conv2/biases/read (Identity) /device:GPU:0
save/Assign_48 (Assign) /device:GPU:0

2019-07-19 10:34:52.888775: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv3/weights/Initializer/random_uniform/shape (Const)
stage_5/mid_conv3/weights/Initializer/random_uniform/min (Const)
stage_5/mid_conv3/weights/Initializer/random_uniform/max (Const)
stage_5/mid_conv3/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_5/mid_conv3/weights/Initializer/random_uniform/sub (Sub)
stage_5/mid_conv3/weights/Initializer/random_uniform/mul (Mul)
stage_5/mid_conv3/weights/Initializer/random_uniform (Add)
stage_5/mid_conv3/weights (VariableV2) /device:GPU:0
stage_5/mid_conv3/weights/Assign (Assign) /device:GPU:0
stage_5/mid_conv3/weights/read (Identity) /device:GPU:0
save/Assign_51 (Assign) /device:GPU:0

2019-07-19 10:34:52.889057: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv3/biases/Initializer/zeros (Const)
stage_5/mid_conv3/biases (VariableV2) /device:GPU:0
stage_5/mid_conv3/biases/Assign (Assign) /device:GPU:0
stage_5/mid_conv3/biases/read (Identity) /device:GPU:0
save/Assign_50 (Assign) /device:GPU:0

2019-07-19 10:34:52.889327: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv4/weights/Initializer/random_uniform/shape (Const)
stage_5/mid_conv4/weights/Initializer/random_uniform/min (Const)
stage_5/mid_conv4/weights/Initializer/random_uniform/max (Const)
stage_5/mid_conv4/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_5/mid_conv4/weights/Initializer/random_uniform/sub (Sub)
stage_5/mid_conv4/weights/Initializer/random_uniform/mul (Mul)
stage_5/mid_conv4/weights/Initializer/random_uniform (Add)
stage_5/mid_conv4/weights (VariableV2) /device:GPU:0
stage_5/mid_conv4/weights/Assign (Assign) /device:GPU:0
stage_5/mid_conv4/weights/read (Identity) /device:GPU:0
save/Assign_53 (Assign) /device:GPU:0

2019-07-19 10:34:52.889576: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv4/biases/Initializer/zeros (Const)
stage_5/mid_conv4/biases (VariableV2) /device:GPU:0
stage_5/mid_conv4/biases/Assign (Assign) /device:GPU:0
stage_5/mid_conv4/biases/read (Identity) /device:GPU:0
save/Assign_52 (Assign) /device:GPU:0

2019-07-19 10:34:52.889851: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv5/weights/Initializer/random_uniform/shape (Const)
stage_5/mid_conv5/weights/Initializer/random_uniform/min (Const)
stage_5/mid_conv5/weights/Initializer/random_uniform/max (Const)
stage_5/mid_conv5/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_5/mid_conv5/weights/Initializer/random_uniform/sub (Sub)
stage_5/mid_conv5/weights/Initializer/random_uniform/mul (Mul)
stage_5/mid_conv5/weights/Initializer/random_uniform (Add)
stage_5/mid_conv5/weights (VariableV2) /device:GPU:0
stage_5/mid_conv5/weights/Assign (Assign) /device:GPU:0
stage_5/mid_conv5/weights/read (Identity) /device:GPU:0
save/Assign_55 (Assign) /device:GPU:0

2019-07-19 10:34:52.890099: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv5/biases/Initializer/zeros (Const)
stage_5/mid_conv5/biases (VariableV2) /device:GPU:0
stage_5/mid_conv5/biases/Assign (Assign) /device:GPU:0
stage_5/mid_conv5/biases/read (Identity) /device:GPU:0
save/Assign_54 (Assign) /device:GPU:0

2019-07-19 10:34:52.890361: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv6/weights/Initializer/random_uniform/shape (Const)
stage_5/mid_conv6/weights/Initializer/random_uniform/min (Const)
stage_5/mid_conv6/weights/Initializer/random_uniform/max (Const)
stage_5/mid_conv6/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_5/mid_conv6/weights/Initializer/random_uniform/sub (Sub)
stage_5/mid_conv6/weights/Initializer/random_uniform/mul (Mul)
stage_5/mid_conv6/weights/Initializer/random_uniform (Add)
stage_5/mid_conv6/weights (VariableV2) /device:GPU:0
stage_5/mid_conv6/weights/Assign (Assign) /device:GPU:0
stage_5/mid_conv6/weights/read (Identity) /device:GPU:0
save/Assign_57 (Assign) /device:GPU:0

2019-07-19 10:34:52.890609: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv6/biases/Initializer/zeros (Const)
stage_5/mid_conv6/biases (VariableV2) /device:GPU:0
stage_5/mid_conv6/biases/Assign (Assign) /device:GPU:0
stage_5/mid_conv6/biases/read (Identity) /device:GPU:0
save/Assign_56 (Assign) /device:GPU:0

2019-07-19 10:34:52.890875: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv7/weights/Initializer/random_uniform/shape (Const)
stage_5/mid_conv7/weights/Initializer/random_uniform/min (Const)
stage_5/mid_conv7/weights/Initializer/random_uniform/max (Const)
stage_5/mid_conv7/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_5/mid_conv7/weights/Initializer/random_uniform/sub (Sub)
stage_5/mid_conv7/weights/Initializer/random_uniform/mul (Mul)
stage_5/mid_conv7/weights/Initializer/random_uniform (Add)
stage_5/mid_conv7/weights (VariableV2) /device:GPU:0
stage_5/mid_conv7/weights/Assign (Assign) /device:GPU:0
stage_5/mid_conv7/weights/read (Identity) /device:GPU:0
save/Assign_59 (Assign) /device:GPU:0

2019-07-19 10:34:52.891125: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_5/mid_conv7/biases/Initializer/zeros (Const)
stage_5/mid_conv7/biases (VariableV2) /device:GPU:0
stage_5/mid_conv7/biases/Assign (Assign) /device:GPU:0
stage_5/mid_conv7/biases/read (Identity) /device:GPU:0
save/Assign_58 (Assign) /device:GPU:0

2019-07-19 10:34:52.891398: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv1/weights/Initializer/random_uniform/shape (Const)
stage_6/mid_conv1/weights/Initializer/random_uniform/min (Const)
stage_6/mid_conv1/weights/Initializer/random_uniform/max (Const)
stage_6/mid_conv1/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_6/mid_conv1/weights/Initializer/random_uniform/sub (Sub)
stage_6/mid_conv1/weights/Initializer/random_uniform/mul (Mul)
stage_6/mid_conv1/weights/Initializer/random_uniform (Add)
stage_6/mid_conv1/weights (VariableV2) /device:GPU:0
stage_6/mid_conv1/weights/Assign (Assign) /device:GPU:0
stage_6/mid_conv1/weights/read (Identity) /device:GPU:0
save/Assign_61 (Assign) /device:GPU:0

2019-07-19 10:34:52.891648: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv1/biases/Initializer/zeros (Const)
stage_6/mid_conv1/biases (VariableV2) /device:GPU:0
stage_6/mid_conv1/biases/Assign (Assign) /device:GPU:0
stage_6/mid_conv1/biases/read (Identity) /device:GPU:0
save/Assign_60 (Assign) /device:GPU:0

2019-07-19 10:34:52.891913: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv2/weights/Initializer/random_uniform/shape (Const)
stage_6/mid_conv2/weights/Initializer/random_uniform/min (Const)
stage_6/mid_conv2/weights/Initializer/random_uniform/max (Const)
stage_6/mid_conv2/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_6/mid_conv2/weights/Initializer/random_uniform/sub (Sub)
stage_6/mid_conv2/weights/Initializer/random_uniform/mul (Mul)
stage_6/mid_conv2/weights/Initializer/random_uniform (Add)
stage_6/mid_conv2/weights (VariableV2) /device:GPU:0
stage_6/mid_conv2/weights/Assign (Assign) /device:GPU:0
stage_6/mid_conv2/weights/read (Identity) /device:GPU:0
save/Assign_63 (Assign) /device:GPU:0

2019-07-19 10:34:52.892160: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv2/biases/Initializer/zeros (Const)
stage_6/mid_conv2/biases (VariableV2) /device:GPU:0
stage_6/mid_conv2/biases/Assign (Assign) /device:GPU:0
stage_6/mid_conv2/biases/read (Identity) /device:GPU:0
save/Assign_62 (Assign) /device:GPU:0

2019-07-19 10:34:52.892418: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv3/weights/Initializer/random_uniform/shape (Const)
stage_6/mid_conv3/weights/Initializer/random_uniform/min (Const)
stage_6/mid_conv3/weights/Initializer/random_uniform/max (Const)
stage_6/mid_conv3/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_6/mid_conv3/weights/Initializer/random_uniform/sub (Sub)
stage_6/mid_conv3/weights/Initializer/random_uniform/mul (Mul)
stage_6/mid_conv3/weights/Initializer/random_uniform (Add)
stage_6/mid_conv3/weights (VariableV2) /device:GPU:0
stage_6/mid_conv3/weights/Assign (Assign) /device:GPU:0
stage_6/mid_conv3/weights/read (Identity) /device:GPU:0
save/Assign_65 (Assign) /device:GPU:0

2019-07-19 10:34:52.892673: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv3/biases/Initializer/zeros (Const)
stage_6/mid_conv3/biases (VariableV2) /device:GPU:0
stage_6/mid_conv3/biases/Assign (Assign) /device:GPU:0
stage_6/mid_conv3/biases/read (Identity) /device:GPU:0
save/Assign_64 (Assign) /device:GPU:0

2019-07-19 10:34:52.892935: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv4/weights/Initializer/random_uniform/shape (Const)
stage_6/mid_conv4/weights/Initializer/random_uniform/min (Const)
stage_6/mid_conv4/weights/Initializer/random_uniform/max (Const)
stage_6/mid_conv4/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_6/mid_conv4/weights/Initializer/random_uniform/sub (Sub)
stage_6/mid_conv4/weights/Initializer/random_uniform/mul (Mul)
stage_6/mid_conv4/weights/Initializer/random_uniform (Add)
stage_6/mid_conv4/weights (VariableV2) /device:GPU:0
stage_6/mid_conv4/weights/Assign (Assign) /device:GPU:0
stage_6/mid_conv4/weights/read (Identity) /device:GPU:0
save/Assign_67 (Assign) /device:GPU:0

2019-07-19 10:34:52.893182: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv4/biases/Initializer/zeros (Const)
stage_6/mid_conv4/biases (VariableV2) /device:GPU:0
stage_6/mid_conv4/biases/Assign (Assign) /device:GPU:0
stage_6/mid_conv4/biases/read (Identity) /device:GPU:0
save/Assign_66 (Assign) /device:GPU:0

2019-07-19 10:34:52.893445: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv5/weights/Initializer/random_uniform/shape (Const)
stage_6/mid_conv5/weights/Initializer/random_uniform/min (Const)
stage_6/mid_conv5/weights/Initializer/random_uniform/max (Const)
stage_6/mid_conv5/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_6/mid_conv5/weights/Initializer/random_uniform/sub (Sub)
stage_6/mid_conv5/weights/Initializer/random_uniform/mul (Mul)
stage_6/mid_conv5/weights/Initializer/random_uniform (Add)
stage_6/mid_conv5/weights (VariableV2) /device:GPU:0
stage_6/mid_conv5/weights/Assign (Assign) /device:GPU:0
stage_6/mid_conv5/weights/read (Identity) /device:GPU:0
save/Assign_69 (Assign) /device:GPU:0

2019-07-19 10:34:52.893702: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv5/biases/Initializer/zeros (Const)
stage_6/mid_conv5/biases (VariableV2) /device:GPU:0
stage_6/mid_conv5/biases/Assign (Assign) /device:GPU:0
stage_6/mid_conv5/biases/read (Identity) /device:GPU:0
save/Assign_68 (Assign) /device:GPU:0

2019-07-19 10:34:52.893966: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv6/weights/Initializer/random_uniform/shape (Const)
stage_6/mid_conv6/weights/Initializer/random_uniform/min (Const)
stage_6/mid_conv6/weights/Initializer/random_uniform/max (Const)
stage_6/mid_conv6/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_6/mid_conv6/weights/Initializer/random_uniform/sub (Sub)
stage_6/mid_conv6/weights/Initializer/random_uniform/mul (Mul)
stage_6/mid_conv6/weights/Initializer/random_uniform (Add)
stage_6/mid_conv6/weights (VariableV2) /device:GPU:0
stage_6/mid_conv6/weights/Assign (Assign) /device:GPU:0
stage_6/mid_conv6/weights/read (Identity) /device:GPU:0
save/Assign_71 (Assign) /device:GPU:0

2019-07-19 10:34:52.894212: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv6/biases/Initializer/zeros (Const)
stage_6/mid_conv6/biases (VariableV2) /device:GPU:0
stage_6/mid_conv6/biases/Assign (Assign) /device:GPU:0
stage_6/mid_conv6/biases/read (Identity) /device:GPU:0
save/Assign_70 (Assign) /device:GPU:0

2019-07-19 10:34:52.894468: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Mul: CPU XLA_CPU XLA_GPU
Add: CPU XLA_CPU XLA_GPU
Sub: CPU XLA_CPU XLA_GPU
RandomUniform: CPU XLA_CPU XLA_GPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv7/weights/Initializer/random_uniform/shape (Const)
stage_6/mid_conv7/weights/Initializer/random_uniform/min (Const)
stage_6/mid_conv7/weights/Initializer/random_uniform/max (Const)
stage_6/mid_conv7/weights/Initializer/random_uniform/RandomUniform (RandomUniform)
stage_6/mid_conv7/weights/Initializer/random_uniform/sub (Sub)
stage_6/mid_conv7/weights/Initializer/random_uniform/mul (Mul)
stage_6/mid_conv7/weights/Initializer/random_uniform (Add)
stage_6/mid_conv7/weights (VariableV2) /device:GPU:0
stage_6/mid_conv7/weights/Assign (Assign) /device:GPU:0
stage_6/mid_conv7/weights/read (Identity) /device:GPU:0
save/Assign_73 (Assign) /device:GPU:0

2019-07-19 10:34:52.894722: W tensorflow/core/common_runtime/colocation_graph.cc:1016] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
/job:localhost/replica:0/task:0/device:CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=-1 requested_device_name_='/device:GPU:0' assigned_device_name_='' resource_device_name_='/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
Assign: CPU
Identity: CPU XLA_CPU XLA_GPU
VariableV2: CPU
Const: CPU XLA_CPU XLA_GPU

Colocation members, user-requested devices, and framework assigned devices, if any:
stage_6/mid_conv7/biases/Initializer/zeros (Const)
stage_6/mid_conv7/biases (VariableV2) /device:GPU:0
stage_6/mid_conv7/biases/Assign (Assign) /device:GPU:0
stage_6/mid_conv7/biases/read (Identity) /device:GPU:0
save/Assign_72 (Assign) /device:GPU:0

2019-07-19 10:34:53.181507: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
Load model 0:00:10.898345
1st
[(179.0, 299.0), (107.0, 228.0), (164.0, 140.0), (228.0, 131.0), (244.0, 155.0), (268.0, 188.0)]

Process finished with exit code 0

@chengchu88
Copy link

I got the same problem too.

@AmitMY
Copy link
Author

AmitMY commented Jul 19, 2019

Actually, works fine on the same machine hardware with CUDA Version 10.0.130
It doesn't work on CUDA Version 8.0.61

My bad, I checked CUDA version on the wrong server when reporting the issue.

Is this version of Tensorflow even supposed to work for v8? if not, I'll close this issue.

@sanjoy
Copy link
Contributor

sanjoy commented Jul 19, 2019

The failure is almost certainly because of the cuda version, going by the log:

2019-07-19 10:34:52.807368: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807494: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807617: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807745: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807868: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.807989: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory
2019-07-19 10:34:52.815678: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-07-19 10:34:52.815715: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2

The build probably bakes in TF_CUDA_VERSION as 10. I'm not sure if we have a build that supports older cuda version, but you could try building one for yourself to check (configure.py should ask you for the cuda version).

@chengchu88
Copy link

chengchu88 commented Jul 19, 2019 via email

@alaayadi
Copy link

alaayadi commented Jul 24, 2019 •
edited

Same problem here , not detecting any GPU device after upgrading the tensorflow version
tensorflow only detect GPU XLA
But now , I cannot go back to tensorflow1.10.0 as it was been deleted from pip
Any help ? cuda version V9.0.176

@chengchu88
Copy link

chengchu88 commented Jul 24, 2019 via email

@alaayadi
Copy link

I downgraded the tensorflow to an older version and the problem is resolved.. command is like conda install tensorflow-gpu==1.9.0
…
On Wed, Jul 24, 2019 at 2:20 PM alaayadi @.***> wrote: Same problem here , not detecting any GPU device after upgrading the tensorflow version tensorflow only detect GPU XLA But now , I cannot go back to tensorflow1.10.0 as it was been deleted from pip Any help ? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#30748?email_source=notifications&email_token=AJKLC6C56BDJDTRSDWU762LQBDBQFA5CNFSM4ID7TGW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2XUDJA#issuecomment-514802084>, or mute the thread https://github.com/notifications/unsubscribe-auth/AJKLC6H37AS6XXGYE6MMM63QBDBQFANCNFSM4ID7TGWQ .

That's just working ! thanks

@albertNod
Copy link

albertNod commented Jul 25, 2019 •
edited

If you have cuda 10.1 you need to update cuDNN to 7.6.2
https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html

That's what got it working for me without downgrading Tensorflow

@ir0nt0ad
Copy link

For me, it was the placement of ArgMax ops on GPUs that broke it; relocating them to CPU did the trick.

@albertNod
Copy link

For me, it was the placement of ArgMax ops on GPUs that broke it; relocating them to CPU did the trick.

I'm having the same issue after accidentally upgrading CUDA and then downgrading again, how did you manage to relocate the ArgMax ops to CPU?

@ir0nt0ad
Copy link

@albertNod something along these lines: https://github.com/aymericdamien/TensorFlow-Examples/blob/042c25ce2c3d91bf5a2e0a308fea578b1e290f82/examples/6_MultiGPU/multigpu_cnn.py#L110
just add 'ArgMax' to the list

@morpheusthewhite morpheusthewhite mentioned this issue Sep 20, 2019
Closed
@penguinbing
Copy link

I met the same problem on ubuntu 18.04, cuda 10.1 and Tensorflow 1.14.0. However, I uninstalled the pip version tensorflow using pip uninstall tensorflow-gpu and then use conda install -c anaconda tensorflow-gpu to install conda version, and it works for me. You can have a try. @AmitMY

@AmitMY AmitMY closed this as completed Sep 29, 2019
@tensorflow-bot
Copy link

Are you satisfied with the resolution of your issue?
Yes
No

@hur-kyuh-leez
Copy link

I met the same problem on ubuntu 18.04, cuda 10.1 and Tensorflow 1.14.0. However, I uninstalled the pip version tensorflow using pip uninstall tensorflow-gpu and then use conda install -c anaconda tensorflow-gpu to install conda version, and it works for me. You can have a try. @AmitMY

works great!
Also, if you run into out of memory error
CUDA runtime implicit initialization on GPU:0 failed. Status: out of memory
reducing the batch size will fix the problem

@KunpengSong
Copy link

I met the same problem on ubuntu 18.04, cuda 10.1 and Tensorflow 1.14.0. However, I uninstalled the pip version tensorflow using pip uninstall tensorflow-gpu and then use conda install -c anaconda tensorflow-gpu to install conda version, and it works for me. You can have a try. @AmitMY

Thank you! Saved my day. I'm on ubuntu 16.04, cuda 10.1. And this works:
pip uninstall tensorflow
pip uninstall tensorflow-gpu
conda install tensorflow-gpu==1.13.1
Then check GPU availability by example

@albertz albertz mentioned this issue Nov 21, 2019
Closed
@beeRitu beeRitu mentioned this issue Jan 9, 2020
Open
@ikasumi
Copy link

ikasumi commented Mar 12, 2020

By installing CUDA 10.0, this problem may be solved.

sudo apt-get install --no-install-recommends \
    cuda-10-0 \
    libcudnn7=7.6.5.32-1+cuda10.0  \
    libcudnn7-dev=7.6.5.32-1+cuda10.0

@t-thanh
Copy link

t-thanh commented May 23, 2020 •
edited

I have similar issue with Tensorflow and Keras. And, don't have any issue with pytorch:

  • Tensorflow 2.2
  • CUDA: 10.2
  • CUDNN: 7.6.5

Notice that I got feedback Could not load dynamic library 'libcudart.so.10.1 when checking GPU with test.is_gpu_available().

Since I got libcudart.so.10.2 installed, I solved this issue by: make link libcudart.so.10.1 from libcudart.so.10.2 in /usr/local/cuda-10.2/targets/x86_64-linux/lib and make sure that /usr/local/cuda-10.2/targets/x86_64-linux/lib is in LD_LIBRARY_PATH.

Update: got similar issue again in jupyter, need to close, then start jupyter again, but doesn't solve CUDNN_STATUS_INTERNAL_ERROR
issue 100%. Switch to train with .py >> it works

See more in this repository

@Saduf2019 Saduf2019 mentioned this issue Jun 15, 2020
Closed
@gitgithan
Copy link

gitgithan commented Jun 21, 2020 •
edited 

import ctypes

cuda_path = "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.2\\bin\\"
to_load = ["cudart64_102.dll",
           "cublas64_10.dll",
           "cufft64_10.dll",
           "curand64_10.dll",
           "cusolver64_10.dll",
           "cusparse64_10.dll",
           "cudnn64_7.dll"]

for dll in to_load:
    ctypes.WinDLL(cuda_path+dll)

Doesn't matter CUDA 10.0 , 10.1 10.2, for all dll not found, manually load them into ram. I have done all the setting PATH environment variable and nothing worked for me except this.
My setup is Windows 10 RTX2070 tf2.2 cuda 10.2 cudnn 7.6.5 avx2 from https://github.com/fo40225/tensorflow-windows-wheel.

I usually use wheels built by others because official tensorflow software builds lag the hardware (tf2.2 is only tested up to cudnn 7.4 from https://www.tensorflow.org/install/source_windows). Last year June when i was trying to install tf1.4, the official tf only went up to 10.0 but my GPU required 10.1

EDIT
I stopped doing this hack. The issue was python installed from Windows Store could not read PATH. Solution is install python from https://www.python.org/

@gitgithan gitgithan mentioned this issue Jun 21, 2020
Closed
@sanjoy sanjoy removed their assignment Jun 21, 2020
@drasmuss drasmuss mentioned this issue Jul 15, 2020
Closed
@zhihou7 zhihou7 mentioned this issue Sep 7, 2020
Closed
@Saduf2019 Saduf2019 mentioned this issue Jul 18, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
comp:gpu GPU related issues TF 1.14 for issues seen with TF 1.14 type:support Support issues
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests