Permission Error in Windows

[Skorkmaz88]

Tensoflow cloud version: (0.1.5)
Python version : 3.7.6
OS: Windows 10 64-bit

When running from a notebook:

import tensorflow_cloud as tfc
tfc.run(entry_point='mnist.py',
        docker_image_bucket_name="a-test",
)

mnist.py file is as in example :

import tensorflow as tf

(x_train, y_train), (_, _) = tf.keras.datasets.mnist.load_data()

x_train = x_train.reshape((60000, 28 * 28))
x_train = x_train.astype('float32') / 255

model = tf.keras.Sequential([
  tf.keras.layers.Dense(512, activation='relu', input_shape=(28 * 28,)),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(loss='sparse_categorical_crossentropy',
              optimizer=tf.keras.optimizers.Adam(),
              metrics=['accuracy'])

model.fit(x_train,  y_train, epochs=10, batch_size=128)

Result:

INFO:tensorflow_cloud.core.containerize:Uploading files to GCS.

INFO:tensorflow_cloud.core.containerize:Building and publishing docker image using Google Cloud Build: gcr.io/dydra- 282320/tf_cloud_train:371ef57f_1db8_4c96_aae6_09e373926732

---

PermissionError Traceback (most recent call last)
in
1 import tensorflow_cloud as tfc
2 tfc.run(entry_point='mnist.py',
----> 3 docker_image_bucket_name="a-test",
4 )

~\anaconda3\lib\site-packages\tensorflow_cloud\core\run.py in run(entry_point, requirements_txt, distribution_strategy, docker_base_image, chief_config, worker_config, worker_count, entry_point_args, stream_logs, docker_image_bucket_name, job_labels, **kwargs)
224 # Delete all the temporary files we created.
225 if preprocessed_entry_point is not None:
--> 226 os.remove(preprocessed_entry_point)
227 for f in container_builder.get_generated_files():
228 os.remove(f)

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\semih\AppData\Local\Temp\tmpkjul06z0.py'

[christianversloot]

Running into the same issue. Did you manage to find a solution?

[christianversloot]

It could be that this issue is caused because the temporary files created with mkstemp() are not closed before they are removed. This has resulted in similar PermissionErrors in other settings: https://stackoverflow.com/a/49369487

I'm currently editing the TF Cloud source code locally to see whether I can make things work (this requires me to change a few things here and there). If so, I'll make a pull request.

[christianversloot]

This commit will fix this issue: https://github.com/christianversloot/cloud/commit/f6cf754bf5168de039d98a177e40829682641338
(followed by bc2e521, so perhaps it's better to use the master:HEAD)

Jobs get submitted correctly:

INFO:tensorflow_cloud.core.containerize:Successfully tagged gcr.io/tf-cloud-292417/tf_cloud_train:8b281834_3eba_4e00_a912_cd41c45893bf
INFO:tensorflow_cloud.core.containerize:Publishing docker image: gcr.io/tf-cloud-292417/tf_cloud_train:8b281834_3eba_4e00_a912_cd41c45893bf

Job submitted successfully.