_replace_atomic not working on Windows 10

[kiudee]

I write to a file as follows:

with AtomicWriter(data_path, mode="wb", overwrite=True).open() as f:
    np.savez_compressed(f, arr)

On linux this works, but on Windows I receive the following error:

  File "C:\Users\Karlson\Anaconda3\envs\chess-tuning-tools\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "C:\Users\Karlson\Anaconda3\envs\chess-tuning-tools\lib\site-packages\atomicwrites\__init__.py", line 169, in _open
    self.commit(f)
  File "C:\Users\Karlson\Anaconda3\envs\chess-tuning-tools\lib\site-packages\atomicwrites\__init__.py", line 202, in commit
    replace_atomic(f.name, self._path)
  File "C:\Users\Karlson\Anaconda3\envs\chess-tuning-tools\lib\site-packages\atomicwrites\__init__.py", line 99, in replace_atomic
    return _replace_atomic(src, dst)
  File "C:\Users\Karlson\Anaconda3\envs\chess-tuning-tools\lib\site-packages\atomicwrites\__init__.py", line 81, in _replace_atomic
    _windows_default_flags | _MOVEFILE_REPLACE_EXISTING
  File "C:\Users\Karlson\Anaconda3\envs\chess-tuning-tools\lib\site-packages\atomicwrites\__init__.py", line 76, in _handle_errors
    raise WinError()
PermissionError: [WinError 5] Access is denied

The script is located in the home folder and should not be run with admin privileges.

Using the following versions:

atomicwrites              1.4.0                      py_0
numpy                     1.18.1           py37h93ca92e_0
python                    3.7.6                h60c2a47_2

[untitaker]

@kiudee could it be that your file is just created in a folder you have no permissions for? I'm not entirely sure what I need to do to repro this.

[kiudee]

No, a normal

np.savez_compressed(path, arr)

works, which is why I use the following workaround for now:

if os.name == "nt":
    np.savez_compressed(data_path, arr)
else:
    with AtomicWriter(data_path, mode="wb", overwrite=True).open() as f:
        np.savez_compressed(f, arr)

edit: I will try if I can reproduce the problem in different environments and folders.

[kiudee]

I was able to reproduce the problem now using the following steps:

import numpy as np
from atomicwrites import AtomicWriter

arr = np.linspace(0, 1, num=10)
# Works without a problem:
with AtomicWriter("test.npz", mode="wb", overwrite=True).open() as f:
    np.savez_compressed(f, arr)

data = np.load("test.npz")
# PermissionError happens here:
with AtomicWriter("test.npz", mode="wb", overwrite=True).open() as f:
    np.savez_compressed(f, arr)

Reading up on numpy.load(), it turns out that in the case of .npz (i.e. compressed) files it actually opens the file and it needs to be explicitly closed again. I wonder why the PermissionError does not happen on Linux systems.

But since this is not an issue with atomicwrites, I will close the issue.