A lot of unittests tests fail under Windows due to usage of NamedTemporaryFile

[Chary314]

I noticed, that a lot of tests fail under Windows due to usage of NamedTemporaryFile.

e.g.

  File "C:\_code\py\2009 beancount fix\beancount\beancount\utils\text_utils_test.py", line 28, in test_fix_ampersand
    tidy_file = text_utils.entitize_ampersand(input_file.name)
  File "C:\_code\py\2009 beancount fix\beancount\beancount\utils\text_utils.py", line 51, in entitize_ampersand
    with open(filename) as infile:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\chary\\AppData\\Local\\Temp\\tmpgas9aode'

Most likely it is related to different functionality of NamedTemporaryFile under Linux and Windows
As described here, NamedTemporaryFile actually creates and opens the file for you. Unix allows you to open it second time, Windows (starting from NT) apparently not.

But I can see in the code, that the usage of NamedTemporaryFile in tests already opens the file, hence operational code does not work later (on Windows)

E.g.:
beancount/utils/text_utils_test.py

    def test_fix_ampersand(self):
        #Chary: Temporary file with the name input_file is created and open here 
        with tempfile.NamedTemporaryFile('w') as input_file:  
            input_file.write('S&P 500. Some other thing and &entity;')
            input_file.flush()
            # Chary: The name of the temporary file is then gets passed to operational code, whilst the file itself is still open
            # one can try to do a input_file.close() here, but this is not the solution, 
            # as NamedTemporaryFile functionality is that file gets deleted immediately on closure 
            tidy_file = text_utils.entitize_ampersand(input_file.name) 

and then in operational code
beancount/utils/text_utils.py

def entitize_ampersand(filename):
   """    
    Chary: some code removed here
   """
    # Chary: an attempt to open file again is done here, in operational code. This attempt fails on Windows 
    with open(filename) as infile: 

Apparently it has been recognized, that it is difficult to write a cross-platform code with NamedTemporaryFile and there is an existing enhancement 14243 for python open since 2012

One of the possible googled solutions is not to use NamedTemporaryFile to open the file but only to generate candidate name as described here.

This will require however to manually take care of deleting file afterwards, which is in the current code a job of a context manager

[josephw]

For that specific failure, this commit suggests removing that method entirely. In most other cases, this suggestion to use:

tmpf = NamedTemporaryFile(delete=False)
...
tmpf.close()
os.unlink(tmpf.name)

is probably the least impactful targeted workaround.

[Chary314]

There is also now an open PR (python/cpython#22431) to have this python issue 14243 solved in the python tempfile library directly

[blais]

I have to admit I'm not a big fan of their solution... (commented). It means you'd have to use this new parameter everywhere. Maybe to wrap the contextmanager in a function. Meh.

…

On Mon, Oct 5, 2020 at 10:59 AM Chary314 ***@***.***> wrote: There is also now an open PR to have this python issue 14243 <https://bugs.python.org/issue14243> solved in python tempfile library directly python/cpython#22431 <python/cpython#22431> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#550 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACSBE5XWEPZYC7YKKHUDSDSJHNLLANCNFSM4Q52IB2Q> .

[Ev2geny]

I think this is a duplicate issue for the previously created one #222

Any way, in python 3.12 there is now new functionality, which allows to re-write the tests in a way that they will work on Windows as well:

The tempfile.NamedTemporaryFile function has a new optional parameter delete_on_close ( gh-58451)