bpo-14243: Optionally delete NamedTemporaryFile on content manager exit but not on file close

[Ev2geny]

This fixes this issue: https://bugs.python.org/issue14243

tempfile.NamedTemporaryFile is too hard to use portably when you need to open the file by name after writing it. To do that, you need to close the file first (on Windows), which means you have to pass delete=False, which in turn means that you get no help in cleaning up the actual file resource,

Hence at the moment there is no out of the box solution to use tempfile.NamedTemporaryFile on Windows in such scenario (which is often used in unit testing):

• in test module:

1. create and open temporary file
2. write data to it
3. pass name of the temporary file to the operational code

• In operational code, being tested

1. open file, using name of the temporary file
2. read data from this temporary file

In this Pull Request the issue is solved by adding an additional optional argument to NamedTemporaryFile() 'delete_on_close' (default is True). It works in combination with already existing argument 'delete', and determines whether created temporary file gets deleted on close (which until this change was the only functionality available and it remains default now).

If 'delete' = True and 'delete_on_close' = False then temporary file gets deleted on context manager exit only (which is a new functionality).

If 'delete' is not true, then value of 'delete_on_close' is ignored.

So, the change shall be fully backwards compatible.

In the bug discussion such option was discussed by r.david.murray but mainly by eryksun

I have also added several unit tests to test new functionality in different combinations

https://bugs.python.org/issue14243

[the-knights-who-say-ni]

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find a bugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@Ev2geny

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

[Ev2geny]

I just can't figure out why bedevere/issue-number check is failing. Where do I have to mention it?

[blais]

Hmm, I'm not sure I like the API in this solution. If I want to write portable code (Linux and Windows), I would then have to use delete=True and delete_on_close=False in every since call, right? This is going to look at bit convoluted. I'd much rather this was a distinct type of NamedTemporaryFile.

[Ev2geny]

Hmm, I'm not sure I like the API in this solution. If I want to write portable code (Linux and Windows), I would then have to use delete=True and delete_on_close=False in every since call, right? This is going to look at bit convoluted. I'd much rather this was a distinct type of NamedTemporaryFile.

@blais , yes, this PR assumes, you would need to use delete_on_close=False (delete=True is already a default), if you want to write a portable code (Linux and Windows), but only if you need to refer to a created temporary file by its name, not by a file object. And this is a new functionality all together, as current functionality of the NamedTemporaryFile implicitly assumes, that you need to refer to a file by its file object (at least on Windows). Though I agree with you, that in test modules, one needs to often refer to temporary file by its name (hence the PR).

There were a lot of suggestions on how to fix this in the original issue14243 discussion with no clear conclusion on what is the best approach. I have chosen this one as it is backwards compatible and does not change philosophy of existing code. I will wait for code reviews to provide their feedback. There must be someone, who takes a final decision on this.

[blais]

It would be wiser to create an alternative interface for this that is safe to use between platforms by default, even if it is more constrained, even if that means providing a completely different context manager.

[bozhinov]

OK. So I read the comments in the bug and the workaround here.
Here is what I did though (on Windows) - Created a temporary directory - put my temp file in there - script ends - all is gone
Hope that helps

[willingc]

@zooba Do you have time to take a look at this PR?

[Ev2geny]

@zooba , just checking whether you managed to look at this

[MaxwellDupre]

Can you re-target for 3.12 (Doc/whatsnew/3.10.rst) please?
Also, in NEWS I don't think you need double back ticks (``).

[AlexWaygood]

Also, in NEWS I don't think you need double back ticks (``).

@MaxwellDupre this is incorrect. Double backticks are usually preferred over single backticks in .rst files, unless a specific ReST role is being used.

Please read https://devguide.python.org/documenting/ for a full guide on ReST syntax.

[zooba]

Code change looks good, but we need to get the docs in order.

[zooba]

This is way more details than we want here - this will end up in a list with 1000 other changes.

Most of this should move to specific documentation, and then this entry can link to it with :class:`NamedTemporaryFile`

[Ev2geny]

This is fixed. The spesific documentation is updated and linked to with :class:tempfile.NamedTemporaryFile``

[zooba]

Needs to move to 3.12.rst, and can be about half the length of what's currently in the NEWS file

[Ev2geny]

Implemented. Whats new is moved to 3.12.rst and only 2 lines now

[bedevere-bot]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[Ev2geny]

Looks like you made a git mistake. Hopefully someone else can help you (maybe just close this PR and create a new one from a fresh clone of the main branch).

Yes, this was something with git. I merged master branch into my PR, but it marked lots of files as being changed. I reverted this meanwhile. But I need some guidelines how to move forward.
Since there is a conflict between my PR and a master branch I guess I have to merge or rebase it with master somehow

[terryjreedy]

Below is the the fixed up Improved Modules section, with each item in proper alphabetical order. I could not commit it because git requires all conflicts fixed in one merge, and I have no idea how to merge your tempfile change with what someone else did since your patch. Use git log or git blame to find the issue that made the alternative change and what it was meant to do.

replacement text

Improved Modules ================

dis

• Pseudo instruction opcodes (which are used by the compiler but
  do not appear in executable bytecode) are now exposed in the
  :mod:dis module.
  :data:~dis.HAVE_ARGUMENT is still relevant to real opcodes,
  but it is not useful for pseudo instructions. Use the new
  :data:~dis.hasarg collection instead.
  (Contributed by Irit Katriel in :gh:94216.)

os

• Add :data:os.PIDFD_NONBLOCK to open a file descriptor
  for a process with :func:os.pidfd_open in non-blocking mode.
  (Contributed by Kumar Aditya in :gh:93312.)

pathlib

• Add :meth:~pathlib.Path.walk for walking the directory trees and generating
  all file or directory names within them, similar to :func:os.walk.
  (Contributed by Stanislav Zmiev in :gh:90385.)

sqlite3

• Add a :ref:command-line interface <sqlite3-cli>.
  (Contributed by Erlend E. Aasland in :gh:77617.)

tempfile

The :class:tempfile.NamedTemporaryFile class has a new optional parameter
delete_on_close (Contributed by Evgeny Zorin in :gh:58451.)

threading

• Add :func:threading.settrace_all_threads and
  :func:threading.setprofile_all_threads that allow to set tracing and
  profiling functions in all running threads in addition to the calling one.
  (Contributed by Pablo Galindo in :gh:93503.)

unicodedata

• The Unicode database has been updated to version 15.0.0. (Contributed by
  Benjamin Peterson in :gh:96734).

Note: a few months ago, you did a force push instead of a simple update merge and push. Force pushes sometimes result in the behavior experienced today and are seldom needed. Unclear exactly what you did today. I recommend you fix conflict in your local clone, then merge upstream/main, then push to fork.

[Ev2geny]

@terryjreedy , thanks for this information, but I am afraid I don't think I understand what to do with it and how exactly to move forward.

What I want is to refresh my pull request with whatever is latest in the current main branch. so did merge with the current main, which obviousely introduced a lot of new files, but when I pushed it to github it looks like it made impression, that I tried to modify them, whilst I only transfered them fom the main.
I now have reverted the change, but I still sit with the situation that I want to refresh by PR with whatever is the current main (and yes, a couple of merge conflicts will have to be resolved).

Also, the work on PR is not finished yet, as there were some more recent reviews from @eryksun

[JelleZijlstra]

There are still conflicts. I'd recommend closing this PR and starting over with the changes you want to make.

[Ev2geny]

Note: a few months ago, you did a force push instead of a simple update merge and push. Force pushes sometimes result in the behavior experienced today and are seldom needed. Unclear exactly what you did today. I recommend you fix conflict in your local clone, then merge upstream/main, then push to fork.

Just for the record: back then I did rebasing from the master (rather than merge). And then I had to do force push, if I remember correctly

[Ev2geny]

Based on the suggestions of @gvanrossum and @JelleZijlstra I will close this PR and create a new one

[Ev2geny]

I new PR 97015 has been created to replace this one.