PermissionError: [Errno 13] Permission denied --onefile FileIO Issue #6467
Comments
|
Do you have an anti-virus program running on the system? This looks like an AV blocking the access to the exe file... |
I just have windows-defender, and tried with disabling it, too. However, issue still exists. |
|
Also, I used this program in the past, and it was working. It got broken after some changes I think. |
The rest of your system is not exactly frozen in time, either.
If you can pinpoint the breaking change (using git bisect), we can investigate further. Otherwise, I'm inclined to believe that this is specific to your system/environment; if it was the case of a file not being closed fast enough, as you suspect, then our CI would be falling apart as well, and we'd be likely seeing more reports like this. The ones that we do get, however, typically boil down to AV, e.g., #6463. But even if this is a legit issue in PyInstaller, I don't think retry loop is a proper solution... |
|
I am also having the permission denied issue: I tried disabling the antivirus but it's the same. |
|
Does bouncing back a PyInstaller version or two make any difference? Edit: I see you tried with |
|
@bwoodsend No I have not. Which version do you recommend? |
|
I created the exe after using 4.4 build of py installer, but when I run it, I get this error I used this command : |
You'd get that error with any version of PyInstaller; what you need to do is ensure that data files of Back to the original topic, I suppose 4.4 working means that changes made in #6252 (and released as part of 4.6) might have made us less resilient to AV software's interference: previously, we created a copy of stock bootloader exe to a temporary file, then added icon, resources and manifest. Then we copied the modified exe from the temporary location into final destination and at the same time appended the pkg (without closing the file descriptor in between these two steps). So I suppose if AV kicks in after manifest is embedded, but locks the file only for modification (i.e., still allowing reading/copying), we might copy the prepared bootloader without issues in spite of the AV. And because both the copy and PKG appending is done without closing the file descriptor in between, AV may kick in again only after the whole procedure is complete. In the simplified, post-#6252 pipeline, we make a copy of stock bootloader exe to the file destination, add icon, resources and manifest, and then append PKG to it. This means that if AV kicks in after manifest is embedded and locks the file for writing, we may fail to open it again in append mode to append the PKG to it. So perhaps we should go back to using an intermediate temporary file to prepare the bootloader... |
|
Now I am getting this error : |
So you need to also add You should probably also add a At any rate, this has nothing to do with the permission denied problem, so if you have any further problems, open a new discussion. |
Yeah, I agree. It could be that AV is forcing a scan due to the tainted hash of the original bootloader copy (and suppose that it fails the race when we are embedding icon, resources and manifest, so it scans it after that). I wish I could reproduce this, but no luck... @fatihkabakk @Aarekaz Does the permission-denied problem also occur if you add |
I will try this. Do I add all those parameters in the same line? |
Haven't tried this. Will look into it. |
|
The simple solution is to not give a to : After all is done, I just renamed the executable file. p.s. |
|
@ArtemConstantinov thanks for the fix. Effectively that is the problem, the And of course, I can confirm this bug using version |
@rokm I just tried this and the same error happens. The only solution is to get rid of the |
|
As you can see, the problem has been fixed (or rather, dealt with, as it's not a bug on I've made a very crude modification to |
|
I'd prefer the changing the suffix option to injecting pauses and retries. It sounds less likely spring back at us the next time an AV engine update occurs. |
|
One question to those saying that they've disabled their antivirus: Which did you disable? It's rare that Windows machines only have one. Microsoft always puts theirs on but most laptop providers are normally endorsed to install some other (invariably the most useless) one too. e.g. Dell and Lenovo come with MacAffee installed and I believe that HPs come with AVG. |
|
A change in suffix and a later rename sounds good, too, as long as it does not fires the race condition in Windows Defender. And yes, that race should not happen in the first place, and it's not My only real concern with pausing and retrying is the duration of the pause. To me, one second sounds great, but timing a race condition can lead to problems later, you are right. |
I only have Windows Defender, which BTW is quite difficult to completely disable, and the problem still persists even when disabling it, adding the involved folders and binaries to the exceptions list, or even killing the service. And it's not a good solution IMHO, users should not have to disable their AV in order to run a program. |
Yeah I agree. I just idealistically say that an AV issue should be fixed on the AV's side or the AV should be replaced instead of ducked and dodged around by any compiler or build tool that happens to want to copy an executable file and then modify it - particularly when the AV is provided by a company with as many resources as Microsoft and the build tool is maintained by two volunteers who don't even use Windows. Rant ranty rant rant... I think that this magic pause explains things a bit though. Presumably, when PyInstaller first writes the executable an AV scan is triggered. The AV has to open the file in read mode whilst it scans it and, because on Windows attempting to write to a file that is already open by some other application leads to a permission error, PyInstaller hits this error when it tries to start modifying it again. The short delay is presumably just how long it takes for the scan to complete and the AV to close the file so that PyInstaller can start writing to it again. That would at least explain why #6468 did anything at all - after the 500th or so attempt to open the file, the AV scan had finished so the next attempt worked. So I guess, all we really have to do is either keep the file handle open from start to finish (although I don't think that the bit of Windows API that adds the icon allows this) or just accommodate this pause by merging #6468 (preferably with some |
|
I don't think magic pause is a solution, because we (re)open the file for modification in several places, and the race can occur at any of them (e.g., this report is about appending PKG to exe, while #6506 is about a later stage, when we modify the build timestamp and set exe chekcsum). I.e., there's no reason to believe that re-scan does not occur after every modification. I'd also prefer avoiding the delay loops at every point where we (re)open the file, because those involve different APIs and different abstraction levels (raw open, WinAPI, pefile). I find the idea of changing/removing suffix during the assembly process and then renaming the file the least invasive one. That relies on the current behavior of the AV, though (i.e., picking on the file based on its extension instead of the PE header). |
It's not a rant, I totally agree with you, Microsoft should do their laundry here and fix this issue because it's not normal at all you cannot modify an executable you just created. This said, we both know they are not going to fix it anytime soon and meanwhile it's I agree with @rokm and as long as the AV does not look at the headers but the suffix, it's better to just avoid that suffix until the very last file write. I think it's quite pragmatic and it's easier to implement. |
|
Changing the suffix it is. I'll try to resist the urge to set the temporary suffix to something immature like |
Can I +1 that temporary suffix? Because I utterly love it, maybe out of immaturity on my part but 🤣 Now seriously, thanks a lot for dealing with this. |
Whenever an intermediate/temporary executable file needs to be written, give it the filename extension .notanexecutable instead of .exe. This fools Microsoft Defender into thinking that the file is not an executable which requires scanning. Even though the scan does not quarantine the EXE, its being opened and read by Defender means that it can't be opened in write mode by a later step in the executable assembly process. I'm choosing to add this suffix on all platforms despite it only being needed on Windows so as to minimise platform dependent logic.
Whenever an intermediate/temporary executable file needs to be written, give it the filename extension .notanexecutable instead of .exe. This fools Microsoft Defender into thinking that the file is not an executable which requires scanning. Even though the scan does not quarantine the EXE, its being opened and read by Defender means that it can't be opened in write mode by a later step in the executable assembly process. I'm choosing to add this suffix on all platforms despite it only being needed on Windows so as to minimise platform dependent logic.
Whenever an intermediate/temporary executable file needs to be written, give it the filename extension .notanexecutable instead of .exe. This fools Microsoft Defender into thinking that the file is not an executable which requires scanning. Even though the scan does not quarantine the EXE, its being opened and read by Defender means that it can't be opened in write mode by a later step in the executable assembly process. I'm choosing to add this suffix on all platforms despite it only being needed on Windows so as to minimise platform dependent logic.
|
Can someone with this issue give the fix a test? Then try building something as normal. |
|
@bwoodsend , your fix is quite similar to the one I was testing, and it works perfectly in a computer where I could reproduce the problem all the time. Thanks A LOT for the fix! That was fast! If you need me to perform any other test, I've set up a separate venv for testing this in one of the machines where I can reproduce the problem. |
BTW, I just noticed a typo in Notice Emedding rather than Embedding. If you prefer I open a new PR for this issue so it is addressed separately, just let me know. |
|
I wouldn't bother chasing spelling errors. PyInstaller is littered with them. One day I'll get bored enough to run |
Whenever an intermediate/temporary executable file needs to be written, give it the filename extension .notanexecutable instead of .exe. This fools Microsoft Defender into thinking that the file is not an executable which requires scanning. Even though the scan does not quarantine the EXE, its being opened and read by Defender means that it can't be opened in write mode by a later step in the executable assembly process. I'm choosing to add this suffix on all platforms despite it only being needed on Windows so as to minimise platform dependent logic.
Whenever an intermediate/temporary executable file needs to be written, give it the filename extension .notanexecutable instead of .exe. This fools Microsoft Defender into thinking that the file is not an executable which requires scanning. Even though the scan does not quarantine the EXE, its being opened and read by Defender means that it can't be opened in write mode by a later step in the executable assembly process. I'm choosing to add this suffix on all platforms despite it only being needed on Windows so as to minimise platform dependent logic.
Whenever an intermediate/temporary executable file needs to be written, give it the filename extension .notanexecutable instead of .exe. This fools Microsoft Defender into thinking that the file is not an executable which requires scanning. Even though the scan does not quarantine the EXE, its being opened and read by Defender means that it can't be opened in write mode by a later step in the executable assembly process. I'm choosing to add this suffix on all platforms despite it only being needed on Windows so as to minimise platform dependent logic.
|
Fixed in #6629. Next release will probably be in a few days so you shouldn't have to wait long. |
Whenever an intermediate/temporary executable file needs to be written, give it the filename extension .notanexecutable instead of .exe. This fools Microsoft Defender into thinking that the file is not an executable which requires scanning. Even though the scan does not quarantine the EXE, its being opened and read by Defender means that it can't be opened in write mode by a later step in the executable assembly process. I'm choosing to add this suffix on all platforms despite it only being needed on Windows so as to minimise platform dependent logic.
Description of the issue
When creating single exe file in
--onefilemode, the file\lib\site-packages\PyInstaller\building\api.pyatline 734,with open(self.name, 'ab') as outf:the program raises a PermissionError.I have tried adding while loop and an exception catcher, it looped 509 times before properly opening the file.
I suppose that file does not get closed completely before the append process starts.
I would make a pull request about this problem, but I'm not completely sure how to solve this problem properly.
It can be solved without including sleep, but then program loops so many times (509 as stated before).
My fix is added below;
timemodule)My Program's Import Details (None of Importance I think, see A minimal example program which shows the error)
Context information (for bug reports)
Output of
pyinstaller --version:4.7(Tried with development version (5.0), too)Version of Python: 3.9
Platform: Windows (Turkish Q) (chcp 65001 [utf-8])
How you installed Python: python.org/downloads
Did you also try this on another platform? Does it work there? No, not relevant.
try the latest development version, using the following command:
(https://github.com/pyinstaller/pyinstaller/wiki/If-Things-Go-Wrong) and
Make sure everything is packaged correctly
--noupxor setupx=Falsein your .spec-file--debugtopyi-makespecorpyinstalleror useEXE(..., debug=1, ...)in your .spec file.A minimal example program which shows the error
.pyfilepyinstaller --onefile x.py, raises the error.Stacktrace / full error message
Please also see https://github.com/pyinstaller/pyinstaller/wiki/How-to-Report-Bugs
for more about what would use to solve the issue.
The text was updated successfully, but these errors were encountered: