Failure to uninstall egg-link on windows

[shacharoo]

Hey there!

For the past few days I've been struggling with a bug in pydocstyle, which is described in this long and tedious SOVF question.
To sum it up, when I was running in a pypy environment and tried to test setup.py develop and setup.py develop --uninstall, windows raised error 32 saying that the egg-link file can't be removed for it is being used by another process.

I came across this bug report, and in particular:

> The problem is that there's no reason to believe anything
> he did here _does_ leave files open.

Except that he's hitting the file system quite heavily, and
asyncronously.  My guess is that Windows simply gets behind
(a quick filemon test indicates that this is indeed the
case; just before a crash, I see the events CREATE/SUCCESS,
QUERY/SUCCESS, QUERY/SUCCESS, WRITE/SUCCESS, and
OPEN/SHARING VIOLATION for the failing file, with lots of
requests for other files interleaved).

Unless someone wants to fix Windows, a simple workaround
would be to retry the os.remove a few times before giving up
(with a time.sleep(0.1) in between).

I opened setuptools\command\develop.py and in uninstall_links (line 152) I saw this:

egg_link_file = open(self.egg_link)
contents = [line.rstrip() for line in egg_link_file]
egg_link_file.close()
...
if not self.dry_run:
    os.unlink(self.egg_link)

So I thought to myself, "maybe the file is being held by the same process? perhaps windows doesn't clear the resources fast enough?". So I added this bit of code to it:

if not self.dry_run:
    for i in range(5):
        try:
            os.unlink(self.egg_link)
            break
        except Exception:
            import time
            time.sleep(0.1)

And... voila.

(pypy) C:\Users\shach\code\bla\funniest>pypy setup.py develop --uninstall
running develop
Removing c:\users\shach\code\bla\funniest\.tox\pypy\site-packages\funniest.egg-link (link to .)
Removing funniest 0.1 from easy-install.pth file

I'd open a PR, but I seriously doubt that that's the way to go about it. I'm open to suggestions and will happily send a PR to fix it 😃

Shachar.

EDIT:

Well, obviously I had to emberrass myself an write a bug in my fix 😊

The error still happens after changing "fixing" the "fix" to:

if not self.dry_run:
    for i in range(5):
        try:
            os.unlink(self.egg_link)
            break
        except Exception:
            import time
            time.sleep(0.1)
            if i == 4:
                raise

Any ideas?

Shachar.

[ghost]

You're doing it wrong. You want pip install -e . and pip uninstall .. Don't run setup.py directly.

[shacharoo]

Thanks @xoviat, it works fine now with pip, but why shouldn't it work directly? And how pip handles that?

[ghost]

I don't know how pip does it but installing is a frontend responsibility. It won't be fixed in setuptools.

[ghost]

See PEP 427, 518, and 517 for more information.

[jaraco]

@FarmerSez : That's really surprising that an attempt to remove a file immediately after closing it would fail. That sounds like a bug in PyPy to me. Can you try replicating the failure and if you can with a simple Python script file the issue upstream with PyPy? I'd be happy to fix it here except that the behavior of Setuptools as I understand it is correct.

[shacharoo]

@jaraco sure, I'll look into it 😃