"./mach clean" fails in case of symlinks.

[kunalmohan]

if the build fails, Running ./mach clean gives the following error:

Removing virtualenv directory: /mnt/sda1/Documents/webpages/rust/servo/python/_virtualenv
Error running mach:

    ['clean']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

OSError: [Errno 39] Directory not empty: '/mnt/sda1/Documents/webpages/rust/servo/python/_virtualenv/lib/python2.7/site-packages/.libs_cffi_backend'

  File "/mnt/sda1/Documents/webpages/rust/servo/python/servo/build_commands.py", line 762, in clean
    shutil.rmtree(virtualenv_path)
  File "/usr/lib/python2.7/shutil.py", line 270, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib/python2.7/shutil.py", line 270, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib/python2.7/shutil.py", line 270, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib/python2.7/shutil.py", line 270, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib/python2.7/shutil.py", line 279, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/usr/lib/python2.7/shutil.py", line 277, in rmtree
    os.rmdir(path)

Environment: Linux (Ubuntu 19.04)

[SimonSapin]

I’m not sure this happens because compilation with Cargo failed. Removing the python/_virtualenv directory should get you "unstuck". If you see this happen again, could you provide steps to reproduce?

[kunalmohan]

removing the python/_virtualenv and then ./mach clean again recreates the directory and gives the same error.
Here's what happened from the start:

1. I ran ./mach build -d
2. There was an error during the build of one of the modules so I stopped the further build with keyboard interrupt.
3. Now on running ./mach clean gives the above error.

[SimonSapin]

This doesn’t reproduce on my machine.

./mach clean calls Python’s shutil.rmtree("python/_virtualenv"):

servo/python/servo/build_commands.py

Lines 759 to 762 in ae0b086

	virtualenv_path = path.join(self.get_top_dir(), 'python', '_virtualenv')
	if path.exists(virtualenv_path):
	print('Removing virtualenv directory: %s' % virtualenv_path)
	shutil.rmtree(virtualenv_path)

Either there’s a bug in Python, or something is (re) creating the file python/_virtualenv/lib/python2.7/site-packages/.libs_cffi_backend before shutil.rmtree calls os.rmdir to remove a directory that is then supposed to be empty, and either after it called os.listdir to find the files and sub-directories to remove or after it called os.remove to remove that file.

Do you have something running in another terminal that could be interfering?

[kunalmohan]

Nope. Only one terminal is running.

[kunalmohan]

@SimonSapin I guess I found the reason for this behaviour- Symlinks. My OS is in my SSD and the documents folder (in which this repository exists locally on my system) is a symlink to a folder in my HDD. It looks like shutil.rmtree does not work well with symlinks (https://bugs.python.org/issue1669). This is indeed bug in python.

[highfive]

cc @wafflespeanut