New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use shutil.rmtree instead of os.removedirs #365

Merged
merged 1 commit into from Jun 16, 2016

Conversation

Projects
None yet
5 participants
@esc
Member

esc commented Jun 15, 2016

This fixes a bug in corner case in the setup.py shim/scaffolding generated by
PyBuilder. When using install_file and friends and installing the project via
git+ssh protocol the scaffolding setup.py crashes during cleanup, because
certain directories are non-empty.

In order to further demonstrate the issue, I have created a PyBuilder project
at:

https://github.com/esc/pybuilder-bug-expose

If you now try to install this project, you get:

% pip install "git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug"
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug
  Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git to /tmp/pip-build-BE_AGP/bug
    Complete output from command python setup.py egg_info:
    pyb 0.11.8
    PyBuilder version 0.11.8
    Build started at 2016-06-15 16:46:45
    ------------------------------------------------------------
    [INFO]  Building bug version 1.0.dev0
    [INFO]  Executing build in /tmp/pip-build-BE_AGP/bug
    [INFO]  Going to execute tasks: clean, install_build_dependencies, package
    [INFO]  Removing target directory /tmp/pip-build-BE_AGP/bug/target
    [INFO]  Installing build dependencies
    [INFO]  Installing dependency 'mockito'
    [INFO]  Building distribution in /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
    [INFO]  Copying scripts to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/scripts
    [INFO]  Copying resources matching 'foo/bar' from /tmp/pip-build-BE_AGP/bug to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
    [INFO]  Writing MANIFEST.in as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/MANIFEST.in
    [INFO]  Writing setup.py as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/setup.py
    ------------------------------------------------------------
    BUILD SUCCESSFUL
    ------------------------------------------------------------
    Build finished at 2016-06-15 16:46:45
    Build took 0 seconds (598 ms)
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-BE_AGP/bug/setup.py", line 66, in <module>
        os.removedirs(target_file_name)
      File "/home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/os.py", line 170, in removedirs
        rmdir(name)
    OSError: [Errno 39] Directory not empty: '/tmp/pip-build-BE_AGP/bug/foo'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-BE_AGP/bug/

This pull-request fixes this issue and ensures that the setup.py can
actually remove the directories.

use shutil.rmtree instead of os.removedirs
This fixes a bug in corner case in the `setup.py` shim/scaffolding generated by
PyBuilder. When using `install_file` and friends and installing the project via
git+ssh protocol the scaffolding `setup.py` crashes during cleanup, because
certain directories are non-empty.

In order to further demonstrate the issue, I have created a PyBuilder project
at:

https://github.com/esc/pybuilder-bug-expose

If you now try to install this project, you get:

```
% pip install "git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug"
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug
  Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git to /tmp/pip-build-BE_AGP/bug
    Complete output from command python setup.py egg_info:
    pyb 0.11.8
    PyBuilder version 0.11.8
    Build started at 2016-06-15 16:46:45
    ------------------------------------------------------------
    [INFO]  Building bug version 1.0.dev0
    [INFO]  Executing build in /tmp/pip-build-BE_AGP/bug
    [INFO]  Going to execute tasks: clean, install_build_dependencies, package
    [INFO]  Removing target directory /tmp/pip-build-BE_AGP/bug/target
    [INFO]  Installing build dependencies
    [INFO]  Installing dependency 'mockito'
    [INFO]  Building distribution in /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
    [INFO]  Copying scripts to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/scripts
    [INFO]  Copying resources matching 'foo/bar' from /tmp/pip-build-BE_AGP/bug to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
    [INFO]  Writing MANIFEST.in as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/MANIFEST.in
    [INFO]  Writing setup.py as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/setup.py
    ------------------------------------------------------------
    BUILD SUCCESSFUL
    ------------------------------------------------------------
    Build finished at 2016-06-15 16:46:45
    Build took 0 seconds (598 ms)
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-BE_AGP/bug/setup.py", line 66, in <module>
        os.removedirs(target_file_name)
      File "/home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/os.py", line 170, in removedirs
        rmdir(name)
    OSError: [Errno 39] Directory not empty: '/tmp/pip-build-BE_AGP/bug/foo'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-BE_AGP/bug/
```

This pull-request fixes this issue and ensures that the `setup.py` can
actually remove the directories.
@esc

This comment has been minimized.

Show comment
Hide comment
@esc

esc Jun 15, 2016

Member

I hit this nice little bug today...

Member

esc commented Jun 15, 2016

I hit this nice little bug today...

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jun 15, 2016

Coverage Status

Coverage remained the same at 72.53% when pulling 5c898e6 on esc:fix_generated_setup.py into 599e2d9 on pybuilder:master.

coveralls commented Jun 15, 2016

Coverage Status

Coverage remained the same at 72.53% when pulling 5c898e6 on esc:fix_generated_setup.py into 599e2d9 on pybuilder:master.

@esc

This comment has been minimized.

Show comment
Hide comment
@esc

esc Jun 15, 2016

Member

There is another branch, fixed which you can use to verify that the fix works and that foo/bar is indeed installed correctly:

% pip install -vvv "git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug" 
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug
  Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git (to fixed) to /tmp/pip-build-d46YGV/bug
  Running command git clone -q ssh://git@github.com/esc/pybuilder-bug-expose.git /tmp/pip-build-d46YGV/bug
  Running command git show-ref
  41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/heads/master
  41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/remotes/origin/HEAD
  211d509c9d2bc9b0832ad4382f90fe5b887f28a7 refs/remotes/origin/fixed
  41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/remotes/origin/master
  Running command git rev-parse HEAD
  41b87297dbfe6628bb29ba38ed8d10feca365f8c
  Running command git checkout -q 211d509c9d2bc9b0832ad4382f90fe5b887f28a7
  Running setup.py (path:/tmp/pip-build-d46YGV/bug/setup.py) egg_info for package bug
    Running command python setup.py egg_info
    pyb 0.11.8
    PyBuilder version 0.11.8
    Build started at 2016-06-15 17:11:20
    ------------------------------------------------------------
    [INFO]  Building bug version 1.0.dev0
    [INFO]  Executing build in /tmp/pip-build-d46YGV/bug
    [INFO]  Going to execute tasks: clean, install_build_dependencies, package
    [INFO]  Removing target directory /tmp/pip-build-d46YGV/bug/target
    [INFO]  Installing build dependencies
    [INFO]  Installing dependency 'mockito'
    [INFO]  Installing plugin dependency coverage
    [INFO]  Installing plugin dependency unittest-xml-reporting
    [INFO]  Building distribution in /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0
    [INFO]  Copying scripts to /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/scripts
    [INFO]  Copying resources matching 'foo/bar' from /tmp/pip-build-d46YGV/bug to /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0
    [INFO]  Writing MANIFEST.in as /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/MANIFEST.in
    [INFO]  Writing setup.py as /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/setup.py
    ------------------------------------------------------------
    BUILD SUCCESSFUL
    ------------------------------------------------------------
    Build finished at 2016-06-15 17:11:22
    Build took 2 seconds (2017 ms)
    running egg_info
    creating pip-egg-info/bug.egg-info
    writing pip-egg-info/bug.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/bug.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/bug.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
    reading manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-d46YGV/bug has version 1.0.dev0, which satisfies requirement bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug
Installing collected packages: bug
  Running setup.py install for bug ...     Running command /home/esc/gw/pybuilder-bug-expose/.venv/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-d46YGV/bug/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-8pkYoU-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/esc/gw/pybuilder-bug-expose/.venv/include/site/python2.7/bug
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    copying helloworld.py -> build/lib.linux-x86_64-2.7
    running build_scripts
    creating build/scripts-2.7
    copying and adjusting scripts/hello-pybuilder -> build/scripts-2.7
    changing mode of build/scripts-2.7/hello-pybuilder from 600 to 755
    running install_lib
    copying build/lib.linux-x86_64-2.7/helloworld.py -> /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages
    byte-compiling /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages/helloworld.py to helloworld.pyc
    running install_data
    creating /home/esc/gw/pybuilder-bug-expose/.venv/foo
    creating /home/esc/gw/pybuilder-bug-expose/.venv/foo/bar
    copying foo/bar -> /home/esc/gw/pybuilder-bug-expose/.venv/foo/bar
    running install_egg_info
    running egg_info
    creating bug.egg-info
    writing bug.egg-info/PKG-INFO
    writing top-level names to bug.egg-info/top_level.txt
    writing dependency_links to bug.egg-info/dependency_links.txt
    writing manifest file 'bug.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'bug.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'bug.egg-info/SOURCES.txt'
    Copying bug.egg-info to /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages/bug-1.0.dev0-py2.7.egg-info
    running install_scripts
    copying build/scripts-2.7/hello-pybuilder -> /home/esc/gw/pybuilder-bug-expose/.venv/bin
    changing mode of /home/esc/gw/pybuilder-bug-expose/.venv/bin/hello-pybuilder to 755
    writing list of installed files to '/tmp/pip-8pkYoU-record/install-record.txt'
done
  Removing source in /tmp/pip-build-d46YGV/bug
Successfully installed bug-1.0.dev0
Cleaning up..
Member

esc commented Jun 15, 2016

There is another branch, fixed which you can use to verify that the fix works and that foo/bar is indeed installed correctly:

% pip install -vvv "git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug" 
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug
  Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git (to fixed) to /tmp/pip-build-d46YGV/bug
  Running command git clone -q ssh://git@github.com/esc/pybuilder-bug-expose.git /tmp/pip-build-d46YGV/bug
  Running command git show-ref
  41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/heads/master
  41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/remotes/origin/HEAD
  211d509c9d2bc9b0832ad4382f90fe5b887f28a7 refs/remotes/origin/fixed
  41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/remotes/origin/master
  Running command git rev-parse HEAD
  41b87297dbfe6628bb29ba38ed8d10feca365f8c
  Running command git checkout -q 211d509c9d2bc9b0832ad4382f90fe5b887f28a7
  Running setup.py (path:/tmp/pip-build-d46YGV/bug/setup.py) egg_info for package bug
    Running command python setup.py egg_info
    pyb 0.11.8
    PyBuilder version 0.11.8
    Build started at 2016-06-15 17:11:20
    ------------------------------------------------------------
    [INFO]  Building bug version 1.0.dev0
    [INFO]  Executing build in /tmp/pip-build-d46YGV/bug
    [INFO]  Going to execute tasks: clean, install_build_dependencies, package
    [INFO]  Removing target directory /tmp/pip-build-d46YGV/bug/target
    [INFO]  Installing build dependencies
    [INFO]  Installing dependency 'mockito'
    [INFO]  Installing plugin dependency coverage
    [INFO]  Installing plugin dependency unittest-xml-reporting
    [INFO]  Building distribution in /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0
    [INFO]  Copying scripts to /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/scripts
    [INFO]  Copying resources matching 'foo/bar' from /tmp/pip-build-d46YGV/bug to /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0
    [INFO]  Writing MANIFEST.in as /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/MANIFEST.in
    [INFO]  Writing setup.py as /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/setup.py
    ------------------------------------------------------------
    BUILD SUCCESSFUL
    ------------------------------------------------------------
    Build finished at 2016-06-15 17:11:22
    Build took 2 seconds (2017 ms)
    running egg_info
    creating pip-egg-info/bug.egg-info
    writing pip-egg-info/bug.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/bug.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/bug.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
    reading manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
  Source in /tmp/pip-build-d46YGV/bug has version 1.0.dev0, which satisfies requirement bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug
Installing collected packages: bug
  Running setup.py install for bug ...     Running command /home/esc/gw/pybuilder-bug-expose/.venv/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-d46YGV/bug/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-8pkYoU-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/esc/gw/pybuilder-bug-expose/.venv/include/site/python2.7/bug
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    copying helloworld.py -> build/lib.linux-x86_64-2.7
    running build_scripts
    creating build/scripts-2.7
    copying and adjusting scripts/hello-pybuilder -> build/scripts-2.7
    changing mode of build/scripts-2.7/hello-pybuilder from 600 to 755
    running install_lib
    copying build/lib.linux-x86_64-2.7/helloworld.py -> /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages
    byte-compiling /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages/helloworld.py to helloworld.pyc
    running install_data
    creating /home/esc/gw/pybuilder-bug-expose/.venv/foo
    creating /home/esc/gw/pybuilder-bug-expose/.venv/foo/bar
    copying foo/bar -> /home/esc/gw/pybuilder-bug-expose/.venv/foo/bar
    running install_egg_info
    running egg_info
    creating bug.egg-info
    writing bug.egg-info/PKG-INFO
    writing top-level names to bug.egg-info/top_level.txt
    writing dependency_links to bug.egg-info/dependency_links.txt
    writing manifest file 'bug.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'bug.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'bug.egg-info/SOURCES.txt'
    Copying bug.egg-info to /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages/bug-1.0.dev0-py2.7.egg-info
    running install_scripts
    copying build/scripts-2.7/hello-pybuilder -> /home/esc/gw/pybuilder-bug-expose/.venv/bin
    changing mode of /home/esc/gw/pybuilder-bug-expose/.venv/bin/hello-pybuilder to 755
    writing list of installed files to '/tmp/pip-8pkYoU-record/install-record.txt'
done
  Removing source in /tmp/pip-build-d46YGV/bug
Successfully installed bug-1.0.dev0
Cleaning up..
@tendryll

This comment has been minimized.

Show comment
Hide comment
@tendryll

tendryll Jun 15, 2016

Please remove from mailing lists.

On Wed, Jun 15, 2016 at 9:56 AM Valentin Haenel notifications@github.com
wrote:

This fixes a bug in corner case in the setup.py shim/scaffolding
generated by
PyBuilder. When using install_file and friends and installing the project
via
git+ssh protocol the scaffolding setup.py crashes during cleanup, because
certain directories are non-empty.

In order to further demonstrate the issue, I have created a PyBuilder
project
at:

https://github.com/esc/pybuilder-bug-expose

If you now try to install this project, you get:

% pip install "git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug"
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug
Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git to /tmp/pip-build-BE_AGP/bug
Complete output from command python setup.py egg_info:
pyb 0.11.8
PyBuilder version 0.11.8
Build started at 2016-06-15 16:46:45
------------------------------------------------------------
[INFO] Building bug version 1.0.dev0
[INFO] Executing build in /tmp/pip-build-BE_AGP/bug
[INFO] Going to execute tasks: clean, install_build_dependencies, package
[INFO] Removing target directory /tmp/pip-build-BE_AGP/bug/target
[INFO] Installing build dependencies
[INFO] Installing dependency 'mockito'
[INFO] Building distribution in /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
[INFO] Copying scripts to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/scripts
[INFO] Copying resources matching 'foo/bar' from /tmp/pip-build-BE_AGP/bug to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
[INFO] Writing MANIFEST.in as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/MANIFEST.in
[INFO] Writing setup.py as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/setup.py
------------------------------------------------------------
BUILD SUCCESSFUL
------------------------------------------------------------
Build finished at 2016-06-15 16:46:45
Build took 0 seconds (598 ms)
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-BE_AGP/bug/setup.py", line 66, in
os.removedirs(target_file_name)
File "/home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/os.py", line 170, in removedirs
rmdir(name)
OSError: [Errno 39] Directory not empty: '/tmp/pip-build-BE_AGP/bug/foo'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-BE_AGP/bug/

This pull-request fixes this issue and ensures that the setup.py can

actually remove the directories.

You can view, comment on, or merge this pull request online at:

#365
Commit Summary

  • use shutil.rmtree instead of os.removedirs

File Changes

Patch Links:


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#365, or mute the thread
https://github.com/notifications/unsubscribe/ACbX7aW4b5ucyL68tDvtoO-Roe689GfVks5qMBKkgaJpZM4I2cpV
.

tendryll commented Jun 15, 2016

Please remove from mailing lists.

On Wed, Jun 15, 2016 at 9:56 AM Valentin Haenel notifications@github.com
wrote:

This fixes a bug in corner case in the setup.py shim/scaffolding
generated by
PyBuilder. When using install_file and friends and installing the project
via
git+ssh protocol the scaffolding setup.py crashes during cleanup, because
certain directories are non-empty.

In order to further demonstrate the issue, I have created a PyBuilder
project
at:

https://github.com/esc/pybuilder-bug-expose

If you now try to install this project, you get:

% pip install "git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug"
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug
Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git to /tmp/pip-build-BE_AGP/bug
Complete output from command python setup.py egg_info:
pyb 0.11.8
PyBuilder version 0.11.8
Build started at 2016-06-15 16:46:45
------------------------------------------------------------
[INFO] Building bug version 1.0.dev0
[INFO] Executing build in /tmp/pip-build-BE_AGP/bug
[INFO] Going to execute tasks: clean, install_build_dependencies, package
[INFO] Removing target directory /tmp/pip-build-BE_AGP/bug/target
[INFO] Installing build dependencies
[INFO] Installing dependency 'mockito'
[INFO] Building distribution in /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
[INFO] Copying scripts to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/scripts
[INFO] Copying resources matching 'foo/bar' from /tmp/pip-build-BE_AGP/bug to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
[INFO] Writing MANIFEST.in as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/MANIFEST.in
[INFO] Writing setup.py as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/setup.py
------------------------------------------------------------
BUILD SUCCESSFUL
------------------------------------------------------------
Build finished at 2016-06-15 16:46:45
Build took 0 seconds (598 ms)
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-BE_AGP/bug/setup.py", line 66, in
os.removedirs(target_file_name)
File "/home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/os.py", line 170, in removedirs
rmdir(name)
OSError: [Errno 39] Directory not empty: '/tmp/pip-build-BE_AGP/bug/foo'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-BE_AGP/bug/

This pull-request fixes this issue and ensures that the setup.py can

actually remove the directories.

You can view, comment on, or merge this pull request online at:

#365
Commit Summary

  • use shutil.rmtree instead of os.removedirs

File Changes

Patch Links:


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#365, or mute the thread
https://github.com/notifications/unsubscribe/ACbX7aW4b5ucyL68tDvtoO-Roe689GfVks5qMBKkgaJpZM4I2cpV
.

@mriehl

This comment has been minimized.

Show comment
Hide comment
@mriehl

mriehl Jun 15, 2016

Member

Whew, nice catch. I'll have a look as soon as I am able (depending on how interesting the scaladays tracks are ;-))

On 15 Jun 2016, at 17:15, Valentin Haenel notifications@github.com wrote:

There is another branch, fixed which you can use to verify that the fix works and that foo/bar is indeed installed correctly:

% pip install -vvv "git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug"
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug
Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git (to fixed) to /tmp/pip-build-d46YGV/bug
Running command git clone -q ssh://git@github.com/esc/pybuilder-bug-expose.git /tmp/pip-build-d46YGV/bug
Running command git show-ref
41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/heads/master
41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/remotes/origin/HEAD
211d509c9d2bc9b0832ad4382f90fe5b887f28a7 refs/remotes/origin/fixed
41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/remotes/origin/master
Running command git rev-parse HEAD
41b87297dbfe6628bb29ba38ed8d10feca365f8c
Running command git checkout -q 211d509c9d2bc9b0832ad4382f90fe5b887f28a7
Running setup.py (path:/tmp/pip-build-d46YGV/bug/setup.py) egg_info for package bug
Running command python setup.py egg_info
pyb 0.11.8
PyBuilder version 0.11.8
Build started at 2016-06-15 17:11:20
------------------------------------------------------------
[INFO] Building bug version 1.0.dev0
[INFO] Executing build in /tmp/pip-build-d46YGV/bug
[INFO] Going to execute tasks: clean, install_build_dependencies, package
[INFO] Removing target directory /tmp/pip-build-d46YGV/bug/target
[INFO] Installing build dependencies
[INFO] Installing dependency 'mockito'
[INFO] Installing plugin dependency coverage
[INFO] Installing plugin dependency unittest-xml-reporting
[INFO] Building distribution in /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0
[INFO] Copying scripts to /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/scripts
[INFO] Copying resources matching 'foo/bar' from /tmp/pip-build-d46YGV/bug to /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0
[INFO] Writing MANIFEST.in as /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/MANIFEST.in
[INFO] Writing setup.py as /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/setup.py
------------------------------------------------------------
BUILD SUCCESSFUL
------------------------------------------------------------
Build finished at 2016-06-15 17:11:22
Build took 2 seconds (2017 ms)
running egg_info
creating pip-egg-info/bug.egg-info
writing pip-egg-info/bug.egg-info/PKG-INFO
writing top-level names to pip-egg-info/bug.egg-info/top_level.txt
writing dependency_links to pip-egg-info/bug.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
reading manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
Source in /tmp/pip-build-d46YGV/bug has version 1.0.dev0, which satisfies requirement bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug
Installing collected packages: bug
Running setup.py install for bug ... Running command /home/esc/gw/pybuilder-bug-expose/.venv/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-build-d46YGV/bug/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-8pkYoU-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/esc/gw/pybuilder-bug-expose/.venv/include/site/python2.7/bug
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
copying helloworld.py -> build/lib.linux-x86_64-2.7
running build_scripts
creating build/scripts-2.7
copying and adjusting scripts/hello-pybuilder -> build/scripts-2.7
changing mode of build/scripts-2.7/hello-pybuilder from 600 to 755
running install_lib
copying build/lib.linux-x86_64-2.7/helloworld.py -> /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages
byte-compiling /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages/helloworld.py to helloworld.pyc
running install_data
creating /home/esc/gw/pybuilder-bug-expose/.venv/foo
creating /home/esc/gw/pybuilder-bug-expose/.venv/foo/bar
copying foo/bar -> /home/esc/gw/pybuilder-bug-expose/.venv/foo/bar
running install_egg_info
running egg_info
creating bug.egg-info
writing bug.egg-info/PKG-INFO
writing top-level names to bug.egg-info/top_level.txt
writing dependency_links to bug.egg-info/dependency_links.txt
writing manifest file 'bug.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found

reading manifest file 'bug.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'bug.egg-info/SOURCES.txt'
Copying bug.egg-info to /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages/bug-1.0.dev0-py2.7.egg-info
running install_scripts
copying build/scripts-2.7/hello-pybuilder -> /home/esc/gw/pybuilder-bug-expose/.venv/bin
changing mode of /home/esc/gw/pybuilder-bug-expose/.venv/bin/hello-pybuilder to 755
writing list of installed files to '/tmp/pip-8pkYoU-record/install-record.txt'

done
Removing source in /tmp/pip-build-d46YGV/bug
Successfully installed bug-1.0.dev0
Cleaning up..

You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Member

mriehl commented Jun 15, 2016

Whew, nice catch. I'll have a look as soon as I am able (depending on how interesting the scaladays tracks are ;-))

On 15 Jun 2016, at 17:15, Valentin Haenel notifications@github.com wrote:

There is another branch, fixed which you can use to verify that the fix works and that foo/bar is indeed installed correctly:

% pip install -vvv "git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug"
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug
Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git (to fixed) to /tmp/pip-build-d46YGV/bug
Running command git clone -q ssh://git@github.com/esc/pybuilder-bug-expose.git /tmp/pip-build-d46YGV/bug
Running command git show-ref
41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/heads/master
41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/remotes/origin/HEAD
211d509c9d2bc9b0832ad4382f90fe5b887f28a7 refs/remotes/origin/fixed
41b87297dbfe6628bb29ba38ed8d10feca365f8c refs/remotes/origin/master
Running command git rev-parse HEAD
41b87297dbfe6628bb29ba38ed8d10feca365f8c
Running command git checkout -q 211d509c9d2bc9b0832ad4382f90fe5b887f28a7
Running setup.py (path:/tmp/pip-build-d46YGV/bug/setup.py) egg_info for package bug
Running command python setup.py egg_info
pyb 0.11.8
PyBuilder version 0.11.8
Build started at 2016-06-15 17:11:20
------------------------------------------------------------
[INFO] Building bug version 1.0.dev0
[INFO] Executing build in /tmp/pip-build-d46YGV/bug
[INFO] Going to execute tasks: clean, install_build_dependencies, package
[INFO] Removing target directory /tmp/pip-build-d46YGV/bug/target
[INFO] Installing build dependencies
[INFO] Installing dependency 'mockito'
[INFO] Installing plugin dependency coverage
[INFO] Installing plugin dependency unittest-xml-reporting
[INFO] Building distribution in /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0
[INFO] Copying scripts to /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/scripts
[INFO] Copying resources matching 'foo/bar' from /tmp/pip-build-d46YGV/bug to /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0
[INFO] Writing MANIFEST.in as /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/MANIFEST.in
[INFO] Writing setup.py as /tmp/pip-build-d46YGV/bug/target/dist/bug-1.0.dev0/setup.py
------------------------------------------------------------
BUILD SUCCESSFUL
------------------------------------------------------------
Build finished at 2016-06-15 17:11:22
Build took 2 seconds (2017 ms)
running egg_info
creating pip-egg-info/bug.egg-info
writing pip-egg-info/bug.egg-info/PKG-INFO
writing top-level names to pip-egg-info/bug.egg-info/top_level.txt
writing dependency_links to pip-egg-info/bug.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
reading manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pip-egg-info/bug.egg-info/SOURCES.txt'
Source in /tmp/pip-build-d46YGV/bug has version 1.0.dev0, which satisfies requirement bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git@fixed#egg=bug
Installing collected packages: bug
Running setup.py install for bug ... Running command /home/esc/gw/pybuilder-bug-expose/.venv/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-build-d46YGV/bug/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-8pkYoU-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/esc/gw/pybuilder-bug-expose/.venv/include/site/python2.7/bug
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
copying helloworld.py -> build/lib.linux-x86_64-2.7
running build_scripts
creating build/scripts-2.7
copying and adjusting scripts/hello-pybuilder -> build/scripts-2.7
changing mode of build/scripts-2.7/hello-pybuilder from 600 to 755
running install_lib
copying build/lib.linux-x86_64-2.7/helloworld.py -> /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages
byte-compiling /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages/helloworld.py to helloworld.pyc
running install_data
creating /home/esc/gw/pybuilder-bug-expose/.venv/foo
creating /home/esc/gw/pybuilder-bug-expose/.venv/foo/bar
copying foo/bar -> /home/esc/gw/pybuilder-bug-expose/.venv/foo/bar
running install_egg_info
running egg_info
creating bug.egg-info
writing bug.egg-info/PKG-INFO
writing top-level names to bug.egg-info/top_level.txt
writing dependency_links to bug.egg-info/dependency_links.txt
writing manifest file 'bug.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found

reading manifest file 'bug.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'bug.egg-info/SOURCES.txt'
Copying bug.egg-info to /home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/site-packages/bug-1.0.dev0-py2.7.egg-info
running install_scripts
copying build/scripts-2.7/hello-pybuilder -> /home/esc/gw/pybuilder-bug-expose/.venv/bin
changing mode of /home/esc/gw/pybuilder-bug-expose/.venv/bin/hello-pybuilder to 755
writing list of installed files to '/tmp/pip-8pkYoU-record/install-record.txt'

done
Removing source in /tmp/pip-build-d46YGV/bug
Successfully installed bug-1.0.dev0
Cleaning up..

You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@mriehl

This comment has been minimized.

Show comment
Hide comment
@mriehl

mriehl Jun 15, 2016

Member

@tendryll this is not a mailing list. You must be watching the repository, please log in to GitHub and unwatch it.

On 15 Jun 2016, at 17:38, tendryll notifications@github.com wrote:

Please remove from mailing lists.

On Wed, Jun 15, 2016 at 9:56 AM Valentin Haenel notifications@github.com
wrote:

This fixes a bug in corner case in the setup.py shim/scaffolding
generated by
PyBuilder. When using install_file and friends and installing the project
via
git+ssh protocol the scaffolding setup.py crashes during cleanup, because
certain directories are non-empty.

In order to further demonstrate the issue, I have created a PyBuilder
project
at:

https://github.com/esc/pybuilder-bug-expose

If you now try to install this project, you get:

% pip install "git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug"
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug
Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git to /tmp/pip-build-BE_AGP/bug
Complete output from command python setup.py egg_info:
pyb 0.11.8
PyBuilder version 0.11.8

Build started at 2016-06-15 16:46:45

[INFO] Building bug version 1.0.dev0
[INFO] Executing build in /tmp/pip-build-BE_AGP/bug
[INFO] Going to execute tasks: clean, install_build_dependencies, package
[INFO] Removing target directory /tmp/pip-build-BE_AGP/bug/target
[INFO] Installing build dependencies
[INFO] Installing dependency 'mockito'
[INFO] Building distribution in /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
[INFO] Copying scripts to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/scripts
[INFO] Copying resources matching 'foo/bar' from /tmp/pip-build-BE_AGP/bug to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
[INFO] Writing MANIFEST.in as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/MANIFEST.in

[INFO] Writing setup.py as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/setup.py

BUILD SUCCESSFUL

Build finished at 2016-06-15 16:46:45
Build took 0 seconds (598 ms)
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-BE_AGP/bug/setup.py", line 66, in
os.removedirs(target_file_name)
File "/home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/os.py", line 170, in removedirs
rmdir(name)
OSError: [Errno 39] Directory not empty: '/tmp/pip-build-BE_AGP/bug/foo'


Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-BE_AGP/bug/

This pull-request fixes this issue and ensures that the setup.py can

actually remove the directories.

You can view, comment on, or merge this pull request online at:

#365
Commit Summary

  • use shutil.rmtree instead of os.removedirs

File Changes

Patch Links:


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#365, or mute the thread
https://github.com/notifications/unsubscribe/ACbX7aW4b5ucyL68tDvtoO-Roe689GfVks5qMBKkgaJpZM4I2cpV
.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Member

mriehl commented Jun 15, 2016

@tendryll this is not a mailing list. You must be watching the repository, please log in to GitHub and unwatch it.

On 15 Jun 2016, at 17:38, tendryll notifications@github.com wrote:

Please remove from mailing lists.

On Wed, Jun 15, 2016 at 9:56 AM Valentin Haenel notifications@github.com
wrote:

This fixes a bug in corner case in the setup.py shim/scaffolding
generated by
PyBuilder. When using install_file and friends and installing the project
via
git+ssh protocol the scaffolding setup.py crashes during cleanup, because
certain directories are non-empty.

In order to further demonstrate the issue, I have created a PyBuilder
project
at:

https://github.com/esc/pybuilder-bug-expose

If you now try to install this project, you get:

% pip install "git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug"
Collecting bug from git+ssh://git@github.com/esc/pybuilder-bug-expose.git#egg=bug
Cloning ssh://git@github.com/esc/pybuilder-bug-expose.git to /tmp/pip-build-BE_AGP/bug
Complete output from command python setup.py egg_info:
pyb 0.11.8
PyBuilder version 0.11.8

Build started at 2016-06-15 16:46:45

[INFO] Building bug version 1.0.dev0
[INFO] Executing build in /tmp/pip-build-BE_AGP/bug
[INFO] Going to execute tasks: clean, install_build_dependencies, package
[INFO] Removing target directory /tmp/pip-build-BE_AGP/bug/target
[INFO] Installing build dependencies
[INFO] Installing dependency 'mockito'
[INFO] Building distribution in /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
[INFO] Copying scripts to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/scripts
[INFO] Copying resources matching 'foo/bar' from /tmp/pip-build-BE_AGP/bug to /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0
[INFO] Writing MANIFEST.in as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/MANIFEST.in

[INFO] Writing setup.py as /tmp/pip-build-BE_AGP/bug/target/dist/bug-1.0.dev0/setup.py

BUILD SUCCESSFUL

Build finished at 2016-06-15 16:46:45
Build took 0 seconds (598 ms)
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-BE_AGP/bug/setup.py", line 66, in
os.removedirs(target_file_name)
File "/home/esc/gw/pybuilder-bug-expose/.venv/lib/python2.7/os.py", line 170, in removedirs
rmdir(name)
OSError: [Errno 39] Directory not empty: '/tmp/pip-build-BE_AGP/bug/foo'


Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-BE_AGP/bug/

This pull-request fixes this issue and ensures that the setup.py can

actually remove the directories.

You can view, comment on, or merge this pull request online at:

#365
Commit Summary

  • use shutil.rmtree instead of os.removedirs

File Changes

Patch Links:


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#365, or mute the thread
https://github.com/notifications/unsubscribe/ACbX7aW4b5ucyL68tDvtoO-Roe689GfVks5qMBKkgaJpZM4I2cpV
.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@esc

This comment has been minimized.

Show comment
Hide comment
@esc

esc Jun 15, 2016

Member

I'm unsure if the fix is correct, because I don't understand the logic in the scaffolding setup.py.

Member

esc commented Jun 15, 2016

I'm unsure if the fix is correct, because I don't understand the logic in the scaffolding setup.py.

@arcivanov

This comment has been minimized.

Show comment
Hide comment
@arcivanov

arcivanov Jun 16, 2016

Contributor

I'll take a look tomorrow. Thanks!

Contributor

arcivanov commented Jun 16, 2016

I'll take a look tomorrow. Thanks!

@arcivanov

This comment has been minimized.

Show comment
Hide comment
@arcivanov

arcivanov Jun 16, 2016

Contributor

@esc the logic actually deletes the possible leftover built dists, whether they are directories or simple files. The change is fine.

Contributor

arcivanov commented Jun 16, 2016

@esc the logic actually deletes the possible leftover built dists, whether they are directories or simple files. The change is fine.

@arcivanov arcivanov merged commit a6bcbd6 into pybuilder:master Jun 16, 2016

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage remained the same at 72.53%
Details

@arcivanov arcivanov removed the in progress label Jun 16, 2016

@esc esc deleted the esc:fix_generated_setup.py branch Jun 17, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment