bdist_msi upload fails

[schmir]

BPO	13719
Nosy	@loewis, @tiran, @tarekziade, @merwok, @zooba, @dstufft
Files	fix-bdist_msi-upload.patch: make python setup.py bdist_msi upload work fix-bdist_msi-upload-with-test.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-02-03.18:26:31.188>
created_at = <Date 2012-01-06.13:18:38.179>
labels = ['type-bug', 'library', 'OS-windows']
title = 'bdist_msi upload fails'
updated_at = <Date 2021-02-03.18:26:31.187>
user = 'https://bugs.python.org/schmir'

bugs.python.org fields:

activity = <Date 2021-02-03.18:26:31.187>
actor = 'steve.dower'
assignee = 'none'
closed = True
closed_date = <Date 2021-02-03.18:26:31.188>
closer = 'steve.dower'
components = ['Distutils', 'Windows']
creation = <Date 2012-01-06.13:18:38.179>
creator = 'schmir'
dependencies = []
files = ['24153', '24483']
hgrepos = []
issue_num = 13719
keywords = ['patch']
message_count = 21.0
messages = ['150729', '150742', '152427', '152900', '153107', '154728', '154913', '154953', '154955', '154956', '154960', '154974', '155017', '155110', '155113', '155116', '155117', '178780', '178790', '182394', '386391']
nosy_count = 10.0
nosy_names = ['loewis', 'christian.heimes', 'schmir', 'tarek', 'eric.araujo', 'alexis', 'python-dev', 'steve.dower', 'dstufft', 'Kieran.Colford']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue13719'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[schmir]

Running setup.py bdist_msi upload fails with something like:

c:\Python27\python.exe setup.py bdist_msi upload
running bdist_msi
running build
running build_ext
installing to build\bdist.win-amd64\msi
running install_lib
creating build\bdist.win-amd64
creating build\bdist.win-amd64\msi
creating build\bdist.win-amd64\msi\Lib
creating build\bdist.win-amd64\msi\Lib\site-packages
copying build\lib.win-amd64-2.7\greenlet.pyd -> build\bdist.win-amd64\msi\Lib\site-packages
running install_headers
creating build\bdist.win-amd64\msi\Include
creating build\bdist.win-amd64\msi\Include\greenlet
copying greenlet.h -> build\bdist.win-amd64\msi\Include\greenlet
running install_egg_info
Writing build\bdist.win-amd64\msi\Lib\site-packages\greenlet-0.3.3-py2.7.egg-info
creating dist
removing 'build\bdist.win-amd64\msi' (and everything under it)
running upload
Linking c:\Users\ralf\home\greenlet\build\lib.win-amd64-2.7\greenlet.pyd to c:\Users\ralf\home\greenlet\greenlet.pyd
error: greenlet-0.3.3: No such file or directory

The attached patch fixes it.

[merwok]

Thanks! Given the fragility of distutils, we try to add a test for every fix we commit, but I don’t remember if we already have a basic HTTP server to test the upload command. If not, I think I’ll commit this after manual testing (my Windows VM crashes on boot, so it may take some time for me to get to this).

[schmir]

It's a bug in bdist_msi not in the upload command. You can check that distribution.dist_files point to valid files after running the command.

[schmir]

there's a workaround in http://pypi.python.org/pypi/xdistutils#the-bdist-msi-fixed-command

[merwok]

I don’t know if the bdist_msi command runs without user interaction. If so, then the test in the attached patch should pass on Windows. Can you please test it?

[merwok]

Ralf, could you test my patch?

[schmir]

sorry, I'm currently short on time and don't use windows at the moment.

bdist_msi runs without user interaction. just let the buildbots test it.

[python-dev]

New changeset 3d1362fa07c3 by Éric Araujo in branch '3.2':
Make distutils’ upload command work with bdist_msi products (bpo-13719).
http://hg.python.org/cpython/rev/3d1362fa07c3

[python-dev]

New changeset 7671932c98d3 by Éric Araujo in branch '2.7':
Make distutils’ upload command work with bdist_msi products (bpo-13719).
http://hg.python.org/cpython/rev/7671932c98d3

[python-dev]

New changeset 4cfcda9e80cb by Éric Araujo in branch '3.2':
Fix NameError from bpo-13719 fix
http://hg.python.org/cpython/rev/4cfcda9e80cb

New changeset 449c9fc2fc2d by Éric Araujo in branch 'default':
Make packaging’ upload command work with bdist_msi products (bpo-13719)
http://hg.python.org/cpython/rev/449c9fc2fc2d

[merwok]

Will watch buildbots tomorrow and close if they are OK. Will backport to distutils2 later.

[merwok]

Buildbots fail when calling listdir; will try to fix this or revert tomorrow. Help would be appreciated.

[merwok]

I’m working on this.

[python-dev]

New changeset 077b42a54803 by Éric Araujo in branch '2.7':
Backout buggy patch for bpo-13719
http://hg.python.org/cpython/rev/077b42a54803

[python-dev]

New changeset 7e629bacec87 by Éric Araujo in branch '3.2':
Backout buggy patch committed for bpo-13719
http://hg.python.org/cpython/rev/7e629bacec87

New changeset 17106d7d34b4 by Éric Araujo in branch 'default':
Remove buggy change for bpo-13719 in packaging
http://hg.python.org/cpython/rev/17106d7d34b4

[merwok]

I finally got a virtual machine up and running and was able to diagnose the problem. There are two things. First, the dist directory (where the msi file will be created) is created relative to the current working directory, which explains why listdir in the temporary project directory caused the test to fail. Second, after your patch bdist_wininst puts a full path in the dist.dist_files list, but it should use a relative path, like other bdist commands do. (This choice comes from the fact that setup scripts must be run from their parent directory.)

Fixing the first problem is just a matter of adding os.chdir(project_dir) in the test; the second problem is also easy, using os.path.join and self.dist_dir. I chose to backout the commits because I won’t be able to make a correct patch right now, I have to configure file sharing between my host and my VM. Let’s have the buildbots green for a while and then break them again (not :).

Debugging this made me realize that all tests should use os.path.join('dist', filename) instead of e.g. 'dist/blah-1.0.rpm' in test_bdist_rpm; I’ll do that in another commit.

[merwok]

I forgot to mention two other problems in my test:

• need to pass name and version keyword arguments to self.create_dist or to change the expected filename to UNKNOWN-UNKNOWN
• need to embed the result of distutils.util.get_platform() into the expected filename

[schmir]

AFAIK pypi.python.org disallows uploads of .msi files. That means I've lost interest in this issue.

Feel free to close it.

[loewis]

schmir: your information on msi support is probably outdated; pypi.python.org does support MSI files, see for example

http://pypi.python.org/pypi/Twisted/12.3.0

[tiran]

I have been bitten by the bug today, too.

[zooba]

Distutils is now deprecated (see PEP-632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools