py_compile cannot create files in current directory

[sjdv1982]

BPO	12618
Nosy	@birkenfeld, @ncoghlan, @pitrou, @merwok, @meadori, @ericsnowcurrently
Files	py_compile.py: py_compile.py, fixed in line 133 py_compile.diff: diff with 3.2 release issue12618-0.patch: Patch v0 against tip (3.3.0a0) issue12618-1.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/meadori'
closed_at = <Date 2011-11-28.15:41:28.779>
created_at = <Date 2011-07-23.07:42:50.631>
labels = ['easy', 'type-bug', 'library']
title = 'py_compile cannot create files in current directory'
updated_at = <Date 2011-11-28.15:41:28.778>
user = 'https://bugs.python.org/sjdv1982'

bugs.python.org fields:

activity = <Date 2011-11-28.15:41:28.778>
actor = 'meador.inge'
assignee = 'meador.inge'
closed = True
closed_date = <Date 2011-11-28.15:41:28.779>
closer = 'meador.inge'
components = ['Library (Lib)']
creation = <Date 2011-07-23.07:42:50.631>
creator = 'sjdv1982'
dependencies = []
files = ['22722', '22724', '23523', '23789']
hgrepos = []
issue_num = 12618
keywords = ['patch', 'easy']
message_count = 25.0
messages = ['140940', '140941', '140945', '140946', '140953', '140959', '140964', '140990', '141064', '146098', '146309', '146372', '146410', '148049', '148189', '148329', '148380', '148400', '148411', '148418', '148419', '148434', '148479', '148491', '148492']
nosy_count = 9.0
nosy_names = ['georg.brandl', 'ncoghlan', 'pitrou', 'eric.araujo', 'Arfrever', 'meador.inge', 'python-dev', 'eric.snow', 'sjdv1982']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12618'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

[sjdv1982]

When you specify cfile to be in the current directory, an error occurs (line 133).
I have fixed the file, see attached

[Arfrever]

Please create a patch in unified format.

[sjdv1982]

The attached file just works.
You can diff with trunk, or wherever python devs store the latest version.

[birkenfeld]

It might work right now, but in case the file changes before your change can be processed, we will lose the previous changes if we just copy in your new file.

IOW, you're making our work much harder and your change is less likely to be applied.

[sjdv1982]

Makes no sense to me: since I don't have the trunk version, I can only diff -c against 3.2 release. Doing a diff against trunk is 1 sec of work for you.

But I am just being a helpful user; so if a diff is what you want, here it is. No trouble for me, since I am on Linux.

Not a very nice policy to any helpful Windows users, though: they won't have diff installed and they would waste an hour or so on this.

[Arfrever]

It's a context patch, not a unified patch, and it is reversed :) .

[birkenfeld]

Well, we can work with this patch. Thanks.

Yes, we can diff against 3.2, but for that you would have at least needed to specify

• that you worked on a released version
• and that that version is 3.2

Otherwise, we cannot know what to diff against.

Note that even Windows users usually use Python from Mercurial for developing, so that they can use "hg diff" to create a patch easily.

[sjdv1982]

Good to hear that the patch is helpful.

Again, I am just trying to be a helpful user, making a (very very little) contribution to make Python better. I am not a Python dev at all: Python is already awesome enough for me, no desire to change it :-)
I just want to say that a "we only accept patches" policy is not being very nice to similar helpful users on Windows, who don't have diff or Mercurial but who did manage to fix a file themselves.

I did indicate "Python 3.2" in the bug report, maybe something went wrong.

[ncoghlan]

It's rare to receive fixes from Windows users that don't have a dev environment set up (i.e. they just edited their stdlib code in place, or copied it and created a modified version). Thanks for persisting in the face of invalid assumptions on our part.

[merwok]

Sjoerd, can you paste the code that produces the bug? It would help create a test.

[sjdv1982]

Hi Éric,

There you go, adapted from http://effbot.org/librarybook/py-compile.htm :

############
# File: py-compile-example-1.py

import py_compile

# explicitly compile this module
py_compile.compile("py-compile-example-1.py","py-compile-example-1.pyc")

############

Also, I tested and this bug is present neither on 3.1 nor on 2.x

cheers
Sjoerd

[merwok]

I can reproduce in 3.2 and 3.3. I’ll commit a test and patch when I get the time, or another dev can take this over.

[meadori]

I think it might be easier to just always use the absolute path rather than looking at the directory length. Maybe something like the attached. I added unit tests as well.

[merwok]

Thanks for the patch Meador, I hadn’t realized we had no tests for py_compile (it is however used in test_import and test_compileall). I think it would be nice to commit the tests first (except for the one that’s the object of this bug report) in order to have a baseline, and then see about fixing this bug. I’ll do that in a few days if nobody objects.

I’m not sure there would be no negative side-effects to using os.path.abspath; we don’t know what people do with symlinks and relative paths out there, so I’d prefer adding a safe special case* rather than always calling abspath. What do you think?

• Instead of using len, something like this would be clear IMO:

      if parent:  # empty string means current directory, skip creating the dir
          os.makedirs(parent)

[merwok]

Meador, maybe you would like to commit the tests (except for the one that’s the object of this report) yourself? I don’t mind doing it, but as you have push rights now maybe you prefer to have your name directly associated with your work.

[meadori]

Éric, sure, I will commit the tests sometime today. Then I will respond to the 'os.path.abspath' question as well.

[python-dev]

New changeset bcc7bf3963cc by Meador Inge in branch '2.7':
Issue bpo-12618: create unit tests for the py_compile module
http://hg.python.org/cpython/rev/bcc7bf3963cc

New changeset 2be3a2e63683 by Meador Inge in branch '3.2':
Issue bpo-12618: create unit tests for the py_compile module
http://hg.python.org/cpython/rev/2be3a2e63683

New changeset f8f58db0715e by Meador Inge in branch 'default':
Issue bpo-12618: create unit tests for the py_compile module
http://hg.python.org/cpython/rev/f8f58db0715e

[pitrou]

The tests break on the Windows buildbots:

======================================================================
ERROR: test_relative_path (test.test_py_compile.PyCompileTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_py_compile.py", line 33, in test_relative_path
    py_compile.compile(os.path.relpath(self.source_path),
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\ntpath.py", line 622, in relpath
    raise ValueError(error)
ValueError: path is on mount 'c:', start on mount 'D:'

[meadori]

On Sat, Nov 26, 2011 at 7:17 AM, Antoine Pitrou <report@bugs.python.org> wrote:

The tests break on the Windows buildbots:

I am investigating now.

[python-dev]

New changeset b23453530d5f by Meador Inge in branch '2.7':
Issue bpo-12618: fix py_compile unit tests to handle different drives on Windows
http://hg.python.org/cpython/rev/b23453530d5f

New changeset 7097d52cacee by Meador Inge in branch '3.2':
Issue bpo-12618: fix py_compile unit tests to handle different drives on Windows
http://hg.python.org/cpython/rev/7097d52cacee

New changeset 5243752e19aa by Meador Inge in branch 'default':
Issue bpo-12618: fix py_compile unit tests to handle different drives on Windows
http://hg.python.org/cpython/rev/5243752e19aa

[meadori]

The tests are fixed now. A relative path was being computed, but on Windows the current working directory drive and the drive of the relative path we were computing was different (and so this test bug would *not* be seen if running on a Windows box with a single "C: drive" setup).

/me sighs at the concept of Windows drives. Thanks for the heads up on the test break Antoine.

[meadori]

we don’t know what people do with symlinks and relative paths out
there, so I’d prefer adding a safe special case* rather than always
calling abspath. What do you think?

Éric, I agree. I didn't know about the strange symlink + relative path
behavior with 'os.path.normpath', but [1] cleared me up. With that in
mind the special casing is OK. I have attached an updated patch with
a unit test.

Also, this is not an issue for Python 2.7. The 2.7 implementation
assumes any directories mentioned in the path already exist. So, I
removed 2.7 from the affected versions.

[1] http://mail.python.org/pipermail/python-dev/2005-December/058452.html

[merwok]

I have attached an updated patch with a unit test.
LGTM.

Also, this is not an issue for Python 2.7. The 2.7 implementation assumes any
directories mentioned in the path already exist.
Cool. The test can still be committed in that branch too.

[python-dev]

New changeset 661fb211f220 by Meador Inge in branch '3.2':
Issue bpo-12618: py_compile cannot create files in current directory
http://hg.python.org/cpython/rev/661fb211f220

New changeset e3647275f468 by Meador Inge in branch 'default':
Issue bpo-12618: py_compile cannot create files in current directory
http://hg.python.org/cpython/rev/e3647275f468

[meadori]

Cool. The test can still be committed in that branch too.

The regression test for this issue was already committed for 2.7 in bcc7bf3963cc as a part of creating the unit test baseline. I just committed the bug fix to 3.2 and default.

Thanks for the fix Sjoerd.