doctest fails to run file based tests with 8bit paths

[bear]

BPO	1274
Nosy	@tim-one, @terryjreedy, @vstinner
Files	ensuring-doctest-file-suite-works-with-unicode-paths.patch: assert unicode paths are properly handled

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-04-02.03:19:31.165>
created_at = <Date 2007-10-12.18:29:49.945>
labels = ['easy', 'type-bug', 'tests']
title = 'doctest fails to run file based tests with 8bit paths'
updated_at = <Date 2013-04-02.03:19:31.164>
user = 'https://bugs.python.org/bear'

bugs.python.org fields:

activity = <Date 2013-04-02.03:19:31.164>
actor = 'terry.reedy'
assignee = 'none'
closed = True
closed_date = <Date 2013-04-02.03:19:31.165>
closer = 'terry.reedy'
components = ['Tests']
creation = <Date 2007-10-12.18:29:49.945>
creator = 'bear'
dependencies = []
files = ['18291']
hgrepos = []
issue_num = 1274
keywords = ['patch', 'easy']
message_count = 12.0
messages = ['56381', '63766', '63836', '63852', '63874', '112198', '112204', '112224', '112251', '112256', '116781', '185798']
nosy_count = 8.0
nosy_names = ['tim.peters', 'terry.reedy', 'jafo', 'vstinner', 'bear', 'ilan', 'meatballhat', 'BreamoreBoy']
pr_nums = []
priority = 'low'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1274'
versions = ['Python 3.2', 'Python 3.3']

[bear]

In our builds the included patch fixes this issue.

Patch by Brian Kirsch, tested at OSAF on Python 2.5.1

[ilan]

Bug is most likely platform specific. Can someone suggest how this
should be handled on multiple platforms?

Mike, can you report on which platform you encountered the bug on?
Can you provide a script that reproduces the bug?

On Mac OS 10.4, Python 2.5 I could not create a file:
>>> f=open('\xed', 'w')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: invalid mode: w

I will submit this as a separate bug because the error
message sould say 'invalid file name' instead of 'invalid mode'.

[bear]

Hi,

it was running on FC4 with UTF-32 support and was using the Japanese locale.

The bug is reproducible using any doctest that is stored in a mixed
character path. where it is in the Chandler tree is not easily pulled
apart but if you really need it I can work up a small subset.

On Mon, Mar 17, 2008 at 4:41 PM, Ilan Schnell <report@bugs.python.org> wrote:
>
>  Ilan Schnell <ilanschnell@gmail.com> added the comment:
>
>  Bug is most likely platform specific.  Can someone suggest how this
>  should be handled on multiple platforms?
>
>  Mike, can you report on which platform you encountered the bug on?
>  Can you provide a script that reproduces the bug?
>
>  On Mac OS 10.4, Python 2.5 I could not create a file:
>  >>> f=open('\xed', 'w')
>  Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>  IOError: invalid mode: w
>
>  I will submit this as a separate bug because the error
>  message sould say 'invalid file name' instead of 'invalid mode'.
>
>  

---

assignee: -> tim_one
nosy: +ilan, tim_one
priority: -> low
versions: +Python 2.6

---

Tracker <report@bugs.python.org>
<http://bugs.python.org/issue1274\>

---

[jafo]

This may be fixed already, or a bug in FC4. Or perhaps you could
provide more information on how the bug is invoked. I was able to
successfully execute a doctest with "\xee" in the path on an F8 box:

Python 2.5.1 (r251:54863, Oct 30 2007, 13:54:11)
[GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2

Also: Please do not quote text in the reply.
Type "help", "copyright", "credits" or "license" for more information.
>>> import doctest
>>> doctest.testfile('foo\xeebar/test.txt')
(0, 1)
>>> 
guin:pytest$ cat fo*/test.txt
The ``example`` module

======================

Using print
-------------------

This is a test example.

   >>> print 'Hello world!'
   Hello world!

[bear]

Not in the system PATH but in the path where the file is stored:

Here is the original traceback from the bug report for Chandler:

Traceback (most recent call last):
  File
"/Development/osaf/chandler_石田リチャード/chandler/release/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/doctest.py",
line 2107, in runTest
    test, out=new.write, clear_globs=False)
  File
"/Development/osaf/chandler_石田リチャード/chandler/release/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/doctest.py",
line 1345, in run
    return self.__run(test, compileflags, out)
  File
"/Development/osaf/chandler_石田リチャード/chandler/release/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/doctest.py",
line 1236, in __run
    got += _exception_traceback(exc_info)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 70:
ordinal not in range(128)

On Mon, Mar 17, 2008 at 10:40 PM, Sean Reifschneider
<report@bugs.python.org> wrote:
>
>  Sean Reifschneider <jafo@tummy.com> added the comment:
>
>  This may be fixed already, or a bug in FC4.  Or perhaps you could
>  provide more information on how the bug is invoked.  I was able to
>  successfully execute a doctest with "\xee" in the path on an F8 box:
>
>  Python 2.5.1 (r251:54863, Oct 30 2007, 13:54:11)
>  [GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2
>
>  Also: Please do not quote text in the reply.
>  Type "help", "copyright", "credits" or "license" for more information.
>  >>> import doctest
>  >>> doctest.testfile('foo\xeebar/test.txt')
>  (0, 1)
>  >>>
>  guin:pytest$ cat fo*/test.txt
>  The ``example`` module
>  

======================

Using print
-------------------

This is a test example.

>>> print 'Hello world!'
Hello world!

----------
nosy: +jafo

---

Tracker <report@bugs.python.org>
<http://bugs.python.org/issue1274\>

---

[meatballhat]

just updating the patch

[meatballhat]

seems to be okay with py3k's unicode StringIO

adding test to show as much (although I'm sure the test could be more elegant than what I've done.)

[vstinner]

I didn't read all comments, but about bug_3740_1.patch: it's not a good idea to fallback to byte strings. If you have an unicode path, you are lucky, and you have to keep all these nice and correctly formatted characters!

I mean: this patch is just an ugly workaround. You should fix the real bug instead.

[meatballhat]

@Haypo - I'm not in favor of using the attached bug_3740_1.patch but instead adding a test to assert that unicode file paths are now handled correctly. I can't remove @bear's original patch myself, though, so ... not sure what to do about that :)

[bear]

wow - that is some old bug history ;)

I'm surprised the patch is even close to being valid

On Sat, Jul 31, 2010 at 21:47, Dan Buch <report@bugs.python.org> wrote:

Dan Buch <daniel.buch@gmail.com> added the comment:

@Haypo - I'm not in favor of using the attached bug_3740_1.patch but instead adding a test to assert that unicode file paths are now handled correctly.  I can't remove @bear's original patch myself, though, so ... not sure what to do about that :)

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue1274\>

---

[BreamoreBoy]

The patch simply changes test_doctest, is it acceptable or not?

[terryjreedy]

I am closing this for lack of evidence that there is a bug in a current release 2.7.3 or 3.3.0 or release candidate for 2.7.4 or 3.3.1. What would be needed is a self-contained example that shows a problem with one of the above.