unicode "support" for shlex.py

[jvr]

BPO	742290
Nosy	@loewis
Files	shlex.patch: shlex.py unicode "support"

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2003-07-07.22:07:01.000>
created_at = <Date 2003-05-23.12:47:58.000>
labels = ['library']
title = 'unicode "support" for shlex.py'
updated_at = <Date 2003-07-07.22:07:01.000>
user = 'https://bugs.python.org/jvr'

bugs.python.org fields:

activity = <Date 2003-07-07.22:07:01.000>
actor = 'loewis'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2003-05-23.12:47:58.000>
creator = 'jvr'
dependencies = []
files = ['5319']
hgrepos = []
issue_num = 742290
keywords = ['patch']
message_count = 4.0
messages = ['43824', '43825', '43826', '43827']
nosy_count = 2.0
nosy_names = ['loewis', 'jvr']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue742290'
versions = []

[jvr]

Due to shlex.py's use of cStringIO, it behaves badly when fed unicode
strings. The attached patch fixes that by always using StringIO instead
of cStringIO.

[jvr]

Logged In: YES
user_id=92689

Ugh, I take that back: it doesn't fix it, there's a gross snippet in shlex.py that
makes it barf:

        if self.posix:
            self.wordchars += ('??·???ÂÊÁËÈÍÎÏÌÓÔ?ÒÚÛÙ??¯???¸???'
                               '¿¡¬????«»? ÀÃÕ????????÷ÿ??????')

Help. I'd love to fix this, but I'm not sure what would be correct (my intuition
says to just yank the above snippet, but I'm sure that'll make _someone_
unhappy...).

[loewis]

Logged In: YES
user_id=21627

To test whether a letter is a wordchar, you should check
whether it .isalnum() or equals '_'. Then you can do away
with self.wordchars, and it works the same for byte strings
and Unicode strings. Non-ASCII characters in byte strings
then work if locale.setlocale had been invoked.

[loewis]

Logged In: YES
user_id=21627

I'll reject that patch for now. If you manage to complete
it, feel free to reopen or submit a new one.