Configure: Cannot disable unicode

[redcomet]

BPO	8767
Nosy	@loewis, @ezio-melotti, @bitdancer, @taschini
Files	issue8767.patch issue8767_interpreter.patch issue8767_stdlib.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-05-20.08:47:13.204>
created_at = <Date 2010-05-19.16:21:53.774>
labels = ['build']
title = 'Configure: Cannot disable unicode'
updated_at = <Date 2012-05-20.10:35:58.873>
user = 'https://bugs.python.org/redcomet'

bugs.python.org fields:

activity = <Date 2012-05-20.10:35:58.873>
actor = 'taschini'
assignee = 'none'
closed = True
closed_date = <Date 2012-05-20.08:47:13.204>
closer = 'loewis'
components = ['Build']
creation = <Date 2010-05-19.16:21:53.774>
creator = 'redcomet'
dependencies = []
files = ['25377', '25418', '25419']
hgrepos = []
issue_num = 8767
keywords = ['patch']
message_count = 11.0
messages = ['106076', '106077', '159389', '159442', '159473', '159673', '159674', '159676', '161190', '161191', '161197']
nosy_count = 6.0
nosy_names = ['loewis', 'ezio.melotti', 'r.david.murray', 'redcomet', 'python-dev', 'taschini']
pr_nums = []
priority = 'low'
resolution = 'fixed'
stage = 'needs patch'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue8767'
versions = ['Python 2.7']

[redcomet]

I am compiling python on AIX 5.3. The normal configure and make works, except it fails to compile the unicodedata module. The assembler reports a bunch of these errors:
Error: value of 000000000001268b too large for field of 2 bytes at 000000000000006d

The module is labeled as optional, but if that one fails python will not install. One of the .py files it tries to compile requires unicodedata so the whole thing fails.

I tried --enable-unicode=ucs4 and got the same results so then I tried just --disable-unicode all together and configure spits out:
checking what type to use for unicode... configure: error: invalid value for --enable-unicode. Use either ucs2 or ucs4 (lowercase).

I had to go into the configure script itself remove a default case statement that produced the error to disable it completely.

Building Python2.7b1 on AIX 5.3

[redcomet]

Seems as though python 2.7 should not support --disable-unicode so this ticket is invalid. I just googled python disable unicode, but it seems that the decision to never disable unicode is recent.
Closing

[bitdancer]

Reopening per this python-dev thread where MvL said that not being able to build 2.7 without unicode is a bug (but someone would need to care enough to contribute a patch to fix it).

[taschini]

Here's the patch. It has four goals:

1. Allow ./configure --disable-unicode to work;

2. Have the naked interpreter running with no unicode support;

3. Be able to compile most of the stdlib;

4. Be able to run the test suite.

The one-line modification to configure.ac (and consequentley autoreconf'ed configure) achieve goal 1.

The changes to the three C files (which are nothing more than a few #ifdef Py_USING_UNICODE) achieve goal 2: you can run "./python -S".

The fix for site.py takes care of posixpath, glob, (and a few other modules) and makes it possible to compile most of the C extensions of the stdlib, goal 3 -- The compilation process under posix requires posixpath and glob to work. The most notable extension that won't be built is, unsurprisingly enough, io. Fortunately it does not play in Python 2.7 the central role it plays in Python 3. Still, a few other modules depend on it and won't be usable.

The changes in Lib/test/script_helper.py and Lib/test/test_support.py make it possible to run the test suite. Roughly one third of the tests will fail, though, but I think that's acceptable. In relation to my purpose for submitting this patch ( bpo-1065986 ) , I note that test_pydoc runs and succeeds.

[loewis]

I find the injection of a fake unicode class too hacky. Instead, I suggest that each module does

try:
_unicode = unicode
except NameError:
# no unicode support in this build
class _unicode:
pass

and then have the isinstance checks look for _unicode

[taschini]

Martin,

That was exactly my first approach. What made me change my mind is that

i) it is also fairly hacky (one might rightfully object that it is the isinstance(x, unicode) tests that should be changed)

ii) it is now a hack spread over a dozen files, instead of the site.py alone.

iii) the alterations in those files are executed even in the case of built-in unicode support, thus increasing the risk of introducing a regression in the stdlib.

In the end I was a bit loath to alter quite a few of the stdlib modules (including some of the "core" ones) for a rather infrequent case. My solution, on the other hand, is such that in the regular case of built-in unicode support those modules are not touched at all, thus reducing the risk of introducing a regression in the stdlib.

Still, if you guys do think that the maintainability risk due to the hackiness of my suggestion exceeds the potential benefits, it might be better to split the issue (and the patch) into two: one for the autoconf and interpreter, and one for the stdlib. In this way, the patch for autconf and interpreter (which should be less controversial) might be accepted sooner, while we bide our time until we come up with a better solution for the stdlib.

[loewis]

it might be better to split the issue (and the patch) into two: one for the autoconf and interpreter, and one for the stdlib.

Please do that. Splitting the patch could be enough, no need to split
the issue.

[taschini]

Here we go.

[python-dev]

New changeset d7aff4423172 by Martin v. Löwis in branch '2.7':
Issue bpo-8767: Restore building with --disable-unicode.
http://hg.python.org/cpython/rev/d7aff4423172

[loewis]

Thanks for the patch. I did the library patch without monkey-patching.

If there are further issues with this feature, please submit new bug reports. I won't consider the many test failures due to usage of Unicode literals in the test cases as bugs, as working around for the lack of Unicode literals is quite tedious, and would reduce readability.

[taschini]

Understood and agreed.