Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: python3 support for configure #30047

Closed
wants to merge 1 commit into from

Conversation

@rvagg
Copy link
Member

rvagg commented Oct 21, 2019

Discovered while messing with some infra switching things around to python3. On Linux, if you call configure directly with a python3 (not letting it find python for you) and have LC_ALL=C set, as we do in some of our infra, apparently open() defaults to ascii. This is a problem for icu version detection because there's a © at the top of deps/icu-small/source/common/unicode/uvernum.h which it reads to find major version number.

$ LC_ALL=C python3 ./configure.py
Traceback (most recent call last):
  File "./configure.py", line 1660, in <module>
    configure_intl(output)
  File "./configure.py", line 1498, in configure_intl
    for line in open(uvernum_h).readlines():
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3: ordinal not in range(128)

This change fixes it and makes it work for Python 2 & 3. Although I don't really know what I'm doing with the conversion back to ascii for later compatibility so someone who knows better should check.

@rvagg rvagg requested review from srl295 and cclauss Oct 21, 2019
@nodejs-github-bot

This comment has been minimized.

@targos

This comment has been minimized.

Copy link
Member

targos commented Oct 21, 2019

This is similar to #30023

@rvagg

This comment has been minimized.

Copy link
Member Author

rvagg commented Oct 21, 2019

indeed it is! but in this case the return value needs to mix with non-utf8 strings and python2 borks at that, hence the str(x).

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

Copy link
Contributor

cclauss left a comment

LGTM.

Minor nit: for line in open(uvernum_h).readlines(): leaves the filehandle open so the preferred approach is:

with open(uvernum_h) as in_file:
    for line in in_file:

because the with open() as context manager will automate the filehandle close().

@rvagg

This comment has been minimized.

Copy link
Member Author

rvagg commented Oct 21, 2019

ok, have tried that approach, ptal @cclauss

@nodejs-github-bot

This comment has been minimized.

@cclauss

This comment has been minimized.

Copy link
Contributor

cclauss commented Oct 21, 2019

I have already approved. For those who want to dig in...
https://www.python.org/dev/peps/pep-0538/ and https://www.python.org/dev/peps/pep-0540/

@rvagg rvagg force-pushed the rvagg:rvagg/python3-icu-utf8 branch from 40c91e1 to 88ecfee Oct 23, 2019
@nodejs-github-bot

This comment has been minimized.

rvagg added a commit that referenced this pull request Oct 23, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@rvagg

This comment has been minimized.

Copy link
Member Author

rvagg commented Oct 23, 2019

Landed in 779d7ef

@rvagg rvagg closed this Oct 23, 2019
@rvagg rvagg deleted the rvagg:rvagg/python3-icu-utf8 branch Oct 23, 2019
targos added a commit that referenced this pull request Oct 24, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@targos targos mentioned this pull request Nov 5, 2019
targos added a commit that referenced this pull request Nov 8, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
targos added a commit that referenced this pull request Nov 10, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
targos added a commit that referenced this pull request Nov 11, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.