Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify ABI for pantsbuild.pants wheel and build with both UCS2 and UCS4 #7235

Merged
Merged
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
a02dde1
Add ext_modules to BUILD entry
Eric-Arellano Feb 7, 2019
1e83f37
Setup UCS2 vs UCS4 travis shards
Eric-Arellano Feb 12, 2019
f7472aa
Merge branch 'master' of github.com:pantsbuild/pants into py2-wheels-…
Eric-Arellano Feb 15, 2019
28b9e8b
Turn on PEX_VERBOSE for OSX ucs4 shard
Eric-Arellano Feb 15, 2019
e87f567
Fix typo in Dockerfile path
Eric-Arellano Feb 17, 2019
170e9c8
Install OpenSSL on OSX UCS4 shard
Eric-Arellano Feb 17, 2019
8d69dc2
Improve naming of Build Wheels shards
Eric-Arellano Feb 17, 2019
6ecb550
Fix Dockerfile copyright year and add comment
Eric-Arellano Feb 17, 2019
7c17c0a
Merge branch 'master' of github.com:pantsbuild/pants into py2-wheels-…
Eric-Arellano Feb 19, 2019
edf81ef
Run PEX with -v*9
Eric-Arellano Feb 19, 2019
0b59e46
Fix gcc no input file issues by passing dummy file
Eric-Arellano Feb 19, 2019
ab534e2
Bootstrap Pants when using new Python install
Eric-Arellano Feb 19, 2019
fb9ef9b
Revert "Run PEX with -v*9"
Eric-Arellano Feb 19, 2019
ad45b2d
Revert "Turn on PEX_VERBOSE for OSX ucs4 shard"
Eric-Arellano Feb 19, 2019
dc36d94
Also deduplicate Pyenv env vars for OSX
Eric-Arellano Feb 19, 2019
513cd50
release.sh still needs to run from PEX
Eric-Arellano Feb 19, 2019
0da5f91
Stop pulling down PEX
Eric-Arellano Feb 19, 2019
621137e
Fix improper call to {osx,linux}_config_env
Eric-Arellano Feb 19, 2019
38e1cf7
Remove unncessary RUN_PANTS_FROM_PEX=0
Eric-Arellano Feb 19, 2019
7da092b
Fix platform default shards bootstrapping
Eric-Arellano Feb 20, 2019
c9e1650
Merge branch 'master' of github.com:pantsbuild/pants into py2-wheels-…
Eric-Arellano Feb 22, 2019
2cd72e4
Add back logging to debug osx ucs4
Eric-Arellano Feb 22, 2019
e502f58
Fix linux ucs4 stage being overriden to cron instead of test
Eric-Arellano Feb 22, 2019
04c4ee0
Move debugging to proper location
Eric-Arellano Feb 22, 2019
78a1aa9
Merge branch 'master' of github.com:pantsbuild/pants into py2-wheels-…
Eric-Arellano Feb 26, 2019
48ef4dd
Squashed commit of the following:
Eric-Arellano Feb 26, 2019
8428376
Fix typo from squashed pex-constraints
Eric-Arellano Feb 26, 2019
223541e
Fix Linux UCS2 using 2.7.13 with UCS4 instead of UCS2 sometimes
Eric-Arellano Feb 26, 2019
f530843
Move unit tests above wheel building shards
Eric-Arellano Feb 26, 2019
1be9e90
Squashed commit of the following:
Eric-Arellano Feb 26, 2019
225f153
Remove bad merge lines
Eric-Arellano Feb 26, 2019
8402c1f
Ensure Linux UCS2 always uses Py2.7.15 (UCS2)
Eric-Arellano Feb 26, 2019
b9efbf0
Also set PANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTS for OSX
Eric-Arellano Feb 26, 2019
f810849
Move interpreter constraints to Docker env entry
Eric-Arellano Feb 26, 2019
c3dd843
Improve wording.
Eric-Arellano Feb 26, 2019
49fe576
Stop hardcoding interpreter constraints
Eric-Arellano Feb 26, 2019
8dd215d
Allow user to set $PY in ci.sh
Eric-Arellano Feb 26, 2019
2166efe
Set $PY to disambiguate which Py2.7 version to use
Eric-Arellano Feb 26, 2019
3bf2d28
Add Pyenv back to Travis path
Eric-Arellano Feb 26, 2019
373ffee
Configure PEX_PYTHON on linux UCS2
Eric-Arellano Feb 27, 2019
0d25bcc
Merge branch 'master' of github.com:pantsbuild/pants into py2-wheels-…
Eric-Arellano Feb 27, 2019
4cb6cae
Squashed commit of the following:
Eric-Arellano Feb 27, 2019
efaae09
Add debugging to release.sh for linux ucs2
Eric-Arellano Feb 27, 2019
8104abc
Merge branch 'master' of github.com:pantsbuild/pants into py2-wheels-…
Eric-Arellano Mar 7, 2019
6272e78
Try setting PY for release.sh script
Eric-Arellano Mar 7, 2019
b755119
Temporarily move Linux UCS2 to top of CI
Eric-Arellano Mar 7, 2019
aedf0ce
Try setting PY for bootstrap also
Eric-Arellano Mar 7, 2019
9798a4b
See if reverting the stages fixes bad interpreter resolve
Eric-Arellano Mar 7, 2019
01b59f0
Also remove PY variable from Linux UCS2
Eric-Arellano Mar 7, 2019
a4cc818
Add back PY to linux UCS2 and skip bootstrap
Eric-Arellano Mar 7, 2019
91d7dd0
Allow ./pants2 to have preset interpreter constraints
Eric-Arellano Mar 7, 2019
d38d80f
Merge branch 'master' of github.com:pantsbuild/pants into py2-wheels-…
Eric-Arellano Mar 7, 2019
23bb6fd
Test if removing Conan default interpreter constraints fixes bootstrap
Eric-Arellano Mar 7, 2019
2057c9a
Properly test that we're building the right UCS
Eric-Arellano Mar 8, 2019
e157960
Allow user to predefine $PY + add interpreter constraints to release.sh
Eric-Arellano Mar 8, 2019
145f484
Remove comment about Python 3
Eric-Arellano Mar 8, 2019
64d5d7f
Use PEX_PYTHON_PATH over PEX_PYTHON
Eric-Arellano Mar 8, 2019
aec536d
Update Dockerfile comment to mention release.sh changes
Eric-Arellano Mar 8, 2019
d45c3fc
Also ensure Linux UCS4 and OSX UCS2 have correct encoding
Eric-Arellano Mar 8, 2019
f961150
Remove bad ./pants.pex -c call
Eric-Arellano Mar 8, 2019
bd15818
Ensure ./pants.pex is correct by inspecting PEX-INFO
Eric-Arellano Mar 8, 2019
0ebb0d8
Squashed commit of the following:
Eric-Arellano Mar 8, 2019
270a0e7
Install unzip on Docker images
Eric-Arellano Mar 8, 2019
67d7fdd
Revert Conan interpreter change
Eric-Arellano Mar 8, 2019
dd930ee
Make new check_pants_pex_abi.sh script
Eric-Arellano Mar 8, 2019
489a3ed
Use `set -x` for check abi script
Eric-Arellano Mar 8, 2019
58b09f9
Also run OSX UCS4 in bootstrap stage
Eric-Arellano Mar 8, 2019
a432f6c
Merge branch 'master' of github.com:pantsbuild/pants into py2-wheels-…
Eric-Arellano Mar 8, 2019
28d9ac9
Fix Centos6 issue
Eric-Arellano Mar 8, 2019
8e8164e
Restore Python 2 bootstrap shards
Eric-Arellano Mar 8, 2019
6a06eec
Remove debugging code
Eric-Arellano Mar 8, 2019
893be87
Remove bad pants.pex
Eric-Arellano Mar 8, 2019
33fd022
Restore full CI suite
Eric-Arellano Mar 8, 2019
a3d5ec5
Add back debugging and limited CI
Eric-Arellano Mar 8, 2019
7eb2741
Revert "Add back debugging and limited CI"
Eric-Arellano Mar 8, 2019
ab502fd
Move up check abi script's opt parsing
Eric-Arellano Mar 8, 2019
cbd325e
Must source common.sh before calling die
Eric-Arellano Mar 8, 2019
0544947
Port check_pants_pex_abi.sh to Python
Eric-Arellano Mar 8, 2019
519ce59
Improve release.sh check for valid interpreter
Eric-Arellano Mar 8, 2019
23fb23b
Explain better why we duplicate travis_ci/Dockerfile
Eric-Arellano Mar 8, 2019
ee030c7
Add comments to pants/BUILD
Eric-Arellano Mar 9, 2019
c7ae6c4
Merge branch 'master' of github.com:pantsbuild/pants into py2-wheels-…
Eric-Arellano Mar 9, 2019
463f38a
Extract out launch_docker_image
Eric-Arellano Mar 9, 2019
d64845f
Restore env_osx_with_pyenv being multiline
Eric-Arellano Mar 9, 2019
67c4557
Remove unused anchor and import
Eric-Arellano Mar 9, 2019
971eadf
Add success message and improve missing pants.pex for new check script
Eric-Arellano Mar 9, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

@@ -170,7 +170,6 @@ py36_osx_config: &py36_osx_config
packages: &py36_osx_config_brew_packages
- openssl
env:
# Fix Python 3 issue linking to OpenSSL
- &py36_osx_config_env >
PATH="/usr/local/opt/openssl/bin:$PATH"
LDFLAGS="-L/usr/local/opt/openssl/lib"
@@ -234,14 +233,14 @@ travis_docker_image: &travis_docker_image
before_script:
- ulimit -c unlimited
script:
# NB: This script definition is very likely to be overridden in a consumer, so we alias it
# for re-inclusion.
- &travis_docker_image_launch docker build --rm -t travis_ci
- >
docker build
--rm -t ${docker_image_name}
--build-arg "TRAVIS_USER=$(id -un)"
--build-arg "TRAVIS_UID=$(id -u)"
--build-arg "TRAVIS_GROUP=$(id -gn)"
--build-arg "TRAVIS_GID=$(id -g)"
build-support/docker/travis_ci/
build-support/docker/${docker_image_name}/
# -------------------------------------------------------------------------
# Bootstrap engine shards
@@ -252,14 +251,21 @@ base_linux_build_engine: &base_linux_build_engine
<<: *travis_docker_image
stage: *bootstrap
script:
- *travis_docker_image_launch
- >
docker build
--rm -t ${docker_image_name}
--build-arg "TRAVIS_USER=$(id -un)"
--build-arg "TRAVIS_UID=$(id -u)"
--build-arg "TRAVIS_GROUP=$(id -gn)"
--build-arg "TRAVIS_GID=$(id -g)"
build-support/docker/${docker_image_name}/
# Note that:
# * We mount ${HOME} to cache the ${HOME}/.cache/pants/rust-toolchain.
# * We also build fs_util, to take advantage of the rust code built during bootstrapping.
- docker run --rm -t
-v "${HOME}:/travis/home"
-v "${TRAVIS_BUILD_DIR}:/travis/workdir"
travis_ci:latest
${docker_image_name}:latest
sh -c "./build-support/bin/ci.sh ${BOOTSTRAP_ARGS} && ./build-support/bin/release.sh -f"
- aws --no-sign-request --region us-east-1 s3 cp ${TRAVIS_BUILD_DIR}/pants.pex ${BOOTSTRAPPED_PEX_URL_PREFIX}.${BOOTSTRAPPED_PEX_KEY_SUFFIX}

@@ -268,6 +274,7 @@ py27_linux_build_engine: &py27_linux_build_engine
<<: *base_linux_build_engine
name: "Build Linux native engine and pants.pex (Py2.7 PEX)"
env:
- docker_image_name=travis_ci
# NB: Only the Py2.7 shard sets PREPARE_DEPLOY to cause the fs_util binary to be uploaded to S3:
# either linux shard could upload this binary, since it does not depend on python at all.
- PREPARE_DEPLOY=1
@@ -280,6 +287,7 @@ py36_linux_build_engine: &py36_linux_build_engine
<<: *base_linux_build_engine
name: "Build Linux native engine and pants.pex (Py3.6 PEX)"
env:
- docker_image_name=travis_ci
- CACHE_NAME=linuxpexbuild.py36
- BOOTSTRAPPED_PEX_KEY_SUFFIX=py36.linux
- BOOTSTRAP_ARGS='-b'
@@ -381,41 +389,128 @@ cargo_audit: &cargo_audit
# Build wheels
# -------------------------------------------------------------------------

# N.B. With Python 2, we must build pantsbuild.pants with both UCS2 and UCS4 to provide full

This comment has been minimized.

Copy link
@benjyw

benjyw Mar 8, 2019

Contributor

Good comment!

# compatibility for end users. This is because we constrain our ABI due to the native engine.
# See https://www.python.org/dev/peps/pep-0513/#ucs-2-vs-ucs-4-builds. Note this distinction is
# not necessary with Python 3.3+ due to flexible storage of Unicode strings (https://www.python.org/dev/peps/pep-0393/).
#
# We treat both Linux UCS4 and OSX UCS2 normally, as these are the defaults for those environments.
# The Linux UCS2 and OSX UCS4 shards, however, must rebuild Python with
# `PYTHON_CONFIGURE_OPTS=--enable-unicode=ucs{2,4}` set, along with bootstrapping Pants again rather
# than pulling the PEX from AWS.

base_build_wheels: &base_build_wheels
stage: *test
env:
- &base_build_wheels_env RUN_PANTS_FROM_PEX=1 PREPARE_DEPLOY=1
- &base_build_wheels_env PREPARE_DEPLOY=1

linux_build_wheels: &linux_build_wheels
py27_linux_build_wheels_no_ucs: &py27_linux_build_wheels_no_ucs
# Similar to the bootstrap shard, we build Linux wheels in a docker image to maximize
# compatibility. This is a Py2.7 shard, so it is not subject to #6985.
<<: *travis_docker_image
<<: *py27_linux_test_config
<<: *base_build_wheels
name: "Build Linux wheels (No PEX)"

py27_linux_build_wheels_ucs2: &py27_linux_build_wheels_ucs2
<<: *py27_linux_config
<<: *py27_linux_build_wheels_no_ucs
<<: *native_engine_cache_config
name: "Build wheels - Linux and cp27m (UCS2)"
env:
- *base_build_wheels_env
- docker_image_name=travis_ci_py27_ucs2
- CACHE_NAME=linuxwheelsbuild.ucs2
script:
- >
docker build
--rm -t ${docker_image_name}
--build-arg "TRAVIS_USER=$(id -un)"
--build-arg "TRAVIS_UID=$(id -u)"
--build-arg "TRAVIS_GROUP=$(id -gn)"
--build-arg "TRAVIS_GID=$(id -g)"
build-support/docker/${docker_image_name}/
- docker run --rm -t
-v "${HOME}:/travis/home"
-v "${TRAVIS_BUILD_DIR}:/travis/workdir"
${docker_image_name}:latest
sh -c "./build-support/bin/ci.sh -2b
&& ./build-support/bin/check_pants_pex_abi.py cp27m
&& RUN_PANTS_FROM_PEX=1 ./build-support/bin/release.sh -n"

py27_linux_build_wheels_ucs4: &py27_linux_build_wheels_ucs4
<<: *py27_linux_build_wheels_no_ucs
<<: *py27_linux_test_config
# `py27_linux_test_config` overrides the stage set by `base_build_wheels`, so we re-override it.
stage: *test
name: "Build wheels - Linux and cp27mu (UCS4)"
env:
- *py27_linux_test_config_env
- *base_build_wheels_env
- CACHE_NAME=linuxwheelsbuild
- docker_image_name=travis_ci
- CACHE_NAME=linuxwheelsbuild.ucs4
script:
- *travis_docker_image_launch
- >
docker build
--rm -t ${docker_image_name}
--build-arg "TRAVIS_USER=$(id -un)"
--build-arg "TRAVIS_UID=$(id -u)"
--build-arg "TRAVIS_GROUP=$(id -gn)"
--build-arg "TRAVIS_GID=$(id -g)"
build-support/docker/${docker_image_name}/
- docker run --rm -t
-v "${HOME}:/travis/home"
-v "${TRAVIS_BUILD_DIR}:/travis/workdir"
travis_ci:latest
sh -c "RUN_PANTS_FROM_PEX=1 ./build-support/bin/release.sh -n"
${docker_image_name}:latest
sh -c "./build-support/bin/check_pants_pex_abi.py cp27mu
&& RUN_PANTS_FROM_PEX=1 ./build-support/bin/release.sh -n"

osx_build_wheels: &osx_build_wheels
<<: *py27_osx_test_config
py27_osx_build_wheels_no_ucs: &py27_osx_build_wheels_no_ucs
<<: *base_build_wheels
name: "Build OSX wheels (No PEX)"
osx_image: xcode8

py27_osx_build_wheels_ucs2: &py27_osx_build_wheels_ucs2
<<: *py27_osx_test_config
<<: *py27_osx_build_wheels_no_ucs
name: "Build wheels - OSX and cp27m (UCS2)"
env:
- *py27_osx_test_config_env
- *base_build_wheels_env
- CACHE_NAME=osxwheelsbuild
- CACHE_NAME=osxwheelsbuild.ucs2
script:
- ./build-support/bin/check_pants_pex_abi.py cp27m
- RUN_PANTS_FROM_PEX=1 ./build-support/bin/release.sh -n

py27_osx_build_wheels_ucs4: &py27_osx_build_wheels_ucs4
<<: *py27_osx_config
<<: *py27_osx_build_wheels_no_ucs
<<: *native_engine_cache_config
name: "Build wheels - OSX and cp27mu (UCS4)"
addons:
brew:
packages:
- openssl
env:
- *base_build_wheels_env
- CACHE_NAME=osxwheelsbuild.ucs4
- >
PATH="/usr/local/opt/openssl/bin:$PATH"
LDFLAGS="-L/usr/local/opt/openssl/lib"
CPPFLAGS="-I/usr/local/opt/openssl/include"
PYENV_ROOT="${HOME}/.pyenv"
PATH="${PYENV_ROOT}/shims:${PATH}"
- PYTHON_CONFIGURE_OPTS=--enable-unicode=ucs4
# We set $PY to ensure the UCS4 interpreter is used when bootstrapping the PEX.
- PY=${PYENV_ROOT}/shims/python2.7
before_install:
- curl -L https://github.com/stedolan/jq/releases/download/jq-1.5/jq-osx-amd64 -o /usr/local/bin/jq
- chmod 755 /usr/local/bin/jq
- ./build-support/bin/install_aws_cli_for_ci.sh
- git clone https://github.com/pyenv/pyenv ${PYENV_ROOT}
- ${PYENV_ROOT}/bin/pyenv install 2.7.13
- ${PYENV_ROOT}/bin/pyenv global 2.7.13
script:
- ./build-support/bin/release.sh -n
- ./build-support/bin/ci.sh -2b
- ./build-support/bin/check_pants_pex_abi.py cp27mu
- RUN_PANTS_FROM_PEX=1 ./build-support/bin/release.sh -n

# -------------------------------------------------------------------------
# Rust tests
@@ -607,7 +702,6 @@ matrix:
include:
- <<: *py27_linux_build_engine
- <<: *py36_linux_build_engine

- <<: *py27_osx_build_engine
- <<: *py36_osx_build_engine

@@ -617,9 +711,6 @@ matrix:
- <<: *linux_rust_clippy
- <<: *cargo_audit

- <<: *linux_build_wheels
- <<: *osx_build_wheels

- <<: *py27_linux_test_config
name: "Unit tests for pants and pants-plugins (Py2.7 PEX)"
stage: *test
@@ -637,6 +728,11 @@ matrix:
script:
- ./build-support/bin/travis-ci.sh -lp

- <<: *py27_linux_build_wheels_ucs2
- <<: *py27_linux_build_wheels_ucs4
- <<: *py27_osx_build_wheels_ucs2
- <<: *py27_osx_build_wheels_ucs4

- <<: *py36_linux_test_config
name: "Integration tests for pants - shard 0 (Py3.6 PEX)"
env:
@@ -0,0 +1,70 @@
#!/usr/bin/env python2.7
This conversation was marked as resolved by Eric-Arellano

This comment has been minimized.

Copy link
@Eric-Arellano

Eric-Arellano Mar 9, 2019

Author Contributor

This is so much better than the bash script. Thanks for the advice to do this and the bash tips as well! I incorporated a lot of your ideas, e.g. only grabbing the file we need from unzip rather than unzipping the whole thing.

# coding=utf-8
# Copyright 2019 Pants project contributors (see CONTRIBUTORS.md).
# Licensed under the Apache License, Version 2.0 (see LICENSE).

# Check that the ./pants.pex was built using the passed abi specification.

from __future__ import absolute_import, division, print_function, unicode_literals

import argparse
import json
import os.path
import zipfile


RED = "\033[31m"
BLUE = "\033[34m"
RESET = "\033[0m"


def main():
if not os.path.isfile("pants.pex"):
die("pants.pex not found! Ensure you are in the repository root, then run " \
"'./build-support/bin/ci.sh -b' to bootstrap pants.pex with Python 3 or " \
"'./build-support/bin/ci.sh -2b' to bootstrap pants.pex with Python 2.")
expected_abi = create_parser().parse_args().abi
with zipfile.ZipFile("pants.pex", "r") as pex:
with pex.open("PEX-INFO", "r") as pex_info:
pex_info_content = str(pex_info.readline())
parsed_abis = {
parse_abi_from_filename(filename)
for filename in json.loads(pex_info_content)["distributions"].keys()
if parse_abi_from_filename(filename) != "none"
}
if len(parsed_abis) < 1:
die("No abi tag found. Expected: {}.".format(expected_abi))
elif len(parsed_abis) > 1:
die("Multiple abi tags found. Expected: {}, found: {}.".format(expected_abi, parsed_abis))
found_abi = list(parsed_abis)[0]
if found_abi != expected_abi:
die("pants.pex was built with the incorrect ABI. Expected: {}, found: {}.".format(expected_abi, found_abi))
success("Success. As expected, pants.pex was built with the ABI {}.".format(expected_abi))

This conversation was marked as resolved by Eric-Arellano

This comment has been minimized.

Copy link
@cosmicexplorer

cosmicexplorer Mar 9, 2019

Contributor

Would add some sort of success message here.


def create_parser():
parser = argparse.ArgumentParser(
description="Check that ./pants.pex was built using the passed abi specification."
)
parser.add_argument("abi", help="The expected abi, e.g. `cp27m` or `abi3`")
return parser


def parse_abi_from_filename(filename):
"""This parses out the abi from a wheel filename.
For example, `configparser-3.5.0-py2-abi3-any.whl` would return `abi3`.
See https://www.python.org/dev/peps/pep-0425/#use for how wheel filenames are defined."""
return filename.split("-")[-2]


def success(message):
print("{}{}{}".format(BLUE, message, RESET))


def die(message):
raise SystemExit("{}{}{}".format(RED, message, RESET))


if __name__ == "__main__":
main()
@@ -7,9 +7,22 @@ set -e
ROOT=$(cd $(dirname "${BASH_SOURCE[0]}") && cd "$(git rev-parse --show-toplevel)" && pwd)
source ${ROOT}/build-support/common.sh

PY=$(which python2.7 || exit 0)
[[ -n "${PY}" ]] || die "You must have python2.7 installed and on the path to release."
export PY
# Set the Python interpreter to be used for the virtualenv. Note we allow the user to
# predefine this value so that they may point to a specific interpreter, e.g. 2.7.13 vs. 2.7.15.
export PY="${PY:-python2.7}"
if ! which "${PY}" >/dev/null; then
die "Python interpreter ${PY} not discoverable on your PATH."
fi
py_major_minor=$(${PY} -c 'import sys; print(".".join(map(str, sys.version_info[0:2])))')
if [[ "${py_major_minor}" != "2.7" ]]; then
die "Invalid interpreter. The release script requires python2.7, and you are using python${py_major_minor}."
fi
This conversation was marked as resolved by Eric-Arellano

This comment has been minimized.

Copy link
@Eric-Arellano

Eric-Arellano Mar 9, 2019

Author Contributor

@benjyw I split out the original one check into two checks to verify 1) the interpreter is on the PATH and 2) it's specifically python2.7, as we don't support Py3 for release.sh (yet).


# Also set PANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTS. We set this to the exact Python version
# to resolve any potential ambiguity when multiple Python interpreters are discoverable, such as
# Python 2.7.13 vs. 2.7.15.
py_major_minor_patch=$(${PY} -c 'import sys; print(".".join(map(str, sys.version_info[0:3])))')
export PANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTS="${PANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTS:-['CPython==${py_major_minor_patch}']}"

function run_local_pants() {
${ROOT}/pants "$@"
@@ -0,0 +1,49 @@
# Copyright 2019 Pants project contributors (see CONTRIBUTORS.md).
# Licensed under the Apache License, Version 2.0 (see LICENSE).

# NB: this file duplicates travis_ci/Dockerfile, except it installs Python 2.7 with UCS2.
# We do not include this change directly in centos6/Dockerfile nor in travis_ci/Dockerfile
# because we only want access to this Python interpreter in the Build Wheels Linux UCS2 shard,
# so it is not helpful to other shards. Rather, it would make those shards more finicky to deal
# with by resulting in two Python 2.7 installs: system vs. pyenv.

# Use our custom Centos6 image for binary compatibility with old linux distros.
FROM pantsbuild/centos6:latest

# Note we use 2.7.15, rather than 2.7.13, as the centos6 image already comes with 2.7.13
# installed, which uses UCS4 instead of UCS2. This allows us to disambiguate which Python 2
# interpreter to use when `ci.sh` and `release.sh` set the interpreter constraints for
# Pants, and thus for the built ./pants.pex. We set $PY to the exact Python 2.7 version we want
# to ensure the PEX is bootstrapped with UCS 2.
ARG PYTHON_2_VERSION=2.7.15
# TODO(7064): remove this yum install line once we update the base Centos6 image to include this dependency.
RUN yum install sqlite-devel -y
ENV PYENV_ROOT /pyenv-docker-build
RUN mkdir ${PYENV_ROOT}
RUN git clone https://github.com/pyenv/pyenv ${PYENV_ROOT}
ENV PYTHON_CONFIGURE_OPTS --enable-unicode=ucs2
RUN /usr/bin/scl enable devtoolset-7 -- bash -c '\
${PYENV_ROOT}/bin/pyenv install ${PYTHON_2_VERSION} \
&& ${PYENV_ROOT}/bin/pyenv global ${PYTHON_2_VERSION}'
ENV PATH "${PYENV_ROOT}/shims:${PATH}"
ENV PY "${PYENV_ROOT}/shims/python2.7"
ENV PEX_PYTHON_PATH "${PYENV_ROOT}/shims/python2.7"

# Setup mount points for the travis ci user & workdir.
VOLUME /travis/home
VOLUME /travis/workdir

# Setup a non-root user to execute the build under (avoids problems with npm install).
ARG TRAVIS_USER=travis_ci
ARG TRAVIS_UID=1000
ARG TRAVIS_GROUP=root
ARG TRAVIS_GID=0

RUN groupadd --gid ${TRAVIS_GID} ${TRAVIS_GROUP} || true
RUN useradd -d /travis/home -g ${TRAVIS_GROUP} --uid ${TRAVIS_UID} ${TRAVIS_USER}
USER ${TRAVIS_USER}:${TRAVIS_GROUP}

# Our newly created user is unlikely to have a sane environment: set a locale at least.
ENV LC_ALL="en_US.UTF-8"

WORKDIR /travis/workdir
@@ -0,0 +1,5 @@
PATH="/usr/local/opt/openssl/bin:$PATH"
LDFLAGS="-L/usr/local/opt/openssl/lib"
CPPFLAGS="-I/usr/local/opt/openssl/include"
PYENV_ROOT="${HOME}/.pyenv"
PATH="${PYENV_ROOT}/shims:${PATH}"
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.