Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BLD: declare support for python 3.8 #14758

Merged
merged 1 commit into from Oct 24, 2019
Merged

Conversation

@mattip
Copy link
Member

mattip commented Oct 22, 2019

The descriptors on PyPI are missing Python 3.8

@mattip

This comment has been minimized.

Copy link
Member Author

mattip commented Oct 22, 2019

the macos build is failing, this seems to be the relevant part of the log:

Installed /Users/runner/hostedtoolcache/Python/3.6.9/x64/lib/python3.6/site-packages/numpy-1.17.4.dev0+2ec2d35-py3.6-macosx-10.14-x86_64.egg
Processing dependencies for numpy==1.17.4.dev0+2ec2d35
Searching for numpy==1.17.4.dev0+2ec2d35
Reading https://pypi.org/simple/numpy/
No local packages or working download links found for numpy==1.17.4.dev0+2ec2d35
@rgommers rgommers changed the title BUILD: declare support for python 3.8 BLD: declare support for python 3.8 Oct 22, 2019
@ihnorton

This comment has been minimized.

Copy link
Contributor

ihnorton commented Oct 23, 2019

FWIW, I'm seeing the same kind of build failure (No local packages or working download links ... Could not find suitable distribution for Requirement.parse('tiledb==0.4.4.dev62')) on azure/macOS (link) for a different package also using the macOS-10.13 image. The problem seems to be a mismatch between platform.mac_ver and distutils.util.get_platform(). With some debugging printouts I get:

platform.mac_ver: ('10.13.6', ('', '', ''), 'x86_64')
distutils.util.get_platform:  macosx-10.14-x86_64

Which I believe blocks matching the locally-built package, because the target minimum is not met by the actual system.

It is unclear where that difference is coming from, because MACOSX_DEPLOYMENT_TARGET is not set by the environment (test, output MACOSX_DEPLOYMENT_TARGET: <none>)

But when I print out distutils.sysconfig.get_config_vars, it shows MACOSX_DEPLOYMENT_TARGET=10.14...

Manually setting MACOSX_DEPLOYMENT_TARGET=10.13 in pipeline environment via yml breaks the psutil build/install 🤷‍♂

Debugging attempts here: TileDB-Inc/TileDB-Py#214

@rgommers

This comment has been minimized.

Copy link
Member

rgommers commented Oct 23, 2019

Thanks @ihnorton. That doesn't look fun to debug. Sounds like a bug in Python itself, please ping us if you file an issue on the Python bug tracker.

@ihnorton

This comment has been minimized.

Copy link
Contributor

ihnorton commented Oct 23, 2019

I believe what is happening is that the Python interpreter from the azure UsePythonVersion task was built with MACOSX_DEPLOYMENT_TARGET=10.14. That's where the cached distutils.sysconfig.get_config_vars value is coming from. This value sets the result of distutils.util.get_platform.

Now, note that eggs and local build tmpdir are named based on the output of distutils.util.get_platform: https://github.com/pypa/setuptools/blob/3b8307541e6d24c5eeb4e4998cf9ee46b6e6506f/setuptools/wheel.py#L82-L86

Working from the error location

Minimal test repo: https://github.com/ihnorton/azure-mac-py-version
Build log: https://dev.azure.com/isaiahnorton/isaiahnorton/_build/results?buildId=228


So, I guess distutils doesn't support targeting an older platform than the interpreter was compiled against. It seems like this is arguably an azure bug, in that they installed a python version that is partially incompatible with the image used. I'll open a bug about it (and for my own CI, I will probably switch to miniconda for now ended up switching to macOS-10.14 image).

@ihnorton

This comment has been minimized.

Copy link
Contributor

ihnorton commented Oct 23, 2019

ihnorton added a commit to TileDB-Inc/TileDB-Py that referenced this pull request Oct 23, 2019
The azure macOS-10.13 UsePythonVersion python binary was built with
  MACOSX_DEPLOYMENT_TARGET 10.14, which breaks distutils/setuptools.

x-ref:
  - microsoft/azure-pipelines-image-generation#1342
  - numpy/numpy#14758

---
DEBUG: pin MACOSX_DEPLOYMENT_TARGET

'setup.py install` is failing with the following message

possibly because of a divergence between the following values:
(from debug build w/ printout)

  platform.mac_ver: ('10.13.6', ('', '', ''), 'x86_64')
  distutils.util.get_platform:  macosx-10.14-x86_64

The build error is:

  """
  Installed /Users/runner/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/tiledb-0.4.4.dev55-py3.7-macosx-10.14-x86_64.egg
  Processing dependencies for tiledb==0.4.4.dev55
  Searching for tiledb==0.4.4.dev55
  Reading https://pypi.org/simple/tiledb/
  No local packages or working download links found for tiledb==0.4.4.dev55
  error: Could not find suitable distribution for Requirement.parse('tiledb==0.4.4.dev55')
  """

DEBUG set MACOSX_DEPLOYMENT_TARGET=10.9
DEBUG: try exporting deployment target in the install task

psutil build is failing due to mismatched configure/sdk target version

'error:  mismatch: now 10.9 but 10.14 during configure'

DEBUG: switch to macOS-10.14 azure image :shrug?:

- psutil install breaks w/ MACOSX_DEPLOYMENT_TARGET=10.13/10.9 because someone (?) doesn't respect MACOSX_DEPLOYMENT_TARGET (presumably it is not passed to 'configure' via '-mmacosx-version-min'), so when it tries to build it errors out on the mismatch.

Set MACOSX_DEPLOYMENT_TARGET and only install psutil on linux
DEBUG: *only* target macOS-10.14
@charris charris merged commit f0b8d74 into numpy:maintenance/1.17.x Oct 24, 2019
13 of 16 checks passed
13 of 16 checks passed
azure-pipeline numpy.numpy Build #20191022.6 failed
Details
azure-pipeline numpy.numpy (macOS) macOS failed
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
LGTM analysis: C/C++ No code changes detected
Details
LGTM analysis: JavaScript No code changes detected
Details
LGTM analysis: Python No new or fixed alerts
Details
Shippable Run 6767 status is SUCCESS.
Details
azure-pipeline numpy.numpy (Linux_PyPy3) Linux_PyPy3 succeeded
Details
azure-pipeline numpy.numpy (Linux_Python_36_32bit_full_with_asserts) Linux_Python_36_32bit_full_with_asserts succeeded
Details
azure-pipeline numpy.numpy (Windows Python35-64bit-full) Windows Python35-64bit-full succeeded
Details
azure-pipeline numpy.numpy (Windows Python36-32bit-fast) Windows Python36-32bit-fast succeeded
Details
azure-pipeline numpy.numpy (Windows Python36-64bit-full) Windows Python36-64bit-full succeeded
Details
azure-pipeline numpy.numpy (Windows Python37-32bit-fast) Windows Python37-32bit-fast succeeded
Details
azure-pipeline numpy.numpy (Windows Python37-64bit-full) Windows Python37-64bit-full succeeded
Details
ci/circleci Your tests passed on CircleCI!
Details
codecov/project 85.75% (target 1%)
Details
@charris

This comment has been minimized.

Copy link
Member

charris commented Oct 24, 2019

Might as well get this in now, I forgot to do it for 1.17.x before the release. Thanks Matti.

@charris

This comment has been minimized.

Copy link
Member

charris commented Oct 24, 2019

@ihnorton Thanks for tracking that down.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.