New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to build/test a python 3.6.0 project using jsonschema lib due to pants bringing in a 2.7 dependency #4490

Closed
jessehersch opened this Issue Apr 19, 2017 · 7 comments

Comments

Projects
None yet
6 participants
@jessehersch

jessehersch commented Apr 19, 2017

Symptom: cannot use the jsonschema lib in a python project targeting 3.6.0. pants tries to bring in functools32 which raises an error on python versions greater than 2.7.

The reason it's bringing in this dependency appears to be this line in jsonschema's setup.py: https://github.com/Julian/jsonschema/blob/master/setup.py#L25

This dependency should not be brought in since the project is targeting 3.6.0.

I created a simple repro for this here: pants_bug.zip

To repro:

  1. extract the zip above.
  2. cd pants_bug
  3. note the pants.ini specifies interpreter_requirement: CPython==3.6.0
  4. attempt to run tests: ./pants test .:

Output:

09:15:45 00:00 [main]
               (To run a reporting server: ./pants server)
09:15:45 00:00   [setup]
09:15:45 00:00     [parse]fatal: Not a git repository (or any of the parent directories): .git

               Executing tasks in goals: bootstrap -> imports -> unpack-jars -> deferred-sources -> gen -> jvm-platform-validate -> resolve -> compile -> resources -> test
09:15:45 00:00   [bootstrap]
09:15:45 00:00     [substitute-aliased-targets]
09:15:45 00:00     [jar-dependency-management]
09:15:45 00:00     [bootstrap-jvm-tools]
09:15:45 00:00     [provide-tools-jar]
09:15:45 00:00   [imports]
09:15:45 00:00     [ivy-imports]
09:15:45 00:00   [unpack-jars]
09:15:45 00:00     [unpack-jars]
09:15:45 00:00   [deferred-sources]
09:15:45 00:00     [deferred-sources]
09:15:45 00:00   [gen]
09:15:45 00:00     [antlr-java]
09:15:45 00:00     [antlr-py]
09:15:45 00:00     [jaxb]
09:15:45 00:00     [protoc]
09:15:45 00:00     [ragel]
09:15:45 00:00     [thrift-java]
09:15:45 00:00     [thrift-py]
09:15:45 00:00     [wire]
09:15:45 00:00   [jvm-platform-validate]
09:15:45 00:00     [jvm-platform-validate]
09:15:45 00:00   [resolve]
09:15:45 00:00     [ivy]
09:15:45 00:00   [compile]
09:15:45 00:00     [compile-jvm-prep-command]
09:15:45 00:00       [jvm_prep_command]
09:15:45 00:00     [scalafmt]
09:15:45 00:00     [compile-prep-command]
09:15:45 00:00     [compile]
09:15:45 00:00     [zinc]
09:15:45 00:00     [jvm-dep-check]
09:15:45 00:00   [resources]
09:15:45 00:00     [prepare]
09:15:45 00:00     [services]
09:15:45 00:00   [test]
09:15:45 00:00     [test-jvm-prep-command]
09:15:45 00:00       [jvm_prep_command]
09:15:45 00:00     [test-prep-command]
09:15:45 00:00     [test]
09:15:45 00:00     [pytest]
09:15:45 00:00       [run]
                     
09:15:45 00:00         [chroot]**** Failed to install functools32-3.2.3-2 (caused by: NonZeroExit("received exit code 1 during execution of `[u'/home/jhersch/.virtualenvs/gemini/bin/python3.6', '-', 'bdist_wheel', '--dist-dir=/tmp/tmpomPs2n']` while trying to execute `[u'/home/jhersch/.virtualenvs/gemini/bin/python3.6', '-', 'bdist_wheel', '--dist-dir=/tmp/tmpomPs2n']`",)
):
stdout:

stderr:
This backport is for Python 2.7 only.



09:15:46 00:01   [complete]
               FAILURE
Exception caught: (<class 'pex.resolver.Untranslateable'>)

Exception message: Package SourcePackage(u'file:///home/jhersch/pants_bug/.pants.d/python-setup/resolved_requirements/CPython-3.6.0/functools32-3.2.3-2.zip') is not translateable by ChainedTranslator(WheelTranslator, EggTranslator, SourceTranslator)
@kwlzn

This comment has been minimized.

Show comment
Hide comment
@kwlzn

kwlzn Apr 20, 2017

Member

fwiw, pex seems to handle the extras resolve just fine on 2.7 vs 3.6:

2.7:

[omerta ~]$ pex --disable-cache jsonschema -o /tmp/throwaway.pex -vvvvvvvvvvvvv 2>&1 |grep "Translating" |grep -v "Building"
pex:     Translating /private/var/folders/bb/msvj_4_s741g93nzgrj19twc0000gn/T/tmpnjcWsf/jsonschema-2.6.0-py2.py3-none-any.whl into distribution: 2.7ms
pex:     Translating /private/var/folders/bb/msvj_4_s741g93nzgrj19twc0000gn/T/tmpzeyOVs/functools32-3.2.3-2.zip into distribution: 312.6ms
[omerta ~]$ /tmp/throwaway.pex
Python 2.7.13 (default, Mar  2 2017, 16:22:01)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import jsonschema
>>>

3.6:

[omerta ~]$ pex --disable-cache jsonschema -o /tmp/throwaway.pex -vvvvvvvvvvvvv 2>&1 |grep "Translating" |grep -v "Building"
pex:     Translating /private/var/folders/bb/msvj_4_s741g93nzgrj19twc0000gn/T/tmp0t56idwp/jsonschema-2.6.0-py2.py3-none-any.whl into distribution: 3.1ms
[omerta ~]$ /tmp/throwaway.pex
Python 3.6.0 (default, Feb 13 2017, 11:02:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import jsonschema
>>>

I think this points to some form of an interpreter mismatch during resolve.

Member

kwlzn commented Apr 20, 2017

fwiw, pex seems to handle the extras resolve just fine on 2.7 vs 3.6:

2.7:

[omerta ~]$ pex --disable-cache jsonschema -o /tmp/throwaway.pex -vvvvvvvvvvvvv 2>&1 |grep "Translating" |grep -v "Building"
pex:     Translating /private/var/folders/bb/msvj_4_s741g93nzgrj19twc0000gn/T/tmpnjcWsf/jsonschema-2.6.0-py2.py3-none-any.whl into distribution: 2.7ms
pex:     Translating /private/var/folders/bb/msvj_4_s741g93nzgrj19twc0000gn/T/tmpzeyOVs/functools32-3.2.3-2.zip into distribution: 312.6ms
[omerta ~]$ /tmp/throwaway.pex
Python 2.7.13 (default, Mar  2 2017, 16:22:01)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import jsonschema
>>>

3.6:

[omerta ~]$ pex --disable-cache jsonschema -o /tmp/throwaway.pex -vvvvvvvvvvvvv 2>&1 |grep "Translating" |grep -v "Building"
pex:     Translating /private/var/folders/bb/msvj_4_s741g93nzgrj19twc0000gn/T/tmp0t56idwp/jsonschema-2.6.0-py2.py3-none-any.whl into distribution: 3.1ms
[omerta ~]$ /tmp/throwaway.pex
Python 3.6.0 (default, Feb 13 2017, 11:02:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import jsonschema
>>>

I think this points to some form of an interpreter mismatch during resolve.

@kwlzn kwlzn added bug python labels Apr 20, 2017

@jessehersch

This comment has been minimized.

Show comment
Hide comment
@jessehersch

jessehersch Jun 27, 2017

We are having to use an old json schema library due to this which is no longer maintained. Any chance this could get some love?

jessehersch commented Jun 27, 2017

We are having to use an old json schema library due to this which is no longer maintained. Any chance this could get some love?

@lexn82

This comment has been minimized.

Show comment
Hide comment
@lexn82

lexn82 Apr 13, 2018

Hit the same issue with pants 1.5.0 trying to use plotly==2.5.1 which requires jsonschema.

lexn82 commented Apr 13, 2018

Hit the same issue with pants 1.5.0 trying to use plotly==2.5.1 which requires jsonschema.

@bjorn-spire

This comment has been minimized.

Show comment
Hide comment
@bjorn-spire

bjorn-spire May 17, 2018

same here with pants 1.7.0rc0 trying to use plotly==2.6.0

bjorn-spire commented May 17, 2018

same here with pants 1.7.0rc0 trying to use plotly==2.6.0

@bjorn-spire

This comment has been minimized.

Show comment
Hide comment
@bjorn-spire

bjorn-spire May 17, 2018

I ended up working around this by:

  1. Create a local build target with provides=py_setup named functools32
  2. Building it with ./pants setup-py <my target>
  3. Converting it to a python3 wheel by pip3 wheel dist/functools32-<version>tar.gz
  4. Uploading it to a private python "find links repo"

It works by the silly fact that there's something for it to download, and then it just places a silly file called functools32_hack.py explaining why it's there.

So if you get stuck on this then there's always this horrid hack until one of us have time to look into fixing it all proper like. :)

bjorn-spire commented May 17, 2018

I ended up working around this by:

  1. Create a local build target with provides=py_setup named functools32
  2. Building it with ./pants setup-py <my target>
  3. Converting it to a python3 wheel by pip3 wheel dist/functools32-<version>tar.gz
  4. Uploading it to a private python "find links repo"

It works by the silly fact that there's something for it to download, and then it just places a silly file called functools32_hack.py explaining why it's there.

So if you get stuck on this then there's always this horrid hack until one of us have time to look into fixing it all proper like. :)

@shuoli84

This comment has been minimized.

Show comment
Hide comment
@shuoli84

shuoli84 Jul 3, 2018

Contributor

Should be fixed by #5906
Verified on 1.9.0.dev0.

Contributor

shuoli84 commented Jul 3, 2018

Should be fixed by #5906
Verified on 1.9.0.dev0.

@stuhood

This comment has been minimized.

Show comment
Hide comment
@stuhood

stuhood Jul 3, 2018

Member

Thanks @shuoli84!

Member

stuhood commented Jul 3, 2018

Thanks @shuoli84!

@stuhood stuhood closed this Jul 3, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment