Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] pythonPackages.pysmt: init at 0.8.0 #68422

Closed
wants to merge 1 commit into from

Conversation

@Pamplemousse
Copy link
Member

@Pamplemousse Pamplemousse commented Sep 10, 2019

Motivation for this change

I am trying to make angr, the binary analysis framework, available on NixOS.
This is part of the modules it requires.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.
@Pamplemousse Pamplemousse force-pushed the Pamplemousse:pysmt branch from 88df1a5 to d0b3389 Sep 10, 2019
@Pamplemousse
Copy link
Member Author

@Pamplemousse Pamplemousse commented Sep 10, 2019

Mmmh, don't know where the comment about slow tests came from...
There just seem to be broken using PyPI: pysmt/pysmt#608 .

Hence, I fetch the package from GitHub, and the tests run fine.

@jonringer
Copy link
Contributor

@jonringer jonringer commented Sep 10, 2019

They most likely are filtering those in their MANIFEST, I would just checkout src from their github, it will most certainly include the configs

@jonringer
Copy link
Contributor

@jonringer jonringer commented Sep 10, 2019

However, the ticket might still be relevant for them to decide to include tests at all if they're going to remove one of the dependencies like that

@Pamplemousse
Copy link
Member Author

@Pamplemousse Pamplemousse commented Sep 10, 2019

@jonringer , I was unclear:

  • I changed the nix expression to use fetchFromGitHub;
  • I opened the issue for the reasons you highlighted.
@jonringer
Copy link
Contributor

@jonringer jonringer commented Sep 10, 2019

I was referring to how they packaged the pypi tarball, they kept in the tests directory, but left out the tests/config directory

@Pamplemousse
Copy link
Member Author

@Pamplemousse Pamplemousse commented Sep 10, 2019

@jonringer yes yes, but you said:

I would just checkout src from their github [...]

While that was already what the code was updated to do.

@jonringer
Copy link
Contributor

@jonringer jonringer commented Sep 10, 2019

oh, i was probably looking at the first commit still

@Pamplemousse Pamplemousse force-pushed the Pamplemousse:pysmt branch from d0b3389 to 88b27a2 Sep 10, 2019
@lheckemann lheckemann added this to the 20.03 milestone Sep 12, 2019
@Pamplemousse Pamplemousse force-pushed the Pamplemousse:pysmt branch from 88b27a2 to 0f92636 Sep 12, 2019
@Pamplemousse Pamplemousse force-pushed the Pamplemousse:pysmt branch from 0f92636 to abf6ed4 Sep 19, 2019
@Pamplemousse Pamplemousse changed the title pythonPackages.pysmt: init at 0.8.0 [WIP] pythonPackages.pysmt: init at 0.8.0 Sep 19, 2019
@Pamplemousse
Copy link
Member Author

@Pamplemousse Pamplemousse commented Sep 19, 2019

There is something strange happening:

  • nix-build -A pythonPackages.PySMT builds fine
  • nix-build -A python3Packages.PySMT triggers an error
See stack trace
these derivations will be built:
  /nix/store/35vhwb3c4zyvrgq0annx64l3ibi9r78k-python3.7-PySMT-unstable-2019-09-03.drv
building '/nix/store/35vhwb3c4zyvrgq0annx64l3ibi9r78k-python3.7-PySMT-unstable-2019-09-03.drv'...
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing setuptools-check-hook
Using setuptoolsCheckPhase
unpacking sources
unpacking source archive /nix/store/wckwlj4vnvlp03gg43rb38j917jvfaxr-source
source root is source
setting SOURCE_DATE_EPOCH to timestamp 315619200 of file source/shell.py
patching sources
configuring
no configure script, doing nothing
building
Executing setuptoolsBuildPhase
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/pysmt
copying [...]
running egg_info
creating PySMT.egg-info
writing PySMT.egg-info/PKG-INFO
writing dependency_links to PySMT.egg-info/dependency_links.txt
writing entry points to PySMT.egg-info/entry_points.txt
writing requirements to PySMT.egg-info/requires.txt
writing top-level names to PySMT.egg-info/top_level.txt
writing manifest file 'PySMT.egg-info/SOURCES.txt'
reading manifest file 'PySMT.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'PySMT.egg-info/SOURCES.txt'
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/pysmt
copying [...]
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
adding license file "NOTICE" (matched pattern "NOTICE*")
creating build/bdist.linux-x86_64/wheel/PySMT-0.8.1.dev1.dist-info/WHEEL
creating 'dist/PySMT-0.8.1.dev1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding [...]
removing build/bdist.linux-x86_64/wheel
Finished executing setuptoolsInstallPhase
installing
Executing pipInstallPhase
/build/source/dist /build/source
Processing ./PySMT-0.8.1.dev1-py3-none-any.whl
Requirement already satisfied: six in /nix/store/zljrawcjp2r88lvmpy70pqxiyadp64b5-python3.7-six-1.12.0/lib/python3.7/site-packages (from PySMT==0.8.1.dev1) (1.12.0)
Installing collected packages: PySMT
  WARNING: The script pysmt-install is installed in '/nix/store/q9s32sfpdzg4pgqf5xnvgk846nicmnc3-python3.7-PySMT-unstable-2019-09-03/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed PySMT-0.8.1.dev1
/build/source
Finished executing pipInstallPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/q9s32sfpdzg4pgqf5xnvgk846nicmnc3-python3.7-PySMT-unstable-2019-09-03
strip is /nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/q9s32sfpdzg4pgqf5xnvgk846nicmnc3-python3.7-PySMT-unstable-2019-09-03/lib  /nix/store/q9s32sfpdzg4pgqf5xnvgk846nicmnc3-python3.7-PySMT-unstable-2019-09-03/bin
patching script interpreter paths in /nix/store/q9s32sfpdzg4pgqf5xnvgk846nicmnc3-python3.7-PySMT-unstable-2019-09-03
checking for references to /build/ in /nix/store/q9s32sfpdzg4pgqf5xnvgk846nicmnc3-python3.7-PySMT-unstable-2019-09-03...
Rewriting #!/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/bin/python3.7 to #!/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4
wrapping `/nix/store/q9s32sfpdzg4pgqf5xnvgk846nicmnc3-python3.7-PySMT-unstable-2019-09-03/bin/pysmt-install'...
running install tests
no Makefile or custom buildPhase, doing nothing
pythonCatchConflictsPhase
pythonRemoveBinBytecodePhase
pythonImportsCheckPhase
Executing pythonImportsCheckPhase
setuptoolsCheckPhase
Executing setuptoolsCheckPhase
running test
running egg_info
writing PySMT.egg-info/PKG-INFO
writing dependency_links to PySMT.egg-info/dependency_links.txt
writing entry points to PySMT.egg-info/entry_points.txt
writing requirements to PySMT.egg-info/requires.txt
writing top-level names to PySMT.egg-info/top_level.txt
reading manifest file 'PySMT.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'PySMT.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "/build/source/pysmt/solvers/bdd.py", line 23, in <module>
    import repycudd
ModuleNotFoundError: No module named 'repycudd'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "nix_run_setup", line 8, in <module>
    exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))
  File "setup.py", line 65, in <module>
    'pysmt-install = pysmt.cmd.install:main',
  File "/nix/store/6wihp44ygcykcx1bjc0kly3d4lzx5qj1-python3.7-setuptools-41.2.0/lib/python3.7/site-packages/setuptools-41.2.0-py3.7.egg/setuptools/__init__.py", line 145, in setup
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/nix/store/6wihp44ygcykcx1bjc0kly3d4lzx5qj1-python3.7-setuptools-41.2.0/lib/python3.7/site-packages/setuptools-41.2.0-py3.7.egg/setuptools/command/test.py", line 229, in run
  File "/nix/store/6wihp44ygcykcx1bjc0kly3d4lzx5qj1-python3.7-setuptools-41.2.0/lib/python3.7/site-packages/setuptools-41.2.0-py3.7.egg/setuptools/command/test.py", line 251, in run_tests
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/unittest/main.py", line 100, in __init__
    self.parseArgs(argv)
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/unittest/main.py", line 124, in parseArgs
    self._do_discovery(argv[2:])
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/unittest/main.py", line 244, in _do_discovery
    self.createTests(from_discovery=True, Loader=Loader)
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/unittest/main.py", line 154, in createTests
    self.test = loader.discover(self.start, self.pattern, self.top)
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/unittest/loader.py", line 349, in discover
    tests = list(self._find_tests(start_dir, pattern))
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/unittest/loader.py", line 406, in _find_tests
    full_path, pattern, namespace)
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/unittest/loader.py", line 483, in _find_test_path
    tests = self.loadTestsFromModule(package, pattern=pattern)
  File "/nix/store/6wihp44ygcykcx1bjc0kly3d4lzx5qj1-python3.7-setuptools-41.2.0/lib/python3.7/site-packages/setuptools-41.2.0-py3.7.egg/setuptools/command/test.py", line 55, in loadTestsFromModule
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/unittest/loader.py", line 191, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "/nix/store/6wihp44ygcykcx1bjc0kly3d4lzx5qj1-python3.7-setuptools-41.2.0/lib/python3.7/site-packages/setuptools-41.2.0-py3.7.egg/setuptools/command/test.py", line 55, in loadTestsFromModule
  File "/nix/store/k0igqpz995nmxwf6piyd5jjpcc7gcnyc-python3-3.7.4/lib/python3.7/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/build/source/pysmt/solvers/bdd.py", line 25, in <module>
    raise SolverAPINotFound
pysmt.exceptions.SolverAPINotFound
builder for '/nix/store/35vhwb3c4zyvrgq0annx64l3ibi9r78k-python3.7-PySMT-unstable-2019-09-03.drv' failed with exit code 1
error: build of '/nix/store/35vhwb3c4zyvrgq0annx64l3ibi9r78k-python3.7-PySMT-unstable-2019-09-03.drv' failed
@jonringer
Copy link
Contributor

@jonringer jonringer commented Sep 19, 2019

ModuleNotFoundError: No module named 'repycudd' according to https://libraries.io/github/pysmt/pysmt they have their own solver.

@Pamplemousse
Copy link
Member Author

@Pamplemousse Pamplemousse commented Sep 19, 2019

@jonringer , yes, I figured that out.

However, it feels strange to me that it only shows up when building the python3Package...

@jonringer
Copy link
Contributor

@jonringer jonringer commented Sep 20, 2019

Agreed that it's weird

@Pamplemousse
Copy link
Member Author

@Pamplemousse Pamplemousse commented Dec 6, 2019

I started https://github.com/angr/nixpkgs , and it should be added to NUR following nix-community/NUR#160.
Closed as an angr related package.

@Pamplemousse Pamplemousse deleted the Pamplemousse:pysmt branch Dec 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.