Nix recipe for FiPy #585

wd15 · 2018-07-29T01:19:51Z

Nix recipe for Python 2 and 3. Does not include Trilinos.

@guyer, can you test this on a mac

To test use nix-shell --pure in the base directory. Please read the README.md to get started.

The pip install section should only be used by those that want a development environment for FiPy.

tkphd · 2018-07-31T15:43:32Z

@wd15,

nix installed (single-user mode on my personal machine, running Arch) to /nix using
```
curl https://nixos.org/nix/install | sh
```
nix-shell --pure appears to complete successfully, though it warns that bash: hostname: command not found. This seems to be a conflict with systemd?

python setup.py test fails one test:

======================================================================
FAIL: itemset (fipy.tools.dimensions.physicalField.PhysicalField)
Doctest: fipy.tools.dimensions.physicalField.PhysicalField.itemset
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for fipy.tools.dimensions.physicalField.PhysicalField.itemset
  File "fipy/tools/dimensions/physicalField.py", line 589, in itemset

----------------------------------------------------------------------
File "fipy/tools/dimensions/physicalField.py", line 599, in 
fipy.tools.dimensions.physicalField.PhysicalField.itemset
Failed example:
    try:
        a.itemset(PhysicalField("6 ft"))
    except IndexError:
        # NumPy 1.7 has changed the exception type
        raise ValueError("can only place a scalar for an  array of size 1")
Expected:
    Traceback (most recent call last):
        ...
    ValueError: can only place a scalar for an  array of size 1
Got:
    Traceback (most recent call last):
      File "/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest fipy.tools.dimensions.physicalField.PhysicalField.itemset[4]>", line 2, in <module>
        a.itemset(PhysicalField("6 ft"))
      File "fipy/tools/dimensions/physicalField.py", line 616, in itemset
        self.value.itemset(value)
    ValueError: can only convert an array of size 1 to a Python scalar


----------------------------------------------------------------------
Ran 545 tests in 72.738s

FAILED (failures=1)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Skipped 2 doctest examples because `lsmlib` must be used to run some tests
Skipped 83 doctest examples because the `tvtk` package cannot be imported
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Is the goal of this exercise to migrate away from pip and conda? How portable is this workflow to multi-user resources (e.g. workstations and cluster nodes)?
How maintainable is a nix port of FiPy?

wd15 · 2018-07-31T17:28:35Z

On Tue, Jul 31, 2018 at 11:44 AM, Trevor Keller ***@***.***> wrote: @wd15, nix installed (single-user mode on my personal machine, running Arch) to /nix using curl https://nixos.org/nix/install | sh nix-shell --pure appears to complete successfully, though it warns that bash: hostname: command not found. This seems to be a conflict with systemd?

I think I get the same warning, but it doesn't seem to matter.

python setup.py test fails one test:

Likewise for me. That might just be the numpy version.

Is the goal of this exercise to migrate away from pip and conda?

No. It's just to have an alternative available.

How portable is this workflow to multi-user resources (e.g. workstations and cluster nodes)?

Very portable. That's the entire point of using something like Nix. It's a way to guarantee that you have consistent software across machines for distributed computing. The machines do need to have Nix installed. There is a also Nixops, https://nixos.org/nixops/, which I haven't tried.

How maintainable is a nix port of FiPy?

Nix itself is fairly stable. The Nix build of FiPy could of course break if it isn't tested if FiPy changes in some way such that the dependencies outlined in the Nix files are no longer kept up to date. The dependencies are fixed at a particular version of Nixpkgs. To make it less likely to break then it needs to be added to a CI. What would be a good option for that? We could make the Nix recipe depend on a particular version of FiPy, but the point of the recipe is really as a development tool at the moment.

…

-- Daniel Wheeler

Uncomment pip installation and fix Python 3 installation

python_version is no longer required in the Nix recipe

Add comments explaining how to build pyamg solver library and Sphinx tools for building the documentation. Sphinx could be nixified, but pyamg can't be nixified as it's using $HOME as part of the build process (in rare cases Python packages do this, but shouldn't).

guyer · 2018-09-05T20:46:18Z

@wd15: Fails on on Mac OS

clang -bundle -undefined dynamic_lookup build/temp.macosx-10.10-x86_64-2.7/pysparse/eigen/src/jdsymmodule.o -L/usr/lib -L/usr/lib -L/nix/store/4llw0i5jbb7p5n0r58w9z0azqgx5wk09-python-2.7.14/lib -lblas -lblas -llapack -llapack -o build/lib.macosx-10.10-x86_64-2.7/pysparse/eigen/jdsym.so
ld: library not found for -lblas
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
ld: library not found for -lblas
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
error: Command "clang -bundle -undefined dynamic_lookup build/temp.macosx-10.10-x86_64-2.7/pysparse/eigen/src/jdsymmodule.o -L/usr/lib -L/usr/lib -L/nix/store/4llw0i5jbb7p5n0r58w9z0azqgx5wk09-python-2.7.14/lib -lblas -lblas -llapack -llapack -o build/lib.macosx-10.10-x86_64-2.7/pysparse/eigen/jdsym.so" failed with exit status 1
builder for '/nix/store/y86g8gr18ajb2frcis2dv474jaa1dpi3-python2.7-pysparse-1.1.1-dev.drv' failed with exit code 1
error: build of '/nix/store/2ykvqyvfp41rc1hq2vk4ys66c4msyl07-python2.7-scikit-fmm-0.0.9.drv', '/nix/store/ap25cv9l4gig0sw0vxh3a4vbh967n90w-gmsh-3.0.6.drv', '/nix/store/hmzya72gq1j92fj60zn2labci3ww2mlb-python2.7-tkinter-2.7.14.drv', '/nix/store/p0kriliyjc5cgygyr4j2r1gx42qw1v17-fipy.drv', '/nix/store/y86g8gr18ajb2frcis2dv474jaa1dpi3-python2.7-pysparse-1.1.1-dev.drv', '/nix/store/z7qq3w55djgbf7cilns9wkx6kfs204qx-python2.7-scipy-1.0.0.drv' failed

wd15 · 2018-09-06T14:15:07Z

Would this recent change in Nixpkgs have anything to do with the above error? If you, @guyer, think that might be related then I'll make sure that the recipes build with a recent version of Nixpkgs and then let you have another go.

guyer · 2018-09-06T14:48:02Z

@wd15, I think it's worth a shot

Update Nixpkgs to version 18.09-beta to possibly help with problems building on Mac.

wd15 · 2018-09-07T14:28:49Z

@guyer, I updated the nixpkgs_version.nix to use 18.09-beta. It all built fine for me and fairly quickly. The only things that were explicitly built were

these derivations will be built:
  /nix/store/4kgq383z9dv02fvlcvrdf2fk558090ji-gmsh-3.0.6.drv
  /nix/store/jnr34iyxs71ii5r3j902w037h2k87fp1-python2.7-pysparse-1.1.1-dev.drv
  /nix/store/qcagdaryv2wc3lfjw9nav1v0nb8g109j-python2.7-scikit-fmm-0.0.9.drv
  /nix/store/l056r9m7h8rlg78qmg9wlqhm161r2my1-fipy.drv

Everything else was a downloaded binary.

This is from 5 days ago so I'm assuming it has the change since that change is labeled with the 18.09-beta branch.

guyer · 2018-09-07T18:57:37Z

@wd15: no change

Everything works other than Gmsh. Gmsh is failing to install, but compiles and links.

When building Gmsh with Nix only use -Wno-error on the x86_64-darwin platform.

wd15 · 2018-10-19T19:11:37Z

@guyer, @tkphd, please try running nix-shell --pure --command "python setup.py test" again. Works for me on both linux and x86_64-darwin.

guyer

works for me

tkphd

I get a bunch of warnings about command-line options being valid for C/C++/ObjC/ObjC++ but not for Fortran. Other than that verbose compiler output, this works for me, too.

wd15 · 2018-10-22T15:50:33Z

@guyer, feel free to merge this now.

guyer · 2018-10-22T18:59:26Z

@wd15, actually, could you add a bit of documentation on what to do with it in INSTALLATION.rst. It might not need to be much more than a link to nix/README.md.

Add a link to the Nix instructions to the installation guide. - Remove white space from INSTALLATION.rst - Add complete recipe for building documentation in nix/build.nix

wd15 · 2018-10-23T15:21:12Z

@guyer, over to you

guyer

Please actually integrate the nix README into the sphinx documentation

wd15 · 2018-10-24T22:21:25Z

@guyer, over to you again

Everything else makes a hash out of the table of contents

wd15 added 6 commits July 26, 2018 17:16

Implement Nix build in Python 2 and 3

dfa30ab

Add Readme and pip installation capability.

056836f

Improve Nix docs and enable pip install

c2e7c81

Comment out pip install section of Nix build

d5a5cb8

The pip install section should only be used by those that want a development environment for FiPy.

Fix broken link in nix/README.md

abbf667

Fix broken link in nix/README.md

4641784

wd15 requested review from guyer and tkphd July 29, 2018 01:20

wd15 assigned guyer and tkphd Jul 29, 2018

tkphd assigned wd15 and unassigned tkphd Jul 31, 2018

wd15 added 4 commits August 6, 2018 11:01

Uncomment pip installation

b5d03ed

Uncomment pip installation and fix Python 3 installation

Merge remote-tracking branch 'origin/nix' into nix

3f9a7f7

Remove useless expression from Nix recipe

c12682e

python_version is no longer required in the Nix recipe

Update Nixpkgs version to 18.09-beta

c82d781

Update Nixpkgs to version 18.09-beta to possibly help with problems building on Mac.

wd15 added 4 commits October 19, 2018 11:08

Make Nix recipe work on OSX

99b149b

Everything works other than Gmsh. Gmsh is failing to install, but compiles and links.

Add Jupyter to Nix recipe

dcd9c93

Merge remote-tracking branch 'origin/nix' into nix

f841e4e

Only use -Wno-error on x86_64-darwin platform for Gmsh

26c33b7

When building Gmsh with Nix only use -Wno-error on the x86_64-darwin platform.

guyer approved these changes Oct 20, 2018

View reviewed changes

guyer removed their assignment Oct 20, 2018

tkphd approved these changes Oct 21, 2018

View reviewed changes

wd15 removed their assignment Oct 22, 2018

wd15 added 2 commits October 23, 2018 11:10

Add link to nix/README.md in INSTALLATION.rst

bd79134

Add a link to the Nix instructions to the installation guide. - Remove white space from INSTALLATION.rst - Add complete recipe for building documentation in nix/build.nix

Merge branch 'develop' into nix

8dc4648

guyer requested changes Oct 23, 2018

View reviewed changes

Switch Nix guide to rst

354d553

guyer added 2 commits October 25, 2018 11:40

Fix markup

bd58460

Integrate Nix instructions into INSTALLATION.rst as include

c47d8f9

Everything else makes a hash out of the table of contents

guyer approved these changes Oct 25, 2018

View reviewed changes

guyer merged commit 11ac915 into usnistgov:develop Oct 25, 2018

wd15 deleted the nix branch October 30, 2018 14:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nix recipe for FiPy #585

Nix recipe for FiPy #585

wd15 commented Jul 29, 2018 •

edited

Loading

tkphd commented Jul 31, 2018

wd15 commented Jul 31, 2018 via email

guyer commented Sep 5, 2018

wd15 commented Sep 6, 2018

guyer commented Sep 6, 2018

wd15 commented Sep 7, 2018 •

edited

Loading

guyer commented Sep 7, 2018

wd15 commented Oct 19, 2018 •

edited

Loading

guyer left a comment

tkphd left a comment

wd15 commented Oct 22, 2018

guyer commented Oct 22, 2018

wd15 commented Oct 23, 2018

guyer left a comment

wd15 commented Oct 24, 2018

Nix recipe for FiPy #585

Nix recipe for FiPy #585

Conversation

wd15 commented Jul 29, 2018 • edited Loading

tkphd commented Jul 31, 2018

wd15 commented Jul 31, 2018 via email

guyer commented Sep 5, 2018

wd15 commented Sep 6, 2018

guyer commented Sep 6, 2018

wd15 commented Sep 7, 2018 • edited Loading

guyer commented Sep 7, 2018

wd15 commented Oct 19, 2018 • edited Loading

guyer left a comment

Choose a reason for hiding this comment

tkphd left a comment

Choose a reason for hiding this comment

wd15 commented Oct 22, 2018

guyer commented Oct 22, 2018

wd15 commented Oct 23, 2018

guyer left a comment

Choose a reason for hiding this comment

wd15 commented Oct 24, 2018

wd15 commented Jul 29, 2018 •

edited

Loading

wd15 commented Sep 7, 2018 •

edited

Loading

wd15 commented Oct 19, 2018 •

edited

Loading