This repository has been archived by the owner. It is now read-only.

parallel-netcdf 1.7.0 (new formula) #3466

Closed
wants to merge 6 commits into
from

Conversation

Projects
None yet
3 participants
@mkrapp
Contributor

mkrapp commented Mar 24, 2016

Here's a new formula for parallel-netcdf, which works with and without MPI support.

@mkrapp mkrapp changed the title from New formula: parallel-netcdf 1.7.0 to parallel-netcdf 1.7.0 (new formula) Mar 24, 2016

parallel-netcdf.rb
+ url "http://cucis.ece.northwestern.edu/projects/PnetCDF/Release/parallel-netcdf-1.7.0.tar.gz"
+ sha256 "52f0d106c470a843c6176318141f74a21e6ece3f70ee8fe261c6b93e35f70a94"
+
+ deprecated_option "without-check" => "without-test"

This comment has been minimized.

@dpo

dpo Mar 25, 2016

Contributor

You don't need this for new formulas.

@dpo

dpo Mar 25, 2016

Contributor

You don't need this for new formulas.

parallel-netcdf.rb
+ option "without-test", "Disable checks (not recommended)"
+ option "with-mpi", "Enable parallel compiling"
+
+ depends_on :mpi => [:optional, :cc, :cxx, :f90]

This comment has been minimized.

@dpo

dpo Mar 25, 2016

Contributor

Line 12 is redundant with this. You only need line 14.

@dpo

dpo Mar 25, 2016

Contributor

Line 12 is redundant with this. You only need line 14.

parallel-netcdf.rb
+ deprecated_option "without-check" => "without-test"
+
+ option "without-cxx", "Don't compile C++ bindings"
+ option "without-fortran", "Don't compile Fortran bindings"

This comment has been minimized.

@dpo

dpo Mar 25, 2016

Contributor

Here you need depends_on :fortran => :optional instead.

@dpo

dpo Mar 25, 2016

Contributor

Here you need depends_on :fortran => :optional instead.

parallel-netcdf.rb
+
+ args = common_args.clone
+ if build.with? "mpi"
+ args << "--enable-mpi"

This comment has been minimized.

@dpo

dpo Mar 25, 2016

Contributor

What's the point of having both args and common_args?

@dpo

dpo Mar 25, 2016

Contributor

What's the point of having both args and common_args?

parallel-netcdf.rb
+
+ v2 = 0, 1, 2, 3 ;
+ }
+ */

This comment has been minimized.

@dpo

dpo Mar 25, 2016

Contributor

Wondering if all those comments are necessary for the test. Isn't this file found elsewhere?

@dpo

dpo Mar 25, 2016

Contributor

Wondering if all those comments are necessary for the test. Isn't this file found elsewhere?

parallel-netcdf.rb
+ if build.with? "mpi"
+ system "mpicc", "test_mpi.c", "-L#{lib}", "-I#{include}", "-lpnetcdf", "-o", "test"
+ system "mpiexec", "-n", "4", "./test", "output.nc"
+ assert File.exist?("output.nc")

This comment has been minimized.

@dpo

dpo Mar 25, 2016

Contributor

Great test! Thanks! Don't you want to check the contents of output.nc though?

@dpo

dpo Mar 25, 2016

Contributor

Great test! Thanks! Don't you want to check the contents of output.nc though?

@dpo dpo added the needs response label Mar 30, 2016

@dpo

This comment has been minimized.

Show comment
Hide comment
@dpo

dpo Jun 11, 2016

Contributor

Will this be updated?

Contributor

dpo commented Jun 11, 2016

Will this be updated?

@mkrapp

This comment has been minimized.

Show comment
Hide comment
@mkrapp

mkrapp Jun 12, 2016

Contributor

Sorry. I forgot about this. I can work on it next week.
On Jun 12, 2016 12:01 AM, "Dominique" notifications@github.com wrote:

Will this be updated?


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#3466 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AFqcVpmRl4TQR5MvTAJWKxnN2-KrV5foks5qKzBTgaJpZM4H32n9
.

Contributor

mkrapp commented Jun 12, 2016

Sorry. I forgot about this. I can work on it next week.
On Jun 12, 2016 12:01 AM, "Dominique" notifications@github.com wrote:

Will this be updated?


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#3466 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AFqcVpmRl4TQR5MvTAJWKxnN2-KrV5foks5qKzBTgaJpZM4H32n9
.

@ghost ghost removed the needs response label Jun 12, 2016

@dpo dpo added the needs response label Aug 21, 2016

@ghost ghost removed the needs response label Sep 5, 2016

New formula: parallel-netcdf 1.7.0
fix MPI issues

fix audit issues and configure

resolved comments, cleaned-up
@jonchang

This comment has been minimized.

Show comment
Hide comment
@jonchang

jonchang Sep 5, 2016

Contributor

Hi @mkrapp, it looks like your build failed in make check:

/usr/local/bin/mpicc  -g -O2   -o tst_nofill tst_nofill.o -L../common  /private/tmp/parallel-netcdf-20160905-8595-1gszs3u/parallel-netcdf-1.7.0/src/lib/libpnetcdf.a -ltestutils  -lm 
rm -f ./scratch.nc
rm -f ./testfile.nc
rm -f ./tooth-fairy.nc
./nc_test -c    -d .
./nc_test -d       .
opening a nonexistent file expects NC_ENOENT, but got NC_EFILE

    FAILURE at line 370 of test_write.c: file ./scratch.nc should not exist
    ### 1 FAILURES TESTING test_ncmpi_abort! Stop ... ###

./nc_test: expects 0 failures ... *** TESTING C   ./nc_test for format CDF-1                         ------ rank 0: MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
fail with 1 mismatches
make[2]: *** [testing] Error 1
make[1]: *** [check-nc_test] Error 2
make: *** [check] Error 2

Audit warnings:

==> brew audit homebrew/science/parallel-netcdf --new-formula
Error: 7 problems in 1 formula
homebrew/science/parallel-netcdf:
  * Use spaces instead of tabs for indentation
  * Use spaces instead of tabs for indentation
  * Use spaces instead of tabs for indentation
  * 127: Trailing whitespace was found
  * Use spaces instead of tabs for indentation
  * 132: Trailing whitespace was found
  * 134: Trailing whitespace was found
Contributor

jonchang commented Sep 5, 2016

Hi @mkrapp, it looks like your build failed in make check:

/usr/local/bin/mpicc  -g -O2   -o tst_nofill tst_nofill.o -L../common  /private/tmp/parallel-netcdf-20160905-8595-1gszs3u/parallel-netcdf-1.7.0/src/lib/libpnetcdf.a -ltestutils  -lm 
rm -f ./scratch.nc
rm -f ./testfile.nc
rm -f ./tooth-fairy.nc
./nc_test -c    -d .
./nc_test -d       .
opening a nonexistent file expects NC_ENOENT, but got NC_EFILE

    FAILURE at line 370 of test_write.c: file ./scratch.nc should not exist
    ### 1 FAILURES TESTING test_ncmpi_abort! Stop ... ###

./nc_test: expects 0 failures ... *** TESTING C   ./nc_test for format CDF-1                         ------ rank 0: MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
fail with 1 mismatches
make[2]: *** [testing] Error 1
make[1]: *** [check-nc_test] Error 2
make: *** [check] Error 2

Audit warnings:

==> brew audit homebrew/science/parallel-netcdf --new-formula
Error: 7 problems in 1 formula
homebrew/science/parallel-netcdf:
  * Use spaces instead of tabs for indentation
  * Use spaces instead of tabs for indentation
  * Use spaces instead of tabs for indentation
  * 127: Trailing whitespace was found
  * Use spaces instead of tabs for indentation
  * 132: Trailing whitespace was found
  * 134: Trailing whitespace was found

@jonchang jonchang self-assigned this Sep 5, 2016

+ url "http://cucis.ece.northwestern.edu/projects/PnetCDF/Release/parallel-netcdf-1.7.0.tar.gz"
+ sha256 "52f0d106c470a843c6176318141f74a21e6ece3f70ee8fe261c6b93e35f70a94"
+
+ option "without-cxx", "Don't compile C++ bindings"

This comment has been minimized.

@jonchang

jonchang Sep 5, 2016

Contributor

It doesn't look like you test for this option anywhere?

@jonchang

jonchang Sep 5, 2016

Contributor

It doesn't look like you test for this option anywhere?

This comment has been minimized.

@mkrapp

mkrapp Sep 5, 2016

Contributor

Will add an option to compile without c++ and fortran bindings.

@mkrapp

mkrapp Sep 5, 2016

Contributor

Will add an option to compile without c++ and fortran bindings.

@mkrapp

This comment has been minimized.

Show comment
Hide comment
@mkrapp

mkrapp Sep 5, 2016

Contributor

The check is successful when building with mpich but apparently the
default MPI isopen-mpi.

On Sep 5, 2016 6:08 PM, "Jonathan Chang" notifications@github.com wrote:

Hi @mkrapp https://github.com/mkrapp, it looks like your build failed
in make check:

/usr/local/bin/mpicc -g -O2 -o tst_nofill tst_nofill.o -L../common /private/tmp/parallel-netcdf-20160905-8595-1gszs3u/parallel-netcdf-1.7.0/src/lib/libpnetcdf.a -ltestutils -lm
rm -f ./scratch.nc
rm -f ./testfile.nc
rm -f ./tooth-fairy.nc
./nc_test -c -d .
./nc_test -d .
opening a nonexistent file expects NC_ENOENT, but got NC_EFILE

FAILURE at line 370 of test_write.c: file ./scratch.nc should not exist
### 1 FAILURES TESTING test_ncmpi_abort! Stop ... ###

./nc_test: expects 0 failures ... *** TESTING C ./nc_test for format CDF-1 ------ rank 0: MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
fail with 1 mismatches
make[2]: *** [testing] Error 1
make[1]: *** [check-nc_test] Error 2
make: *** [check] Error 2

Audit warnings:

==> brew audit homebrew/science/parallel-netcdf --new-formula
Error: 7 problems in 1 formula
homebrew/science/parallel-netcdf:

  • Use spaces instead of tabs for indentation
  • Use spaces instead of tabs for indentation
  • Use spaces instead of tabs for indentation
  • 127: Trailing whitespace was found
  • Use spaces instead of tabs for indentation
  • 132: Trailing whitespace was found
  • 134: Trailing whitespace was found


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#3466 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFqcVoRWe0fuoruh7qp4k-SPMnZdW3lhks5qnD5wgaJpZM4H32n9
.

Contributor

mkrapp commented Sep 5, 2016

The check is successful when building with mpich but apparently the
default MPI isopen-mpi.

On Sep 5, 2016 6:08 PM, "Jonathan Chang" notifications@github.com wrote:

Hi @mkrapp https://github.com/mkrapp, it looks like your build failed
in make check:

/usr/local/bin/mpicc -g -O2 -o tst_nofill tst_nofill.o -L../common /private/tmp/parallel-netcdf-20160905-8595-1gszs3u/parallel-netcdf-1.7.0/src/lib/libpnetcdf.a -ltestutils -lm
rm -f ./scratch.nc
rm -f ./testfile.nc
rm -f ./tooth-fairy.nc
./nc_test -c -d .
./nc_test -d .
opening a nonexistent file expects NC_ENOENT, but got NC_EFILE

FAILURE at line 370 of test_write.c: file ./scratch.nc should not exist
### 1 FAILURES TESTING test_ncmpi_abort! Stop ... ###

./nc_test: expects 0 failures ... *** TESTING C ./nc_test for format CDF-1 ------ rank 0: MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
fail with 1 mismatches
make[2]: *** [testing] Error 1
make[1]: *** [check-nc_test] Error 2
make: *** [check] Error 2

Audit warnings:

==> brew audit homebrew/science/parallel-netcdf --new-formula
Error: 7 problems in 1 formula
homebrew/science/parallel-netcdf:

  • Use spaces instead of tabs for indentation
  • Use spaces instead of tabs for indentation
  • Use spaces instead of tabs for indentation
  • 127: Trailing whitespace was found
  • Use spaces instead of tabs for indentation
  • 132: Trailing whitespace was found
  • 134: Trailing whitespace was found


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#3466 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFqcVoRWe0fuoruh7qp4k-SPMnZdW3lhks5qnD5wgaJpZM4H32n9
.

@jonchang

This comment has been minimized.

Show comment
Hide comment
@jonchang

jonchang Sep 5, 2016

Contributor

Does the library work with open-mpi or is it only compatible with mpich?

Contributor

jonchang commented Sep 5, 2016

Does the library work with open-mpi or is it only compatible with mpich?

@mkrapp

This comment has been minimized.

Show comment
Hide comment
@mkrapp

mkrapp Sep 5, 2016

Contributor

I'm not sure. It's just one simple check that leads to the fail. If I omit that check, i.e., --without-test, the build with open-mpi succeeds.

From the parallel-netcdf INSTALL:

...
PnetCDF can automatically detect the available MPI compilers and compile flags.
If alternate compilers or flags are desired, they can be specified by the
following environment variables and/or configure options.
...
Contributor

mkrapp commented Sep 5, 2016

I'm not sure. It's just one simple check that leads to the fail. If I omit that check, i.e., --without-test, the build with open-mpi succeeds.

From the parallel-netcdf INSTALL:

...
PnetCDF can automatically detect the available MPI compilers and compile flags.
If alternate compilers or flags are desired, they can be specified by the
following environment variables and/or configure options.
...
@jonchang

This comment has been minimized.

Show comment
Hide comment
@jonchang

jonchang Sep 5, 2016

Contributor

Does the test program compile? That is, if you install --without-test, and then run

brew test parallel-netcdf

If that's the case then please just disable the test by default and file an upstream bug about the make check error and add it as a comment. When upstream fixes the issue we can re-enable the check for everyone.

Contributor

jonchang commented Sep 5, 2016

Does the test program compile? That is, if you install --without-test, and then run

brew test parallel-netcdf

If that's the case then please just disable the test by default and file an upstream bug about the make check error and add it as a comment. When upstream fixes the issue we can re-enable the check for everyone.

mkrapp added some commits Sep 5, 2016

disable default check for build with `open-mpi`
* `make check` is disabled by default because it fails to pass if built with `open-mpi`
* A ticket has been submitted: https://trac.mcs.anl.gov/projects/parallel-netcdf/ticket/19
* add check for CXX, Fortran bindings
@jonchang

This comment has been minimized.

Show comment
Hide comment
@jonchang

jonchang Sep 5, 2016

Contributor

@BrewTestBot test this please

Contributor

jonchang commented Sep 5, 2016

@BrewTestBot test this please

@jonchang

This comment has been minimized.

Show comment
Hide comment
@jonchang

jonchang Sep 5, 2016

Contributor

I think this formula works now but it's encountering weird transient errors on the builders. Stay tuned...

Contributor

jonchang commented Sep 5, 2016

I think this formula works now but it's encountering weird transient errors on the builders. Stay tuned...

@jonchang

This comment has been minimized.

Show comment
Hide comment
@jonchang

jonchang Sep 6, 2016

Contributor

It looks like a race condition on the make install step. Could you wrap that step in a deparallelize block, like

ENV.deparallelize do
 ...
end

and re-push?

Contributor

jonchang commented Sep 6, 2016

It looks like a race condition on the make install step. Could you wrap that step in a deparallelize block, like

ENV.deparallelize do
 ...
end

and re-push?

@jonchang jonchang closed this in bd62b89 Sep 6, 2016

@jonchang

This comment has been minimized.

Show comment
Hide comment
@jonchang

jonchang Sep 6, 2016

Contributor

This is now merged. Thanks for being so patient while wrestling with the test bot!

Contributor

jonchang commented Sep 6, 2016

This is now merged. Thanks for being so patient while wrestling with the test bot!

@mkrapp mkrapp deleted the mkrapp:parallel_netcdf branch Sep 6, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.