File IO: clean up and test #498
Conversation
Use with statements so that files are always closed. Remove special error handling for IO errors, since default traceback is just as informative. Write NaNs to aux array if it is not initialized.
- always close files - no special IO error handling, since default traceback is just as informative - combine dimension-independent parts of code
Always close files. Simpler handling of default options.
- always close files - move logger message that could never be reached - remove exceptions that could never be reached - remove options code that does nothing in read function - remove bc_lower and bc_upper which no longer belong to dimension objects - don't use exec - remove comments that merely restate clear code
examples/euler_3d/test_io.py tests the ASCII and HDF5 file formats.
ketch
force-pushed
the
more_io_cleanup
branch
from
5c8ab9f
to
5e2fa50
Compare
|
Can we add the hdf5 module to the travis testing? |
| @@ -0,0 +1,61 @@ | |||
| import os | |||
There was a problem hiding this comment.
This all looks good. Is the plan to leave this test in examples?
There was a problem hiding this comment.
I haven't worried about where to put it, since nose picks up everything. Currently we run nose from the examples directory, though.
|
👍 |
Dang, I forgot that we don't have that installed on Travis. I think the problem is that the install times out if we add more things. Maybe we should try miniconda: http://conda.pydata.org/docs/travis.html |
|
@ahmadia I hate to bother you about this yet again but is there any chance we could go with a <#> build or would it still be too slow without binary builds? |
|
Actually, @cekees came up with a clever method for working around hashdist's binary incompatibility. He's implemented his approach on Shippable, but I should be able to get it going on Travis CI. I can take a swing on it tomorrow. The main idea is to take advantage of the fact that the Travis build will always be of the form: |
|
Ah nice! It will be great to have this issue crossed off our list of problems. |
|
Yeah, all you need to do is find a way to run the image travis or On Fri, Jan 16, 2015 at 10:17 AM, Kyle Mandli notifications@github.com
|
|
I'm building the Travis Python VM on my laptop now. I'll play with putting together a hashed stack together. This would be useful to a number of projects with complex builds. |
|
@ketch, I'll send you a binary file for testing. |
|
Thanks @rjleveque . I have corrected binary.py and it reads just fine now. Further testing reveals that the hdf5 writer is not correct for AMR files (as I expected) so I'm working on that now. |
|
I will add tests for reading binary and writing AMR HDF5 soon. |
|
I added h5py in #499. |
Also moved IO tests to pyclaw/tests/ directory.
Pass name as keyword argument.
* master: Add ability to skip Sedov test (disable PETSc) Need pip in stack python for coverage/coveralls Move coverage tools to hashstack build Add scipy/mpl/h5py, streamline Travis Remove verbosity from untar Fast binary-cached build of hashdist dependencies
ketch
force-pushed
the
more_io_cleanup
branch
from
460a1cf
to
560c78d
Compare
In order to pick up new IO tests.
ketch
force-pushed
the
more_io_cleanup
branch
from
560c78d
to
940ec70
Compare
|
This is now ready to merge. As a bonus, some coverage is being reported again. |
Summary
This PR adds tests for the ASCII and HDF5 file IO routines, in which a solution is read from a binary file and an HDF5 file, written to ASCII and HDF5 files, and then read in again from those files. The resulting solution objects are then checked for equality.
Additionally, this PR refactors most of the IO routines, including the following changes:
Additional notes
Going through this highlighted to me that we are writing different sets of attributes in different cases. It seems worthwhile to define precisely the required and optional attributes of
Finally, the code in io_test.py could be simplified by defining an eq operator for these classes, which would also encode the information about required and optional attributes.