add PlotFileData bindings

[BenWibking]

Enable reading plotfiles with pyAMReX. Also adds a script in tools/ to create images from plotfiles.

Example use-case: https://github.com/BenWibking/plotfile-viewer.

Closes #317.

[ax3l]

@BenWibking thanks a lot, looking good! Ping us when you are ready with the unit test and we can include it maybe in this month's release even?

[BenWibking]

@ax3l I've added a unit test to read a 3D plotfile with a single level. I put the binary data in the PR, but I can move that to some more appropriate place if needed.

When I try to add a test for a 2D plotfile, I get this error:

tests/test_plotfiledata_2d.py:8: in <module>
    import amrex.space2d as amr
../myenv/lib/python3.12/site-packages/amrex/space2d/__init__.py:22: in <module>
    from . import amrex_2d_pybind
E   ImportError: generic_type: type "AMReX" is already registered!

Do 2D tests need to have some special configuration?

[BenWibking]

@ax3l Aside from the 2D test issue, everything should be working now.

[ax3l]

@BenWibking Thanks for the question!

Yes, please do not check in binary files into the repository, to keep it small. We clone pyAMReX in many workflows, e.g., superbuilds and a slick repo size is important.

Could you potentially for the test generate the data on the fly, e.g., by running a generating function first that you read back?

[ax3l]

I am surprised the pre-commit.ci check check-added-large-files that we run on CI does not fail on the 6 MB binary file... concerning :-o

[BenWibking]

My concern is that I don't know how to independently verify that the plotfile generation was correct if it's generated on the fly. Would it be possible to upload the test files as an extra .tar.gz file in a GitHub release, or to re-use the test data on the yt website under "boxlib frontend"?

[ax3l]

Downloading the test data on the fly is ok.
There is a downside to this, making some test runs slow and others impossible (w/o internet).

My concern is that I don't know how to independently verify that the plotfile generation was correct if it's generated on the fly.

I think this can be verified once manually or in a separate test. (We also do not verify a downloaded files continuously, unless I misunderstand something here.)

[BenWibking]

Ok, that makes sense. I think verifying the checksum for the generated plotfile would work instead of downloading a file (since that should be checksummed anyway).

[ax3l]

@BenWibking when you have the time, can you still update this? I would merge the PR once the large files are not checked in anymore :)

[baperry2]

This is awesome! I was hoping for this capability. In general, to use pyamrex for pre/postprocessing workflows having as much I/O capability as possible is super helpful.

Another example would be reading/writing individual FAB data (appears to be commented out here:

pyamrex/src/Base/FArrayBox.cpp

Line 38 in 63cae48

/*

), which would have been useful for something I was looking at doing a while ago, but am not longer planning on.

[BenWibking]

This is awesome! I was hoping for this capability. In general, to use pyamrex for pre/postprocessing workflows having as much I/O capability as possible is super helpful.

Another example would be reading/writing individual FAB data (appears to be commented out here:

pyamrex/src/Base/FArrayBox.cpp

Line 38 in 63cae48

/*

), which would have been useful for something I was looking at doing a while ago, but am not longer planning on.

That would be useful. I'm not sure why that's commented out. Maybe @ax3l knows?

In the future, I would like to add the ability to lazy-load FABs for out-of-core analysis, but that might require additional changes in AMReX.

[ax3l]

Awesome, yes those function should be added, too, but maybe in a separate PR.

There are some comments in #82 and tests/test_farraybox.py how to bind C++ iostreams and Python io. I simply had no time to test it and no need yet, but PRs are very welcome on it (with a test please) 🙏