Addition of facility to read and write the entire grid to FITS file.#92
Merged
allegroLeiden merged 18 commits intolime-rt:masterfrom Nov 8, 2016
Merged
Addition of facility to read and write the entire grid to FITS file.#92allegroLeiden merged 18 commits intolime-rt:masterfrom
allegroLeiden merged 18 commits intolime-rt:masterfrom
Conversation
A new module grid2fits.c has been added which contains code to do this. There is also a new function processFitsError() in messages.c. The appropriate necessary additions have been made to lime.h and the makefile. Note I also made some cosmetic changes to the Makefile, mostly evening out the tab counts in SRCS and OBJS lines. So far I have not added any machinery by which the user can access the new FITS i/o routines. The functionality of the task is unaltered. I just wanted to add the bulk code first (which has already been tested elsewhere) and make sure it all compiled before proceeding further.
Here I have added some places where the grid is written to file. Everything seems to work ok, and the output values look sensible, but I have not yet done any exhaustive tests of the values in the fits files. Some new elements of the struct inputPars have been necessary. The default values of these cause no fits output to be written. In fact there should be no changes in the way the software functions with the standard model.c file. I also added a new function printMessage() to messages.c and some lines in main.c which print the number of threads used when this is more than 1. The fits-read functionality remains unused. Adding the ability to read the grid from file should perhaps wait on discussion among the developers about the future of the pregrid etc machinery.
This commit covers numerous minor issues to get them out of the way before the big one (or ones). A summary is as follows: - Includes fix for issue lime-rt#131: where nmol was not being set for sink points in LTE. - C++->C style comments in aux.c:levelPops(). - conv renamed nItersDone in aux.c:levelPops(). - Replaced 'if(fileName)' with 'if(fileName!=NULL)' in a few places in aux.c:levelPops(). - Replaced 'g' by 'gp' and 'm' by 'md' in numerous places. - Number of density values returned is now counted (and used to set par->collPart) in aux.c:parseInput() rather than grid.c:gridAlloc(). - Added warning in aux.c:parseInput() if could not find moldata file. - Grid velocity now set at the same time as dens, abun etc in grid.c:readOrBuildGrid() rather than in getVelosplines(). - New header file gridio.h with a couple of macros relating to file types. - New structs molInfoType, gridInfoType to contain sizes of various fields in grid info which has been read from file. (Ultimately these should be merged into a single struct which contains all the grid data.) - messages.c:done() renamed to printDone(). - photon.c:photon(): unused variable x[] removed. - In smooth.c:smooth(): qhull() and distCalc() were called at the end of each iteration; now they are called at the beginning of each. In grid.c:buildGrid() the calls to qhull() and distCalc() which preceded the call to smooth() now succeed it. The functionality is unchanged but it allows one to read a grid file at stage 1 and have the triangulation performed by qhull() without perturbation of the grid point locations.
Velocity polynomial interpolation coefficients are now calculated with respect to an independent variable d/ds-0.5 instead of d. The new variable is symmetrical about the midpoint of the distance ds between grid points and so allows easy conversion of the coefficients from one member of a pair of connected points to the other. I took the opportunity to rename and rewrite functions as follows: - velospline.c:getVelosplines() renamed to calcInterpCoeffs() (same for _lin() versions). - photon.c:velocityspline() renamed to calcAvRelLineAmp() (same for _lin() versions). - Added photon.c:interpolate().
I replaced the NITERATIONS macro by a user-settable parameter par->nSolveIters. Mostly this is to make it easier to iterate a previous population solution further, when this can eventually be implemented.
This is a substantial commit, but it is hard to see how it could have been easily broken into smaller ones. Here I have reworked the previous facility for IO to and from FITS-format files. There is another, generic IO layer now implemented in a new module gridio.c, with FITS relegated to a choie of file format. Hopefully this will make it easier if anybody else wants to write an interface to HDF5 or a DB or whatever. Some detail on the changes: - New pars par->gridInFile, par->gridOutFiles[] (user-settable) and par->writeGridAtStage[], par->dataStageI (not intended to be user settable). - New function grid.c:mallocAndSetDefaultGrid(). - grid.c:buildGrid() rewritten and renamed readOrBuildGrid(). - main.c changed to include the fix to lime-rt#130.
Also added data-stage flag macros. None of this stuff is used yet, I am just clearing the decks before the big change.
There are two main changes here, although there are a lot of changed lines of code. - A generic i/o layer has been added, with choice of FITS just one of the (currently the only) possibilities for the format of the disk files. Hopefully this will make it easier for people to add interfaces to HDF5 or whatever at a later date. - The information which can legally be read from or written to the files has now been made much more flexible. When reading the file, if a particular element of the disk struct, say density, was not supplied, the task will automatically seek to fill in these missing values via the standard route. The completion state of the grid struct is now indicated by bit flags rather than a single 'stage' integer.
Code still seg faults. I'll commit this now and squash it after I fix things.
In addition, I further straightened out the basic structure. There is more work still to be done, but it is becoming more acceptable. I am not yet able to read in pre-calculated popluations and go straight to ray-tracing. This will require reform to the way dust/knu values are stored (see lime-rt#151).
I also moved some IO-specific structs and function prototypes out of lime into a new header file grid2fits.h. Some bug fixes also done.
Contributor
Author
|
As soon as this completes its Travis check it will be ready to merge. I'll do that tomorrow if nobody objects. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Much of the additional material in this PR is in the module grid2fits.c. In this module are comprehensive and tested routines to read and write the data in the list of grid structs from/to a FITS file. One problem with doing this is that the grid list contains different amounts of information at differing places in the code. It may not always be desired to wait the full processing time before outputting the grid - for example if the user just wants to look at the distribution of the points. For this reason the various scenarios have been divided into 4 stages in which grid contains differing amounts of information. The details of this are described in a long comment at the start of grid2fits.c.
For the time being I have included only some fairly rudimentary support for actually writing the grid (i.e., making use of the new functionality), and none at all for reading it. A more comprehensive service should probably wait on discussion about the future design of LIME's i/o.
17 June 2016: I reworked this PR substantially. I've now done the best job I can to (i) make the IO interface generic and not tied absolutely to FITS, and (ii) to implement fully read-and-write capability for the grid data. For the 2nd of these I have not yet been able to read the data at stage 4 (i.e. after all the level populations have values for them) and feed that either to further iterations of solver or direct to ray-tracing. Before that can be done, there needs to be a much more orderly succession of operations preparatory and during the level solving, which I felt was a bit beyond the scope of the present PR.