Save spots #552
Save spots #552
Conversation
| dataset = dataset.asstr() | ||
|
|
||
| return dataset[()] | ||
| from .hdf5 import h5py_read_string |
There was a problem hiding this comment.
Nice! After this PR is merged, we can make sure paths like this one are updated in HEXRDGUI to be the new path instead, and then we can at some point remove this file.
| @@ -0,0 +1,100 @@ | |||
| """HDF5 Tools | |||
There was a problem hiding this comment.
Nice to have these in a separate file!
| else: | ||
| d[key] = tmp | ||
|
|
||
|
|
| detector_id, iRefl, peak_id, hkl_id, hkl, | ||
| tth_edges, eta_edges, np.radians(ome_eval), | ||
| xyc_arr, ijs, frame_indices, patch_data, | ||
| ang_centers[i_pt], xy_centers[i_pt], | ||
| meas_angs, meas_xy, sum_int, max_int |
There was a problem hiding this comment.
When there are many arguments, I think it's usually preferable to use kwargs, since it is less error-prone (if you forget a kwarg, the error is clearer than if you forget an arg, and you don't have to worry about accidentally swapping two arguments if you use kwargs, which can lead to subtle bugs if two args are similar).
But since this is already using args, we can just leave it!
| hkl_id: in | ||
|
|
||
| detector_iRefl, peak_id, hkl_id, hkl, | ||
| tth_edges, eta_edges, ome_eval, | ||
| xyc_arr, ijs, frame_indices, patch_data, | ||
| ang_centers, xy_centers, meas_angs, meas_xy | ||
|
|
There was a problem hiding this comment.
Looks like this docstring isn't finished yet.
This pull request is a work in progress and not intended to merged immediately.
I expect it to evaluated to determine worthwhile improvements and updated accordingly.
The intent is to provide a way to save reflection data for spots for examination.
The current hexrd saves a summary table of all the reflections of each grain in a text file.
The
GrainDataWriter_h5class allows you to save full information for each reflection.It is enabled in
pull_spotsbut not accessible through the config file interface;also it was set up to write a single file for all grains, creating a huge file.
This pull request allows you save spot intensities, one file per grain, accessibly through
the
fitgrainsconfig file.In addition, the summary and full detail are both available as well a new option to
save the intensities only, creating a much smaller file.
This also cleans up much code from
hedm_instrument.There were separate classes for writing patches (spots) to text files (summary) and
to hdf5 (detailed), and there was much code handling file names, directories and output
formats within the pull_spots() method, distracting from the real functionality of it.
All of that is now handled in the spots module.
Usage
You set writing options in the fitgrains config file. The
output_formatisa list of formats to write the spot outputs. There are three choices, "summary",
"full", and "sparse"; "full" is not recommended for more than a few grains.
Corresponding output files (for grain 1) are "spots00001.out" for the summary,
"spots00001.hdf5" for the full output, and "spots00001-sparse.hdf5" for the sparse output.
The "sparse" is not actually sparse (yet).
Currently only the intensities are saved, and because they are mostly zero, the hdf5
compression makes the file size small, about 2MB for the example I ran, compared with
40MB for the full. The full output also contains the intensities, but it also contains
ij-arrays and xy-arrays, which are dense over the same range as the intensities, making
the file much bigger.
For the purpose of finding overlapping peaks, I am thinking that if the distortion is small,
then then you won't actually need the xy values to mark a reflection as having overlapping peaks.
But if we need to include more information we can.
Changes
The main changes were to create a new
spotsmodule for handling reflection data.There is a Spot class for information about a single reflection.
Existing classes
GrainDataWriter_h5andPatchDataWriterwere moved fromhedm_instrumentto the new spots module.Modules Changed
hexrd.instrument.hedm_instrument:GrainDataWriter_h5andPatchDataWritertospotscenters_of_edge_vec()function togridutil, keeping a reference to it via importunwrap_h5_to_dictandunwrap_dict_to_h5to new modulehexrd.utils.hdf5, but kept references to themhexrd.utils.hdf5h5py_read_stringtohexrd.utils.hdf5, but kept reference to ithexrd.config.fitgrains: removed (unused)skip_on_estimateproperty and addedoutput_formatpropertyhexrd.fitgrains: modified to pass output options topull_spotshexrd.gridutil: addedcenters_of_edge_vec()functionhexrd.instrument.spots: new fileSpotsclass containing full patch data and summary valuesSpotsWriterclass for ouput of spots summary text file, full hdf5 file or a simplified hdf5 filehexrd.utils.compatibility.py: movedh5py_read_stringtohexrd.utils.hdf5.py, keeping reference to ithexrd.materials.py: changed import fromutils.compatibilitytoutils.hdf5tests/config/test_fit_grains.py: added unit tests for the newoutput_formatproperty