Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapted quantile interpolation #174

Merged
merged 15 commits into from
Apr 13, 2022
Merged

Adapted quantile interpolation #174

merged 15 commits into from
Apr 13, 2022

Conversation

RuneDominik
Copy link
Member

@RuneDominik RuneDominik commented Jan 31, 2022

This is an alternative interpretation to the approach taken in #163. Instead of following the steps of this tech report closely, @maxnoe suggested to implement different solutions to the steps summarized in the tech reports Fig. 5.

On the upside, this method should scale easily to nD interpolation variables without using Delaunay simplices. This comes, however, at the price of worse results, at least in the testing case in my comment in #161.

@chaimain
Copy link

Is there any update on this or #163?

Copy link
Member

@maxnoe maxnoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A general comment, also for the other PR:

Please don't implement the actual interpolation logic inside interpolate_energy_dispersion.

Implement functions interpolate_binned_pdf_<method>(binned_pdfs, edges, grid_points, interpolation_point, **options) (That should be the required arguments, right?)

This function should then be just applied along the relevant axes in interpolate_energy_dispersion and interpolate_psf which could have a method= argument if we decide to merge both approaches.

We cannot make the assumption of equal distance bins, at least not in general. We could make them in the interpolation method if that is strictly required, but then interpolate_energy_dispersion should check for equal-distance logspace binning in migra and then do the interpolation in logspace.

Also please split up the steps inside into single functions. E.g. the resampling of the interpolated pdf back to the original bin edges.

pyirf/interpolation.py Outdated Show resolved Hide resolved
pyirf/interpolation.py Outdated Show resolved Hide resolved
pyirf/interpolation.py Outdated Show resolved Hide resolved
* Renamed hists-variable to binned_pdf
* Array broadcasting instead of repeating
* Isolated interpolation algorithm into function
* Segmented interpolation algorithm into subfunctions
* Handled a problem where cdfs could become constant and inverting would raise an error
* Typos
pyirf/interpolation.py Outdated Show resolved Hide resolved
pyirf/interpolation.py Outdated Show resolved Hide resolved
pyirf/interpolation.py Outdated Show resolved Hide resolved
pyirf/interpolation.py Outdated Show resolved Hide resolved
pyirf/interpolation.py Outdated Show resolved Hide resolved
pyirf/interpolation.py Outdated Show resolved Hide resolved
pyirf/interpolation.py Outdated Show resolved Hide resolved
pyirf/interpolation.py Outdated Show resolved Hide resolved
Minor changes:
* Removed two comments
* Renamed reconstruct_pdf_values to pdf_from_ppf
* Astropy units in A_eff interpolation defaults
* Reverted one change introduced by black
* Fixed a problem where pdfs that started with a bin != 0 couse problems
due to a cdf consequently starting with a value != 0 by introducing
a dummy bin with content 0 in front of every pdf. This bin is most often
ignored due to the selective ppf computation.
* Moved bin_mids computation inside ppf_values-function as they are only
needed there
pyirf/interpolation.py Outdated Show resolved Hide resolved
* Assuming correct GADF IRFs so the axis keyword is no longer needed
* Reworked Doc-Strings
* Deleted tests for interpolate_energy_dispersion and interpolate_psf_table
* Added tests for the interpolation code itself using linearly changing
Gaussians on a 1D Grid
@codecov
Copy link

codecov bot commented Apr 12, 2022

Codecov Report

Merging #174 (0bf07a4) into master (3f9cfba) will decrease coverage by 0.36%.
The diff coverage is 95.32%.

❗ Current head 0bf07a4 differs from pull request most recent head b8dc536. Consider uploading reports for the commit b8dc536 to get more accurate results

@@            Coverage Diff             @@
##           master     #174      +/-   ##
==========================================
- Coverage   91.10%   90.73%   -0.37%     
==========================================
  Files          41       41              
  Lines        1607     1598       -9     
==========================================
- Hits         1464     1450      -14     
- Misses        143      148       +5     
Impacted Files Coverage Δ
pyirf/interpolation.py 92.85% <91.52%> (-7.15%) ⬇️
pyirf/tests/test_interpolation.py 100.00% <100.00%> (ø)
pyirf/_dev_version.py
pyirf/_dev_version/__init__.py 60.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3f9cfba...b8dc536. Read the comment docs.

@RuneDominik RuneDominik marked this pull request as ready for review April 13, 2022 10:49
pyirf/interpolation.py Outdated Show resolved Hide resolved
@RuneDominik
Copy link
Member Author

The cdf_values-function assures that all cdfs max out at 1.


# Interpolate pdf samples and evaluate at bin edges, weight with the bin_width to estimate
# correct bin height via the midpoint rule formulation of the trapezoidal rule
pdf_values = np.apply_along_axis(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this was black? This looks horrible...

Maybe better just define a normal function using def in here instead of using a lambda?

Copy link
Member

@maxnoe maxnoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks very good!

@RuneDominik RuneDominik merged commit 639a8fe into cta-observatory:master Apr 13, 2022
@RuneDominik RuneDominik deleted the AdaptedQuantileInterpolation branch April 13, 2022 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants