Speed up blurring_mask_2d_from for large PSF kernels
#45
Unanswered
GeorgeVassilakis
asked this question in
Ideas & Proposals
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi James!
I'm George, a PhD student at Cambridge. I've been using PyAutoLens for my HWO-SLAPS project on subhalo detectability with the Habitable Worlds Observatory (paper here). I worked with your former collaborators Richard Massey and Qiuhan He on this :)
The project uses really large PSF kernels (up to 999 x 999), and building the convolver got extremely slow and memory hungry as the kernel grew, to the point of OOM-killing jobs on our cluster. I traced it to the
binary_dilationinblurring_mask_2d_from, which uses a densenp.ones((ky, kx))structure (scipy's offsets table for it grows as(ky * kx)**2, which is where the memory goes). Since the structure is a box, the dilation is separable, so a(ky, 1)dilation followed by a(1, kx)dilation gives the exact same mask for a fraction of the cost:The masks are byte-identical, the full
test_autoarraysuite passes, and I added a test that checks against the dense dilation.For the paper I just patched autoarray locally, but I figured it'd be better to get this upstream so anyone else using big PSFs doesn't hit the same wall. I have a PR ready to go if you're happy with the approach. Thanks for all the work on PyAuto, and let me know what you think!
All reactions