-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tickets/DM-20129 #9
Merged
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
7adf6a5
Remove tests already in scarlet
2d51393
Update to new scarlet API
8602b5c
Add tests for LsstSource
58a8a71
Add tests for LsstBlend
9b902b6
Use lsst camel-case convention
c06cf1a
a few more camel-case corrections
cad2e5c
Add test for ScarletDeblendTask
b7df3d2
Fix flake8 error
a0ebe25
Make symmetry the default
8b1720b
Fix test to use symmetry by default
2a90881
Update change tracking tests
32e0683
Update tests to give more tolerance for Jenkins
5c139b6
Add meas_algorithms as a dependency for tests
6b94a32
Update tests with reviewer comments
File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,119 @@ | ||
|
||
|
||
# This file is part of meas_extensions_scarlet. | ||
# | ||
# Developed for the LSST Data Management System. | ||
# This product includes software developed by the LSST Project | ||
# (https://www.lsst.org). | ||
# See the COPYRIGHT file at the top-level directory of this distribution | ||
# for details of code ownership. | ||
# | ||
# This program is free software: you can redistribute it and/or modify | ||
# it under the terms of the GNU General Public License as published by | ||
# the Free Software Foundation, either version 3 of the License, or | ||
# (at your option) any later version. | ||
# | ||
# This program is distributed in the hope that it will be useful, | ||
# but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
# GNU General Public License for more details. | ||
# | ||
# You should have received a copy of the GNU General Public License | ||
# along with this program. If not, see <https://www.gnu.org/licenses/>. | ||
|
||
import unittest | ||
|
||
import numpy as np | ||
|
||
import lsst.utils.tests | ||
import lsst.afw.image as afwImage | ||
from lsst.meas.algorithms import SourceDetectionTask | ||
from lsst.meas.extensions.scarlet import ScarletDeblendTask | ||
from lsst.afw.table import SourceCatalog | ||
from lsst.afw.detection import MultibandFootprint | ||
from lsst.afw.image import Image, MultibandImage | ||
|
||
from utils import initData | ||
|
||
|
||
class TestDeblend(lsst.utils.tests.TestCase): | ||
def test_deblend_task(self): | ||
# Set the random seed so that the noise field is unaffected | ||
np.random.seed(0) | ||
# Test that executing the deblend task works | ||
# In the future we can have more detailed tests, | ||
# but for now this at least ensures that the task isn't broken | ||
shape = (5, 31, 55) | ||
coords = [(15, 25), (10, 30), (17, 38)] | ||
amplitudes = [80, 60, 90] | ||
result = initData(shape, coords, amplitudes) | ||
targetPsfImage, psfImages, images, channels, seds, morphs, targetPsf, psfs = result | ||
B, Ny, Nx = shape | ||
|
||
# Add some noise, otherwise the task will blow up due to | ||
# zero variance | ||
noise = 10*(np.random.rand(*images.shape)-.5) | ||
images += noise | ||
|
||
filters = "grizy" | ||
_images = afwImage.MultibandMaskedImage.fromArrays(filters, images.astype(np.float32), None, | ||
noise.astype(np.float32)) | ||
coadds = [afwImage.Exposure(img, dtype=img.image.array.dtype) for img in _images] | ||
coadds = afwImage.MultibandExposure.fromExposures(filters, coadds) | ||
for b, coadd in enumerate(coadds): | ||
coadd.setPsf(psfs[b]) | ||
|
||
schema = SourceCatalog.Table.makeMinimalSchema() | ||
|
||
detectionTask = SourceDetectionTask(schema=schema) | ||
config = ScarletDeblendTask.ConfigClass() | ||
config.maxIter = 200 | ||
deblendTask = ScarletDeblendTask(schema=schema, config=config) | ||
|
||
table = SourceCatalog.Table.make(schema) | ||
detectionResult = detectionTask.run(table, coadds["r"]) | ||
catalog = detectionResult.sources | ||
self.assertEqual(len(catalog), 1) | ||
_, result = deblendTask.run(coadds, catalog) | ||
|
||
# Changes to the internal workings of scarlet will change these results | ||
# however we include these tests just to track changes | ||
parent = result["r"][0] | ||
self.assertEqual(parent["iterations"], 11) | ||
self.assertEqual(parent["deblend_nChild"], 3) | ||
|
||
heavies = [] | ||
for k in range(1, len(result["g"])): | ||
heavy = MultibandFootprint(coadds.filters, [result[b][k].getFootprint() for b in filters]) | ||
heavies.append(heavy) | ||
|
||
seds = np.array([heavy.getImage(fill=0).image.array.sum(axis=(1, 2)) for heavy in heavies]) | ||
true_seds = np.array([ | ||
[[1665.726318359375, 1745.5401611328125, 1525.91796875, 997.3868408203125, 0.0], | ||
[767.100341796875, 1057.0374755859375, 1312.89111328125, 1694.7535400390625, 2069.294921875], | ||
[8.08012580871582, 879.344970703125, 2246.90087890625, 4212.82470703125, 6987.0849609375]] | ||
]) | ||
|
||
self.assertFloatsAlmostEqual(true_seds, seds, rtol=1e-8, atol=1e-8) | ||
|
||
bbox = parent.getFootprint().getBBox() | ||
data = coadds[:, bbox] | ||
model = MultibandImage.fromImages(coadds.filters, [ | ||
Image(bbox, dtype=np.float32) | ||
for b in range(len(filters)) | ||
]) | ||
for heavy in heavies: | ||
model[:, heavy.getBBox()].array += heavy.getImage(fill=0).image.array | ||
|
||
residual = data.image.array - model.array | ||
self.assertFloatsAlmostEqual(np.abs(residual).sum(), 11601.3867187500) | ||
self.assertFloatsAlmostEqual(np.max(np.abs(residual)), 56.1048278809, rtol=1e-8, atol=1e-8) | ||
|
||
|
||
class MemoryTester(lsst.utils.tests.MemoryTestCase): | ||
pass | ||
|
||
|
||
if __name__ == "__main__": | ||
lsst.utils.tests.init() | ||
unittest.main() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this happen if the pointSource is set to False?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see the above comment, but am just wondering the logic, should there not just be an error thrown or something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Part of this might be the nomenclature. Both
ExtendedSource
andPointSource
are used to model extended sources, they just differ in their initialization. AnExtendedSource
uses a multiband coadd to attempt to initialize large extended sources but requires a certain signal to noise level to function properly. Very faint sources barely above the background will not have enough flux to meet this requirement, so we initialize them as a point source but they are still allowed to grow.