Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-29421: Add AP plots in analysis_ap using analysis_tools #32

Merged
merged 3 commits into from Oct 27, 2022

Conversation

mrawls
Copy link
Contributor

@mrawls mrawls commented Sep 21, 2022

This PR adds a new plotting capability for DiaSources on the sky. The simple version of this plot is here in analysis_tools, and a specialized version for a specific AP QA data set is in analysis_ap.

@mrawls mrawls force-pushed the tickets/DM-29421 branch 2 times, most recently from a96d112 to bb66765 Compare September 21, 2022 17:00
@mrawls mrawls force-pushed the tickets/DM-29421 branch 2 times, most recently from 90d626b to ef0dd6e Compare October 22, 2022 01:27
@mrawls mrawls requested a review from natelust October 22, 2022 01:43
@mrawls
Copy link
Contributor Author

mrawls commented Oct 22, 2022

As part of the review, I need some assistance getting the modified pipeline to run before this is ready to merge. At the moment, when I run

pipetask run -b /sdf/data/rubin/u/parejko/cosmos-DM-36199/repo/butler.yaml -i ap_verify-output -o test_metric_plot -p $ANALYSIS_TOOLS_DIR/pipelines/apCcdVisitQualityCore.yaml

The result is as follows

lsst.daf.butler.cli.utils ERROR: Caught an exception, details are in traceback:
Traceback (most recent call last):
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/ctrl_mpexec/g14023acf34+0a8cb3cd94/python/lsst/ctrl/mpexec/cli/cmd/commands.py", line 160, in run
    if (qgraph := script.qgraph(pipelineObj=pipeline, **kwargs, show=show)) is None:
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/ctrl_mpexec/g14023acf34+0a8cb3cd94/python/lsst/ctrl/mpexec/cli/script/qgraph.py", line 196, in qgraph
    qgraph = f.makeGraph(pipelineObj, args)
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/ctrl_mpexec/g14023acf34+0a8cb3cd94/python/lsst/ctrl/mpexec/cmdLineFwk.py", line 579, in makeGraph
    qgraph = graphBuilder.makeGraph(
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/pipe_base/ge08f41d946+f12b9a963a/python/lsst/pipe/base/graphBuilder.py", line 1335, in makeGraph
    scaffolding = _PipelineScaffolding(pipeline, registry=self.registry)
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/pipe_base/ge08f41d946+f12b9a963a/python/lsst/pipe/base/graphBuilder.py", line 568, in __init__
    datasetTypes = PipelineDatasetTypes.fromPipeline(pipeline, registry=registry)
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/pipe_base/ge08f41d946+f12b9a963a/python/lsst/pipe/base/pipeline.py", line 1143, in fromPipeline
    pipeline = list(pipeline)
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/pipe_base/ge08f41d946+f12b9a963a/python/lsst/pipe/base/pipeline.py", line 665, in toExpandedPipeline
    taskDefs.append(self._buildTaskDef(label))
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/pipe_base/ge08f41d946+f12b9a963a/python/lsst/pipe/base/pipeline.py", line 708, in _buildTaskDef
    overrides.applyTo(config)
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/pipe_base/ge08f41d946+f12b9a963a/python/lsst/pipe/base/configOverrides.py", line 285, in applyTo
    setattr(tmpConfig, finalField, value)
  File "/sdf/group/rubin/sw/conda/envs/lsst-scipipe-4.1.0/share/eups/Linux64/pipe_tasks/ge54c72f270+dce6a40c95/python/lsst/pipe/tasks/configurableActions/_configurableActionStructField.py", line 196, in __setattr__
    valueInst = value(__name=name, __at=at, __label=label)
TypeError: 'str' object is not callable

@mrawls mrawls force-pushed the tickets/DM-29421 branch 4 times, most recently from ee2cc67 to 1234289 Compare October 25, 2022 23:55
@mrawls
Copy link
Contributor Author

mrawls commented Oct 26, 2022

This should be all set now and ready for your actual review!

Copy link
Contributor

@natelust natelust left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments to consider

@@ -20,13 +20,16 @@
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from __future__ import annotations

__all__ = "PlotConfig"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be in () like __all__ = ("PlotConfig", ) otherwise some things will iterate over the string

def setDefaults(self):
super().setDefaults()

self.process.buildActions.ras = LoadVector()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need process at all here you can just do:

self.prep.vectorKeys = ['ra', 'decl']
...
self.produce.panels['panel_main'].ra = 'ra'
self.produce.panels['panel_main'].dec = 'decl'

Again, this is just an alternative way to write this that may be more convenient, or may not be depending on how you think about things



class PanelConfig(Config):
"""Configuration options for plot panels."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider a bit more of a doc string, its not really panels for ANY plot


class DiaSkyPlot(PlotAction):
"""Generic pseudo base class for plotting DiaSources
(or DiaObjects) on the sky.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doc string seems awkward for some reason, is the line break in the right place?

Also add a PanelConfig class to plotUtils that can be used to turn various matplotlib knobs pertaining to subplot layout.
@mrawls mrawls merged commit 52d5591 into main Oct 27, 2022
@mrawls mrawls deleted the tickets/DM-29421 branch October 27, 2022 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants