Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-23436: apdb creation instructions outdated #51

Merged
merged 6 commits into from Feb 14, 2020
Merged

Conversation

jdswinbank
Copy link
Contributor

No description provided.

Copy link
Member

@kfindeisen kfindeisen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused by some of the revised wording, and I'm worried that new readers might also be confused. I'd like to iterate on that at least once before merging.

doc/lsst.ap.pipe/apdb.rst Outdated Show resolved Hide resolved
Comment on lines 17 to 19
The Alert Production Pipeline, as represented by :lsst-task:`lsst.ap.pipe.ApPipeTask` and executed by |ap_pipe|, delegates saving and loading DIASources and DIAObjects to its ``diaPipe`` subtask.
In principle, different implementations of ``diaPipe`` can be selected in defined in `ApPipeTask`'s configuration (by setting :lsst-config-field:`lsst.ap.pipe.ap_pipe.ApPipeConfig.diaPipe`), but the default choice --- :lsst-task:`lsst.ap.association.DiaPipelineTask` --- is expected to be appropriate for most uses.
`~lsst.ap.association.DiaPipelineTask` uses a database --- known as the Alert Production Database, or APDB --- to store its results.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think talking about diaPipe may be too much detail for the context paragraph. The target audience here is people who see the AP pipeline as a single unit and want to know how to run it from (almost) scratch. Why not delay mention of diaPipe or DiaPipelineTask to the instructions in the next section?

Or are you actually forseeing a future where, depending on the config, the AP pipeline might not need any sort of APDB?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point, and I wondered about it when writing the text.

My foresight is not as good as you imply it might be. However, given that the APDB usage is isolated in a retargetable subtask, I'm a bit leery of talking about it as being fundamental to the design of the pipeline. Given your comments, though, maybe that's a worthwhile simplification. I'll do some redrafting.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. We don't want DiaPipelineTask to be a subtask; we wrote it so that it would be a standalone PipelineTask in Gen 3, so there's not even an implicit contract that any alternative has to provide database support.

However, I don't think we'll be able to write a truly generic description of how this should work until DM-22663, so I think the simplification is worthwhile for now.

Comment on lines 26 to 29
|ap_pipe| includes information about the database location and schema in its `ApPipeConfig`, and most users pass this information to the script through the :option:`--config <ap_pipe.py --config>` and :option:`--configfile <ap_pipe.py --configfile>` command-line options.
The database is configured using an instance of `~lsst.dax.apdb.ApdbConfig`, which is made accessible through `ApPipeConfig`'s ``diaPipe.apdb`` option.
|ap_pipe| command line users can pass configuration information to the script through the :option:`--config <ap_pipe.py --config>` and :option:`--configfile <ap_pipe.py --configfile>` command-line options.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's important to make it clear from the first sentence that this paragraph is talking about how the database is configured in the context of ap_pipe.py (which the second paragraph then extends to make_apdb.py). The fact that these two scripts share a config is very useful but potentially very confusing.

How about simply removing the passive voice? "|ap_Pipe| configures the database using an instance..."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused about the point here. Surely ApPipeTask is the fundamental construct? Under what circumstances would I be using ApPipeTask but not configuring the database through ApPipeConfig.diaPipe.apdb? I'm worried that I'm being slow to catch on! :-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ApPipeTask is the fundamental construct, but this page is specifically about how to use make_apdb.py, not ApPipeTask. Mentioning the latter at all, while necessary to explain the shared UI, introduces some mental friction, or at least it has in the past for other AP developers.

Because:

- It's not obvious why this was here rather than in the doc/ dir.
- The fact it has been outdated for some time implies nobody reads it anyway!
Copy link
Member

@kfindeisen kfindeisen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes!

@jdswinbank jdswinbank merged commit 9d7043f into master Feb 14, 2020
@kfindeisen kfindeisen deleted the tickets/DM-23436 branch April 13, 2022 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants