Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-16606: ap_pipe should not create DB automatically #39

Merged
merged 5 commits into from Apr 1, 2019

Conversation

kfindeisen
Copy link
Member

This PR creates a make_ppdb.py script that takes the same config arguments as ap_pipe.py and only calls Ppdb.makeSchema, then removes the corresponding call from ApPipeTask*. This gives users more control over database creation and avoids certain race conditions when ApPipeTask is run in parallel.

*For backward-compatibility, the call from ApPipeTask may be reinstated and scheduled for removal on another ticket. See discussion on #dm-alert-prod.

@kfindeisen kfindeisen changed the title DM-16536: ap_pipe should not create DB automatically DM-16606: ap_pipe should not create DB automatically Mar 13, 2019
Copy link
Contributor

@andy-slac andy-slac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, few minor comments.

@@ -161,7 +163,7 @@ A full command looks like

.. prompt:: bash

ap_pipe.py repo --calib repo/calibs --rerun processed -C $AP_PIPE_DIR/config/calexpTemplates.py -c associator.level1_db.db_name=ppdb/association.db --id visit=123456 ccdnum=42 filter=g --template /path/to/calexp/templates --templateId visit=234567
ap_pipe.py repo --calib repo/calibs --rerun processed -C $AP_PIPE_DIR/config/calexpTemplates.py -c ppdb.isolation_level=READ_UNCOMMITTED -c ppdb.db_url="sqlite:///ppdb/association.db" --id visit=123456 ccdnum=42 filter=g --template /path/to/calexp/templates --templateId visit=234567
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General comment - it would be nice to get rid of tat nuisance parameter ppdb.isolation_level=READ_UNCOMMITTED and have more standard defaults in configuration. Maybe not now, but something for the new ticket.

python/lsst/ap/pipe/make_ppdb.py Outdated Show resolved Hide resolved
tests/test_makeppdb.py Outdated Show resolved Hide resolved
This is a breaking change to the command-line interface.
@kfindeisen kfindeisen merged commit 064edbd into master Apr 1, 2019
@kfindeisen kfindeisen deleted the tickets/DM-16606 branch April 1, 2019 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants