Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-15588: Remove home-brewed SQLite PPDB #32

Merged
merged 4 commits into from
Nov 30, 2018
Merged

DM-15588: Remove home-brewed SQLite PPDB #32

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
821a39f
Change AssociationTask to directly use Ppdb.
morriscb Oct 17, 2018
d14c575
Remove deprecaded DB wrappers.
morriscb Oct 22, 2018
47bc5d3
Add full PPDB schema to DiaSource and Object afw Schems.
morriscb Nov 7, 2018
4c9e5e0
Simplify update unittest.
morriscb Nov 28, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
42 changes: 0 additions & 42 deletions data/l1db-ap-pipe-afw-map.yaml
View file
Open in desktop

This file was deleted.

15 changes: 15 additions & 0 deletions data/ppdb-ap-pipe-afw-map.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This file defines mapping between L1DB column names and afw.table column
# names. Top-level structure is a dictionary indexed by table name and the
# values are dictionaries where key is the name of column in L1DB and value
# is the name of the column in afw.table

DiaObject:
diaObjectId: id
ra: coord_ra
decl: coord_dec

DiaSource:
diaSourceId: id
parentDiaSourceId: parent
ra: coord_ra
decl: coord_dec
8 changes: 8 additions & 0 deletions data/l1db-ap-pipe-schema-extra.yaml → data/ppdb-ap-pipe-schema-extra.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ columns:
nullable: true
description: Time when validity of this diaObject ends.
default: null
- name: nDiaSources
type: INT
nullable: false
description: Total number of DiaSources associated with this DiaObject.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused by this. You add the same thing to dax_ppdb/data/ppdb-schema-extra.yaml; why do you need two more copies here? Which file is actually being used?

Copy link
Contributor Author

@morriscb morriscb Nov 27, 2018 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one being used in ap_pipe is from ap_association. The reason the duplication exists is because the ap_association one as extra columns for the observed DiaSource filter. It also doesn't have the DiaObjectLast table as we don't currently use the Ppdb concept of "dailyJob". The current default config is baseline.


---
# DiaObjectLast uses the same columns as DiaObject but has different index
Expand All @@ -27,6 +31,10 @@ indices:
columns:
- diaObjectId
type: INDEX
- name: nDiaSources
type: INT
nullable: false
description: Total number of DiaSources associated with this DiaObject.

---
# Special PK index for DiaObject table with spacial column being first
Expand Down
2 changes: 0 additions & 2 deletions python/lsst/ap/association/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,4 @@
from .version import *
from .association import *
from .mapApData import *
from .assoc_db_sqlite import *
from .afwUtils import *
from .l1dbproto import *
598 changes: 431 additions & 167 deletions python/lsst/ap/association/afwUtils.py
View file
Open in desktop

Large diffs are not rendered by default.