Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-30267: Produce three-stamp alerts #38

Merged
merged 2 commits into from
Mar 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
32 changes: 16 additions & 16 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,22 +35,22 @@ Steps to update the alert schema (for example, when the APDB schema is updated).
* Checkout the ticket branch for your schema changes.
* Update the default ``schema_root`` kwarg in ``python/lsst/alert/packet/schemaRegistry.py:from_filesystem()`` to your new schema version number.

* New schemas are built from the apdb. Any changes to a field should be done in apdb.yaml, and any changes to what is included/excluded should be made in updateSchema.py.
* To update the schema, you must have the path to the apdb.yaml file and have chosen a version number. If the directory for your version
* does not already exist, ``updateSchema.py`` will create it.

* run ``python updateSchema.py /path/to/LSST/code/sdm_schemas/yml/apdb.yaml Path/To/alert_packet/lsst/alert/packet/schema "6.0"`` All Generated files do not need to be altered.
* Navigate to the new schema. Copy in the previous ``lsst.vX_X.alert.avsc`` file and ``lsst.vX_X.diaNondetectionLimit.avsc``.
* Within the two copied files, update ``"namespace": "lsst.vX_X",`` line at the top of each ``*.avsc`` file to the new version.
* Update the contents of those avro schema files to reflect the new schema.
* Update the sample alert packet in ``sample_data``:

* Update ``alert.json`` to reflect the new schema.
* Change the ``schema_root`` and ``get_by_version`` parameters in ``generate.py`` to your new version number.
* Run ``python generate.py`` to produce a new ``fakeAlert.avro`` file with data filled in from the updated json file above and using the new schema files you made earlier.

* Update the files ``*.avsc`` and ``*.json`` files in ``examples/`` to reflect the new schema.
* Update the contents of ``latest.txt`` to your new schema version number.
* New schemas are built from the apdb. Any changes to a field should be done in sdm_schemas/yml/apdb.yaml, and any changes to what is included/excluded should be made in updateSchema.py.
* To update the schema, you must have the path to the apdb.yaml file and have chosen a version number. If the directory for your version does not already exist, ``updateSchema.py`` will create it.

* run ``python updateSchema.py /path/to/LSST/code/sdm_schemas/yml/apdb.yaml Path/To/alert_packet/lsst/alert/packet/schema "6.0"`` All Generated files do not need to be altered.
* Navigate to the new schema. Copy in the previous ``lsst.vX_X.alert.avsc`` file and ``lsst.vX_X.diaNondetectionLimit.avsc``.
* Within the two copied files, update ``"namespace": "lsst.vX_X",`` line at the top of each ``*.avsc`` file to the new version.
* Update the contents of those avro schema files to reflect the new schema.

* Update the sample alert packet in ``sample_data``:

* Update ``alert.json`` to reflect the new schema.
* Change the ``schema_root`` and ``get_by_version`` parameters in ``generate.py`` to your new version number.
* Run ``python generate.py`` to produce a new ``fakeAlert.avro`` file with data filled in from the updated json file above and using the new schema files you made earlier.

* Update the files ``*.avsc`` and ``*.json`` files in ``examples/`` to reflect the new schema.
* Update the contents of ``latest.txt`` to your new schema version number.

* Add all of your new and updated files to ``git``, commit them with a message that includes the new schema version number and why it was incremented, and push your new branch.
* Test your changes with ap_association and ap_verify (these steps may be more involved if your ticket branch impacts multiple packets):
Expand Down
24 changes: 24 additions & 0 deletions python/lsst/alert/packet/schema/6/1/lsst.v6_1.alert.avsc
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"namespace": "lsst.v6_1",
"type": "record",
"name": "alert",
"doc": "Rubin Avro alert schema v6.1",
"fields": [
{"name": "alertId", "type": "long", "doc": "unique alert identifer"},
{"name": "diaSource", "type": "lsst.v6_1.diaSource"},
{"name": "prvDiaSources", "type": ["null", {
"type": "array",
"items": "lsst.v6_1.diaSource"}], "default": null},
{"name": "prvDiaForcedSources", "type": ["null", {
"type": "array",
"items": "lsst.v6_1.diaForcedSource"}], "default": null},
{"name": "prvDiaNondetectionLimits", "type": ["null", {
"type": "array",
"items": "lsst.v6_1.diaNondetectionLimit"}], "default": null},
{"name": "diaObject", "type": ["null", "lsst.v6_1.diaObject"], "default": null},
{"name": "ssObject", "type": ["null", "lsst.v6_1.ssObject"], "default": null},
{"name": "cutoutDifference", "type": ["null", "bytes"], "default": null},
{"name": "cutoutScience", "type": ["null", "bytes"], "default": null},
{"name": "cutoutTemplate", "type": ["null", "bytes"], "default": null}
]
}
81 changes: 81 additions & 0 deletions python/lsst/alert/packet/schema/6/1/lsst.v6_1.diaForcedSource.avsc
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
{
"type": "record",
"namespace": "lsst.v6_1",
"name": "diaForcedSource",
"fields": [
{
"doc": "Unique id.",
"name": "diaForcedSourceId",
"type": "long"
},
{
"doc": "Id of the DiaObject that this DiaForcedSource was associated with.",
"name": "diaObjectId",
"type": "long"
},
{
"doc": "Id of the ccdVisit where this forcedSource was measured.",
"name": "ccdVisitId",
"type": "long"
},
{
"doc": "Point Source model flux.",
"default": null,
"name": "psfFlux",
"type": [
"null",
"float"
]
},
{
"doc": "Uncertainty of psfFlux.",
"default": null,
"name": "psfFluxErr",
"type": [
"null",
"float"
]
},
{
"doc": "x position at which psfFlux has been measured.",
"default": null,
"name": "x",
"type": [
"null",
"float"
]
},
{
"doc": "y position at which psfFlux has been measured.",
"default": null,
"name": "y",
"type": [
"null",
"float"
]
},
{
"doc": "Flags, bitwise OR tbd",
"default": null,
"name": "flags",
"type": [
"null",
"long"
]
},
{
"doc": "Effective mid-visit time for this diaForcedSource, expressed as Modified Julian Date, International Atomic Time.",
"name": "midpointMjdTai",
"type": "double"
},
{
"doc": "Filter band this source was observed with.",
"default": null,
"name": "band",
"type": [
"null",
"string"
]
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"namespace": "lsst.v6_1",
"name": "diaNondetectionLimit",
"type": "record",
"fields": [
{"name": "ccdVisitId", "type": "long"},
{"name": "midpointMjdTai", "type": "double"},
{"name": "band", "type": "string"},
{"name": "diaNoise", "type": "float"}
]
}