DM-42618: Add startTime to the nextVisit schema in prompt processing #126
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hsinfang
force-pushed
the
tickets/DM-42618
branch
from
2f674f2
to
1e22d74
Compare
kfindeisen
approved these changes
Feb 20, 2024
python/tester/upload.py
Outdated
| @@ -217,7 +218,7 @@ def get_samples_non_lsst(bucket, instrument): | |||
| dome=FannedOutVisit.Dome.OPEN, | |||
| duration=float(EXPOSURE_INTERVAL+SLEW_INTERVAL), | |||
| totalCheckpoints=1, | |||
| private_sndStamp=hsc_metadata[exp_id]["time"], | |||
| private_sndStamp=hsc_metadata[exp_id]["time"]-60, | |||
There was a problem hiding this comment.
For self-consistency, it might be better to calculate the offset from EXPOSURE_INTERVAL and SLEW_INTERVAL.
There was a problem hiding this comment.
Good idea. I'm setting them to 2*duration before the exposure start time. All these timestamps were in the past, as from the original raws' metadata, so they will need to be better simulated to be useful.
python/tester/upload_hsc_rc2.py
Outdated
| @@ -190,7 +191,7 @@ def prepare_one_visit(kafka_url, group_id, butler, visit_id): | |||
| dome=SummitVisit.Dome.OPEN, | |||
| duration=float(EXPOSURE_INTERVAL+SLEW_INTERVAL), | |||
| totalCheckpoints=1, | |||
| private_sndStamp=data_id.records["exposure"].timespan.begin.unix_tai, | |||
| private_sndStamp=data_id.records["exposure"].timespan.begin.unix_tai-60, | |||
There was a problem hiding this comment.
This script also defines EXPOSURE_INTERVAL and SLEW_INTERVAL.
python/tester/utils.py
Outdated
| @@ -220,7 +220,7 @@ def send_next_visit(url, group, visit_infos): | |||
| f"filters: {info.filters} ra: {info.position[0]} dec: {info.position[1]} " | |||
| f"instrument: {info.instrument} survey: {info.survey}") | |||
| records_level = dict(value=asdict(info)) | |||
| value_schema_level = dict(value_schema_id=97, records=[records_level]) | |||
| value_schema_level = dict(value_schema_id=99, records=[records_level]) | |||
There was a problem hiding this comment.
Given your later instructions for updating it, I'd recommend moving this to a global constant so that it's easy to find.
hsinfang
force-pushed
the
tickets/DM-42618
branch
2 times, most recently
from
4915a46
to
05f1c3a
Compare
DM-38635 adds the expected start time to the nextVisit event.
A new schema was added to the schema registry for test.next-visit, which is used by dev testers. This new schema has the "startTime" field that started appearing in the summit nextVisit events since Cycle 35.
Currently the timestamps in the testers' simulated nextVisit events are just old timestamps. Although they aren't used yet, for self-consistency, offset the event publication time to be before the exposure start time.
hsinfang
force-pushed
the
tickets/DM-42618
branch
from
05f1c3a
to
c59f0f6
Compare
When the schema changes, this ID needs to be update. Make it a global constant so it's easier to find.
hsinfang
force-pushed
the
tickets/DM-42618
branch
from
c59f0f6
to
faed917
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.