Skip to content

Conversation

@kfindeisen
Copy link
Member

This is a running PR to test Prompt Processing fixes and the BASE_TAG_LIST system. It uses a different branch name from #97 to ensure that DM-41717-d_2023_11_14 is not overwritten.

The HTMs are difficult to derive by hand from the next_visit metadata,
but are essential to debugging refcat loading problems.
The tract and patch are difficult to derive by hand from the nextVisit
metadata, but are essential to debugging template loading problems.
Copy link
Collaborator

@hsinfang hsinfang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Those extra debug logs will be very useful.

private_seqNum: int = 0 # queue-specific counter. Not the same as Butler/RubinTV seq_num
private_rcvStamp: float = 0.0
private_sndStamp: float = 0.0
private_sndStamp: float = 0.0 # time of visit publication; TAI in unix seconds
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to dig out again what these private_* fields mean and thinking whether adding more info would be useful. Just dumping them here

Sasquatch

SAL timestamps are added automatically by SAL and have the private_ prefix. In particular, the private_sndStamp timestamp is the time at which the message was published by the CSC in TAI, which should be very close to the time at which the telemetry was measured. In fact, many telemetry topics rely on the private_sndStamp timestamp for that.
The private_sndStamp timestamp is also converted to UTC and recorded in the private_efdStamp field, which is used as the time index in the EFD.

slack thread

private_sndStamp, which should be the TAI instant when the SAL message was issued from the CSC, presumably only a short time after the actual event.
private_efdStamp should be the same value in UTC

And slack thread

private_sndStamp ... is the time the sample was published.. this is TAI but is converted to UTC (and stored in private_efdStamp
most topics will rely on private_sndStamp as also being the time the measurement was performed..

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading them again, I think what you've put in the comments are perfect summary for prompt processing context.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I used the Sasquatch page as my source.

Though given the example of private_seqNum, we should probably avoid using the private_ fields if possible. 😬

Many fields have different meanings from Butler or RubinTV fields of
the same names; explain the known differences.
@kfindeisen kfindeisen marked this pull request as ready for review November 17, 2023 19:38
@kfindeisen
Copy link
Member Author

(Oops, apparently you can review a PR that's "not ready for review"!)

@kfindeisen kfindeisen merged commit e6d2e18 into main Nov 17, 2023
@kfindeisen kfindeisen deleted the tickets/DM-41717b branch November 17, 2023 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants