Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add table to track observations run through RTP #538

Merged
merged 23 commits into from
Mar 18, 2021
Merged

Conversation

plaplant
Copy link
Member

@plaplant plaplant commented Mar 4, 2021

This PR adds a new table (RTPLaunchRecord) which keeps track of observations that have been run through RTP. So far the PR defines the new table, and methods to add records and update them. Still needed are scripts to do this from the command line, as well as a related cron job to automatically launch jobs for recent observations. Scripts for adding/updating entries and launching RTP have been added in a834d77 and 0a57a8e.

There are also some incidental changes around deprecated numpy datatypes, which seemed like such a small change that I lumped them into this PR.

Closes #527.

@codecov
Copy link

codecov bot commented Mar 4, 2021

Codecov Report

Merging #538 (dbc80c4) into main (624ec36) will increase coverage by 0.07%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #538      +/-   ##
==========================================
+ Coverage   96.69%   96.77%   +0.07%     
==========================================
  Files          34       34              
  Lines        4787     4893     +106     
==========================================
+ Hits         4629     4735     +106     
  Misses        158      158              
Impacted Files Coverage Δ
hera_mc/mc_session.py 98.43% <100.00%> (+0.06%) ⬆️
hera_mc/rtp.py 100.00% <100.00%> (ø)
hera_mc/weather.py 54.80% <100.00%> (ø)
hera_mc/geo_sysdef.py 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 624ec36...dbc80c4. Read the comment docs.

@plaplant plaplant marked this pull request as ready for review March 10, 2021 05:13
scripts/mc_launch_rtp.py Outdated Show resolved Hide resolved
scripts/mc_launch_rtp.py Outdated Show resolved Hide resolved
scripts/mc_rtp_launch_record.py Outdated Show resolved Hide resolved
scripts/mc_rtp_launch_record.py Show resolved Hide resolved
@bhazelton
Copy link
Member

We also need to update the M&C definition tex doc to include this new table.

@dannyjacobs
Copy link
Contributor

@plaplant We're reviewing this in the commissioning meeting to try and get it online soon as we can. Some questions that came up:

  1. How do new observations get added to this table? Does the correlator do it at the same time it adds entries to the main "obsid" table?
  2. How do we make sure that we don't launch RTP on nights that haven't finished being recorded yet?

@plaplant
Copy link
Member Author

@dannyjacobs to answer your questions:

  1. The idea is that the mc_rtp_launch_record.py script will be called by the correlator after the mc_add_observation.py script (which adds a given observation to M&C).
  2. Right now my plan is "run the RTP launch script well after data taking is finished". It's not a perfect system, but I'm not sure there's a better solution. (The previous redis-based keys method was brittle in cases where the correlator didn't shut down gracefully. Perhaps we could come up with a better solution after the current observing run.)

I'll address the other comments on this PR later today.

@bhazelton
Copy link
Member

@plaplant Are we going to have a daemon on the commissioning repo that calls the launch script once per day at the right time? or is it going to be a cronjob?

@dannyjacobs
Copy link
Contributor

I've added a rough description of this change to http://hera.pbworks.com/w/page/118389669/Real-Time%20Processor. The script and table names are not correct, but its a start.

Copy link
Member

@bhazelton bhazelton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me.

@bhazelton
Copy link
Member

I'm merging this now, but not installing on site yet. @plaplant will do that when he has time (remember to update/restart all the things listed in the readme!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

make a table to track RTP starting
3 participants