Daily Housekeeping

[wtgee]

Determine tasks that need to be accomplished during the Housekeeping state, which is run after the unit is parked. This should either be run directly after parking or at some set time each morning.

• Cleanup observations directory
• Upload images to cloud storage
• Upload metadata to (TBD) cloud.

[jamessynge]

I'm looking at setting up some code to run PIAA analysis on an observation, which would be easier if the last file uploaded for an observation was a sentinel that indicates that the upload is complete. This could be a metadata file, or just an empty file (e.g. UPLOAD-COMPLETE).

[jamessynge]

Closing the issue right after I added a feature request, without a comment as to why, leaves me wondering whether the feature request is already recorded elsewhere, is already handled, or you disagree. Could you elaborate?

…

On Tue, Apr 24, 2018, 11:38 PM Wilfred Tyler Gee ***@***.***> wrote: Closed #42 <#42>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABYmU5siMOKJGLKBPdvF6Fic6XooJrCKks5tr--8gaJpZM4JBEqX> .

[wtgee]

Sorry, I didn't mean to close it. In fact, I had made an earlier comment and had meant to post it but guess I screwed it up on my mobile.

Anyway, yes, I have been thinking something along these lines. Because they can be uploaded in any order I almost think it would need to be a metadata file that lists the names of the files that should have been uploaded. If we wanted to be really sure we could do md5sums or something.

I have just barely got it set up so that it responds to the notifications when something gets uploaded. The idea is that this will then do a plate-solve on the file and ingest metadata from the FITS header into a postgresql database. We can then use this metadb for displaying a site of info.

[jamessynge]

Thanks for the clarification. My goal is somewhat simpler: trigger the light curve production once all files of an observation have been uploaded. As you say, a metadata file can include the names, and maybe the signatures, of the raw files, so a responder can listen for bucket change events, and only if the metadata file and all referenced files are present would it kick off the analysis.

…

On Wed, Apr 25, 2018, 4:45 PM Wilfred Tyler Gee ***@***.***> wrote: Sorry, I didn't mean to close it. In fact, I had made an earlier comment and had meant to post it but guess I screwed it up on my mobile. Anyway, yes, I have been thinking something along these lines. Because they can be uploaded in any order I almost think it would need to be a metadata file that lists the names of the files that should have been uploaded. If we wanted to be really sure we could do md5sums or something. I have just barely got it set up so that it responds to the notifications when something gets uploaded. The idea is that this will then do a plate-solve on the file and ingest metadata from the FITS header into a postgresql database. We can then use this metadb for displaying a site of info. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABYmU064VtFVaCmG7M9MYKVVNsl9X-tvks5tsOBigaJpZM4JBEqX> .

[wtgee]

Always happy to close a 2 year old Issue. :) Closed by #687.