Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline Dashboard Improvements #1480

Merged
merged 15 commits into from Nov 11, 2021
Merged

Pipeline Dashboard Improvements #1480

merged 15 commits into from Nov 11, 2021

Conversation

akremin
Copy link
Member

@akremin akremin commented Nov 10, 2021

This combines a number of improvements I have built up on my "wish list" for the pipeline dashboard. The impetus was (2), which was causing the dashboard to not list some exposures as "processing" even though they were during the Oct 2021 rerun of post-shutdown data.

The main features/changes are:

  1. Linking to the correct log for sciences that fail in standard star fitting
  2. Determines what is processed based on processing table and directories in preproc
  3. Using white space better
  4. Moving TILEID next to EXPID
  5. Adding FA_SURV FAPRGRM
  6. Opening TILE fibersassign images in new tab
  7. Code improvements, including better usage of the pipeline tables and making the columns more readable/editable.

(1) was a long standing limitation that needed to be addressed. (2) became relevant during reprocessing Oct 2021 data when multiple reruns were done and no single processing table contained the nights full data, so some exposures weren't listed as processed even though they were. (3-6) are minor things that I feel add to the dashboard. (5)
The dashboard was getting quite wide, so to add the two new columns I reduced the width of others that had empty white space such that the total width isn't changed by much.

I tested the code in the directory: /global/cfs/cdirs/desi/users/kremin/spectro/redux/dash_arcs/dashboard/

A small example of the past two months was made using the last commit: https://data.desi.lbl.gov/desi/users/kremin/spectro/redux/dash_arcs/dashboard/dashboardlink.html

I had already run a test on all data from 20201214-20211108 and the code ran and displayed the expected results. That test was called "dashboardfull" in the log and html file. The only change after that was opening the fiberassign image in a new tab.

Finally, below I show a screenshot of an example of the new table format in the dashboard:
image

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.03%) to 26.245% when pulling 4236f6f on dash_arcs into 8d1959a on master.

@sbailey sbailey merged commit d1e8fb8 into master Nov 11, 2021
@sbailey sbailey deleted the dash_arcs branch November 11, 2021 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants