Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tile QA simplified #1317

Merged
merged 18 commits into from Jun 18, 2021
Merged

Tile QA simplified #1317

merged 18 commits into from Jun 18, 2021

Conversation

julienguy
Copy link
Contributor

Implemented a "simplification" of the tile QA.
(after discussion with Stephen and David)

 - exposure_goodfiber = (QAFIBERSTATUS&fibermask.mask(bad_qafstatus_mask)==0) , per fiber and exposure
 - tile_fiber_efftime = sum(efftime[exposure_goodfiber]) , per fiber
 - tile_goodfiber = ( tile_fiber_efftime > 0.9*mintfrac*goaltime ) ,  (param fiber_rel_mintfrac = 0.9) , per fiber
 - tile_always_goodfiber = OR(exposure_goodfiber) , per fiber 
 - tile_efftime = median(tile_fiber_efftime[tile_always_goodfiber]) , per tile
 - tile_is_valid = ( tile_efftime > mintfrac*mintfrac*goaltime ) & ( sum(tile_goodfiber) > 3000 ) , (param  min_number_of_good_fibers = 3000)

Automatic validation ('VALID' in header of tile_qa.....fits), can be overridden by visual inspector (that always
has to validate). Overridden entries are noted (if someone wants to double check).

@coveralls
Copy link

coveralls commented Jun 18, 2021

Coverage Status

Coverage decreased (-0.07%) to 26.445% when pulling da35617 on tile_qa_simplified into 8a4e68a on master.

@julienguy
Copy link
Contributor Author

Example

desi_tile_vi --prod daily --qa . --night 20210514:20210618 -o mytiles-new.csv -u jguy --viewer display
...
Showing tiles/cumulative/20240/20210514/tile-qa-20240-20210514.png ...
The code thinks it's a valid tile
PROMPT: Is tile 20240 a valid tile? (y/n/save/exit/pass/help)

tile-qa-20240-20210514

>>> n
USER ENTERED >>> n
Your have overridden the automated validation.

@julienguy
Copy link
Contributor Author

(I actually agree it is a valid tile :-)

@sbailey
Copy link
Contributor

sbailey commented Jun 18, 2021

I did several small updates, but overall looks good. I generated tile QA for all main survey tiles in /global/cfs/cdirs/desi/users/sjbailey/test/tiles/cumulative . Let's vet those a bit more and then we can merge and rerun for the real daily prod (it takes ~10 min with --nproc 32 on an interactive node).

One desi_tile_vi hiccup: the input csv file has default values for QA and USER and "none" so they get read in as a 4-character table, which works fine for "jguy" but longer names like "stephen" get truncated to "step".

In desihub/desitarget#755 @geordie666 expects the new input to be tiles-specstatus.ecsv which gives the opportunity to define the column types of the ascii table to allow more characters, but requires a bit of bootstrapping to get it right. A topic for tomorrow...

@sbailey sbailey merged commit 368dd50 into master Jun 18, 2021
@sbailey sbailey deleted the tile_qa_simplified branch June 18, 2021 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants