New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
allow fiberassign --stdstar duplicates with --mtl #202
Conversation
Hmm; a bunch of Travis test failures, but tests pass at NERSC. Newer fitsio is first suspicion since it made some non-backwards compatible changes. |
fitsio 1.0.x switched to reading string table columns as unicode strings instead of raw bytes, requiring tests like science_rows = np.where(fiber_data["DEVICE_TYPE"] == b"POS")[0] to be changed to science_rows = np.where(fiber_data["DEVICE_TYPE"].astype(str) == "POS")[0] to work with both 0.9.x and 1.0.x Now I'm getting intermittent unicode parsing crashes from fitsio, e.g.
I'm now trying to catch it in the act and cache the problematic test file to see if it is a problem with fitsio or with something we're writing. But it doesn't fail every time the tests run... |
Unicode parsing errors are our fault. assign.py line 859 creates an empty table to fill with fiber assignments:
That includes a BRICKNAME string column that never gets initialized, and thus randomly sometimes contains non-ASCII characters which aren't allowed in FITS. For this particular case, I think replacing it with |
FYI I'm proceeding with my proposed replacement of |
For the record: I handled the uninitialized variables case, but I didn't add the (slow) code to fill in BRICKNAME in this PR. |
This PR fixes #198 by allowing fiberassign to have duplicate targets in the
--stdstar
file and the--mtl
. It also exposures the fba_run options--sciencemask
,--stdmask
, etc. to fiberassign.Background: Legacy
fiberassign
allowed duplicate std/mtl targets and it worked due to how it did internal bookkeeping. The refactoredfba_run
does not allow duplicate targets, but that broke some of the ways we had been using it.This PR: If all
--stdstar
targets are already in MTL, it just ignores the file. If it has some unique targets, those are filtered out and written to a separate temporary file before passing on tofba_run
. This allows users to continue to usefiberassign
in the same manner as before, without worrying about std/mtl duplicates.