Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move mock observing updates into master #648

Merged
merged 35 commits into from Jun 29, 2018
Merged

Conversation

rstaten
Copy link
Contributor

@rstaten rstaten commented Jun 28, 2018

This PR:

  • Adds functionality to map from new to old fibermap format (via @sbailey)
  • Adds expid to raw/fibermap path
  • Pythonically opens/closes configuration file
  • Consolidates QL findfile arguments
  • Adds output plot of sframe spectra (one spectrum per target type)
  • Updates deltamag calculation to include all fibers
  • Uses linalg if cholesky fails for fiber flats
  • Prints status result per QA

@sbailey
Copy link
Contributor

sbailey commented Jun 29, 2018

As a heads up, the disruptive change will be that desispec.io.findfile will look for fibermaps and desi* raw data in per-exposure directories instead of per-night directories. To read previously simulated data after this PR we will need to add simlinks (thanks to @julienguy for the suggestion). But the per-exposure subdirs are what ICS will provide us so we need to move in that direction.

Related: #384 to write simulated data in the new subdirs.

@rstaten has a few more updates that we might get in tonight so not merging just yet...

@sbailey
Copy link
Contributor

sbailey commented Jun 29, 2018

For the record, other changes coming in with this PR:

  • separate QL config files into a $QL_CONFIG_DIR so that they can be updated independently of the code installation
  • temporarily get PSF and fiberflat reference files from $QL_CALIB_DIR until we converge with @julienguy about formats for those in $DESI_CCD_CALIBRATION_DATA svn repo
  • Add file locking when reading the QL config file. For reasons we don't understand, this appears to be necessary when running with docker+NFS at KPNO. Without the locks, sometimes the config file comes back as an empty string (!). We haven't been able to reproduce the problem in separate tests. This PR uses the filelock library, but it is implemented as an optional lock so that you don't have to have it installed.

Tests pass on my laptop; we'll see if Travis agrees...

@sbailey
Copy link
Contributor

sbailey commented Jun 29, 2018

Tests pass; proceeding with merging. Open new issues if this breaks something.

Note: I'm working on tests for companion PR desihub/desisim#384 which will be needed before the full simulation + data chain can work together again.

@sbailey sbailey merged commit c1a7ed1 into master Jun 29, 2018
@sbailey sbailey deleted the ql_TagUpdates_0.21.0 branch June 29, 2018 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants