Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raw data updates #151

Merged
merged 35 commits into from Apr 10, 2023
Merged

Raw data updates #151

merged 35 commits into from Apr 10, 2023

Conversation

weaverba137
Copy link
Member

@weaverba137 weaverba137 commented Mar 22, 2023

This PR primarily updates the data model for the raw data files, desi-EXPID.rst. Remaining items before merging:

  • High-level description of the SPEC HDU.
  • Clean up other raw data model files as much as possible.
  • Verify against raw data files that are already in the public area for release.

The desi-EXPID.rst file can probably be reviewed as-is, and other files primarily have structural changes.

The changes to the code allow the data model verification script to select multiple candidate files for comparison, rather than just the first one it finds.

A few new data model files have been added to reduce the number of extraneous files detected in the raw data.

@weaverba137 weaverba137 added the WIP Work in Progress label Mar 22, 2023
@weaverba137 weaverba137 self-assigned this Mar 22, 2023
@coveralls
Copy link

coveralls commented Mar 22, 2023

Coverage Status

Coverage: 100.0%. Remained the same when pulling 4299264 on raw-data-updates into 5dd8546 on main.

@weaverba137
Copy link
Member Author

@sbailey, I think you could start by reviewing desi-EXPID.rst now while I work on other parts of this.

* main:
  Update change log
  adding type to WEIGHT clustering columns
  Adding more columns related to LSS. Remove some unused columns that had no description, and solve a minor bug in PROB_OBS column
  Adding last set of columns for the description of the LSS datamodel
* main:
  Add descriptions to SkyCam HDUs
@weaverba137
Copy link
Member Author

@sbailey, I'd like to merge this soon. I think I've got about as much information into the raw data model files as we're going to get.

@weaverba137 weaverba137 removed the WIP Work in Progress label Apr 10, 2023
Copy link
Contributor

@sbailey sbailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Merge when ready. I haven't reviewed every file in detail, but this is definitely a step forwards compared to current main, so let's merge and make any additional updates in future PRs if needed. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants