-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DM-43586: Add versioning to the fits module #32
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
c1e374d
Add the first FILE_FORMAT_VERSION number
arunkannawadi 851d588
Write FILE_FORMAT_VERSION to the header
arunkannawadi c35afa1
Add a class method to check compatibility
arunkannawadi 363b77c
Add unit tests for isCompatibleWith
arunkannawadi 63a3374
Add a module level documentation
arunkannawadi ddf98fc
Define a custom IncompatibleVersionError
arunkannawadi df14b72
Check for compatibility before loading the data
arunkannawadi 20f51ec
Start keeping track of changes
arunkannawadi 46e10b4
Add changelog for DM-43516
arunkannawadi e5c4fdc
Add changelog for the current ticket
arunkannawadi 03ddb21
Add a PR template
arunkannawadi File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
## Checklist | ||
|
||
- [ ] ran Jenkins | ||
- [ ] added a release note for user-visible changes to `doc/changes` | ||
- [ ] updated the FILE_FORMAT_VERSION number correctly (if `python/lsst/cell_coadds/_fits.py` was modified) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Allowed `inputs` argument when constructing a `SingleCellCoadd` instance to be any iterable, not just a frozenset. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Included information about `day_obs` and `physical_filter` in `ObservationIdentifiers` so those metadata are available when reading a file. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Started including a semantic version number for the file format. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
Recording Changes | ||
================= | ||
|
||
This directory contains "news fragments" which are small files containing text that will be integrated into release notes. | ||
The files can be in restructured text format or plain text. | ||
|
||
Each file should be named like ``<JIRA TICKET>.<TYPE>`` with a file extension defining the markup format. | ||
The ``<TYPE>`` should be one of: | ||
|
||
* ``feature``: New feature | ||
* ``bugfix``: A bug fix. | ||
* ``api``: An API change. | ||
* ``perf``: A performance enhancement. | ||
* ``doc``: A documentation improvement. | ||
* ``removal``: An API removal or deprecation. | ||
* ``other``: Other Changes and Additions of interest to general users. | ||
* ``misc``: Changes that are of minor interest. | ||
|
||
An example file name would therefore look like ``DM-30291.misc.rst``. | ||
|
||
You can test how the content will be integrated into the release notes by running ``towncrier --draft --version=V.vv``. | ||
``towncrier`` can be installed from PyPI or conda-forge. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
# This file is part of cell_coadds. | ||
# | ||
# Developed for the LSST Data Management System. | ||
# This product includes software developed by the LSST Project | ||
# (https://www.lsst.org). | ||
# See the COPYRIGHT file at the top-level directory of this distribution | ||
# for details of code ownership. | ||
# | ||
# This program is free software: you can redistribute it and/or modify | ||
# it under the terms of the GNU General Public License as published by | ||
# the Free Software Foundation, either version 3 of the License, or | ||
# (at your option) any later version. | ||
# | ||
# This program is distributed in the hope that it will be useful, | ||
# but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
# GNU General Public License for more details. | ||
# | ||
# You should have received a copy of the GNU General Public License | ||
# along with this program. If not, see <https://www.gnu.org/licenses/>. | ||
|
||
import unittest | ||
|
||
import lsst.utils.tests | ||
from lsst.cell_coadds import CellCoaddFitsReader | ||
from lsst.cell_coadds._fits import FILE_FORMAT_VERSION | ||
|
||
|
||
class FitsTestCase(lsst.utils.tests.TestCase): | ||
"""Class for testing FITS specific aspects. | ||
|
||
Some FITS specific aspects are also implemented in test_coadds.py | ||
""" | ||
|
||
def test_read_compatibility(self): | ||
"""Test that the isCompatibleWith method works.""" | ||
self.assertTrue(CellCoaddFitsReader.isCompatibleWith("0.1")) | ||
self.assertFalse(CellCoaddFitsReader.isCompatibleWith("12345.67")) | ||
|
||
# Check that the reader is compatible with itself. | ||
self.assertTrue(CellCoaddFitsReader.isCompatibleWith(FILE_FORMAT_VERSION)) | ||
|
||
|
||
class TestMemory(lsst.utils.tests.MemoryTestCase): | ||
"""Check for resource/memory leaks.""" | ||
|
||
|
||
def setup_module(module): # noqa: D103 | ||
lsst.utils.tests.init() | ||
|
||
|
||
if __name__ == "__main__": | ||
lsst.utils.tests.init() | ||
unittest.main() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's obvious how one maps semver to I/O versioning; I think we need to spell out what is and is not supported under major, minor, and patch version bumps (and we might find that those are not sufficiently granular for the guarantees we might want to make, if both forwards- and backwards-compatibility and read/write permutations are in play).
But I also think going to all that effort might be premature, and maybe a single-integer version would be better for now? The most important thing is getting some kind of number into the files so future code doesn't have to guess about what it's dealing with.
I also have to admit that I'm a little skeptical of the value of the GitHub Action and the policy of bumping the patch version for all changes. Is this something you've seen in other codebases?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm envisioning the semantic versioning to work in exactly the same way as it does for software. I suppose one assumption here is that the version with which we are reading a file is never older than the version used to write a file, since a write operation must preceed a read operation. An example of a major bump would be if we decide to use
ImageHDU
s instead ofBinTableHDU
s and drop the support for the latter. That would be a breaking change rendering the old files unreadable. Examples of minor bump would include this PR, supporting versions with or withoutVERSION
in the header and substituting a default value if not found for newer items likeday_obs
andphysical_filter
we just introduced. The changes in all of the three examples aren't fundamentally different - in one case we decide to drop support, and for the others we don't.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We still rely on the developer doing the due diligence in deciding the correct version after a change. This is also enforceable but much harder. We could also call
0.x.y
versions unstable in that we don't give any semver kind of guarantee until1.0.0
. Bumping minor versions for every change would be equivalent to just an integer numbering that you suggested. It seems wasteful to me to discard old data for any missing metadata. So this is definitely something we should support in production, even if not now.