Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need mechanism for updating file schema from previous versions. #36

Closed
dotsdl opened this issue Jul 1, 2015 · 5 comments
Closed

Need mechanism for updating file schema from previous versions. #36

dotsdl opened this issue Jul 1, 2015 · 5 comments

Comments

@dotsdl
Copy link
Member

dotsdl commented Jul 1, 2015

Upon loading an existing Container, the corresponding ContainerFile subclass should do a version check between the version given by the state file and the current version of MDS. It should then run code that updates the schema to that used by the current version of MDS.

This will require:

  1. an explicitly-documented schema spec for each ContainerFile subclass, which can change each release
  2. a mechanism for performing iterative updates to existing files; in other words, a file that was made with a very old version of MDS gets schema updates for each version of MDS it is behind, until it reaches the current one
  3. the mechanism to update files needs to be stress-tested with its own set of unit tests; it should be robust enough to restart conversion of a file that is only half-converted to a new version, which might happen if the python session dies mid-conversion.
@dotsdl dotsdl added this to the 0.5.1 milestone Jul 1, 2015
@dotsdl
Copy link
Member Author

dotsdl commented Jul 28, 2015

For example, between 0.5.0 and the upcoming 0.5.1, the way selections are stored has changed from a Table to an Array (in PyTables parlance). This has changed the way the selection getters work for the SimFile such that selections stored the old way no longer work. The updater code needs to check for old-style selections and update them, leaving new-style selections alone. Only after it has finished everything should it update the version number of the statefile.

@dotsdl
Copy link
Member Author

dotsdl commented Nov 14, 2015

Moving this to the next release. Since the splitting out of datreant from mdsynthesis the versioning system itself has changed, so it would be a terrible hack to try and make this work for the pre-release schema. Each Sim now has a datreant version and a separate mdsynthesis version, giving the pieces needed for the update scheme to work in the next release.

The restructuring of the package makes room for the machinery we will need to make a smooth version updater, so for now we are good to go.

@dotsdl dotsdl modified the milestones: 0.7.0, 0.6.0 Nov 14, 2015
@orbeckst
Copy link
Contributor

I think you can safely down-grade the priority, given that there isn't a full release yet.

@dotsdl
Copy link
Member Author

dotsdl commented Jan 11, 2016

@orbeckst I think so. :D

@dotsdl
Copy link
Member Author

dotsdl commented Mar 7, 2016

Closing for now. Not really necessary at the moment, and not having some checking every time a Sim is init-ed keeps them cheap to make.

@dotsdl dotsdl closed this as completed Mar 7, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants