Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.3 release planning #339

Closed
rabernat opened this issue Nov 20, 2018 · 28 comments
Closed

2.3 release planning #339

rabernat opened this issue Nov 20, 2018 · 28 comments
Milestone

Comments

@rabernat
Copy link
Contributor

We have a new feature--consolidated metadata--that some of us (me!) are eager to get into production.

Zarr releases quite infrequently compared to other OS projects I'm involved in. (Our last release was 8 months ago.) The list of things to fix for 2.3 contains some hard items which potentially involve revising the spec. @alimanfoo & @jakirkham, are all of the things on that list "must haves" for a new release? Or perhaps we might consider a more frequent and less ambitious release cycle?

Or maybe, although this is not a bugfix, we do a micro release?

@alimanfoo
Copy link
Member

alimanfoo commented Nov 20, 2018 via email

@jakirkham
Copy link
Member

Well personally I try not to release before going on vacation (US Thanksgiving). 😉 That said, not opposed to moving towards a release here.

Would be nice if we could get Numcodecs out the door first though as we've been working towards a release there recently and are pretty close.

Perhaps as a good first step towards a Zarr release it would be nice to have some testing of Zarr currently (particularly new features). Also it would be good to re-triage the 2.3 milestone and look to see what are critical bugs (need fixes) vs. cosmetic things (nice to haves). We can probably re-milestone the latter (unless they are particularly important to us).

@alimanfoo alimanfoo changed the title release cycle 2.3 release planning Nov 20, 2018
@alimanfoo
Copy link
Member

Took the liberty to rename this issue "2.3 release planning", suggest we use issue this to discuss what remaining work is in scope for 2.3.

@alimanfoo
Copy link
Member

Agreeing with @jakirkham, in an ideal world it would be nice to get numcodecs 0.6 out first, then finalise and release zarr 2.3.

Part of the reason for this is that there's a fix in the new version of numcodecs for a bug in the current msgpack codec, which is currently worked around in the zarr tests (in fact the zarr tests assert the broken behaviour). So updating numcodecs would enable the zarr tests to get fixed, and this would also unlock a problem in another piece of work in zarr. See zarr-developers/numcodecs#98 and #324 for details.

However, the two releases are not strictly tied, so if there was pressure to release zarr and we get blocked on numcodecs, we could be flexible.

FWIW for Zarr 2.3 there are three bugs would be good to fix: #263, #272 and #253. Two of these (#272, #263) have a mature PR, the other (#253) hasn't been worked on yet but shouldn't be too complicated. The other issues currently tagged in the 2.3 milestone I could live with postponing.

@jakirkham
Copy link
Member

Some issues cropped up related to NaT values recently. ( #342 ) Expect this is pretty important to fix. So have given it a go. The fix involves changes in Numcodecs and Zarr. Fortunately they appear to require only a light touch. PRs are ready for review.

@rabernat
Copy link
Contributor Author

Thanks for entertaining my suggestion. I'm not saying this is super urgent, just that we might want to consider a release whenever we get a significant new feature or bug fix. Sounds like that is doable.

We are planning a bunch of pangeo demos in early December. So it would be great to have the release before then. But we can always set up environments that use the latest github master, so again, not super urgent.

@alimanfoo
Copy link
Member

alimanfoo commented Nov 22, 2018 via email

@jakirkham
Copy link
Member

At this point, I think we just need more eyes on the PRs for v2.3. Nearly all of them already pass CI, but need a review and/or decision.

@jakirkham
Copy link
Member

jakirkham commented Feb 20, 2019

Alright, I think we should start thinking about the 2.3 release again. My suggestion would be the following remain on the milestone and everything else gets pushed off until the next release.

These all have PRs and AFAICT could be merged and then released today if we wanted to.

The other issues are about tricky Jupyter UI issues, changing getsize's behavior, supporting pathlib, and handling read-only stores through some APIs respectively. All of these will require more work and more thought. So I'd suggest that they be punted on and handled in a later release.

Thoughts @zarr-developers/core-devs?

Edit: Should add everything in @alimanfoo's scoped down list is either included in this or resolved except for getsize.

@rabernat
Copy link
Contributor Author

I favor a release asap. Xarray 0.12.0 is coming soon (pydata/xarray#2776) which has several zarr-related features (e.g. consolidated metadata). It would be nice if these happened around the same time.

@dazzag24
Copy link

Not sure if I'm too late to the party, but I'd love to see

make it into the release.

@jakirkham
Copy link
Member

Personally I'd also like to see PR ( #309 ) integrated into the release as there are some people internally that are eagerly awaiting it. That said, I'd be ok sticking with my previous list (without any additions) to allow a faster turnaround.

@alimanfoo
Copy link
Member

alimanfoo commented Feb 24, 2019 via email

@jakirkham jakirkham added this to the v2.3 milestone Mar 19, 2019
@jakirkham
Copy link
Member

Bumping this again. What are our thoughts on making a release?

@rabernat
Copy link
Contributor Author

rabernat commented Mar 20, 2019

I think we are all good to go for a release. All the PR that have been brought up in the discussion have been merged. Xarray 0.12 just came out. Let's go!

@alimanfoo
Copy link
Member

Thanks so much @jakirkham for keeping up the momentum on this. I am all in favour of making a release. Would you like to do the honours?

@jakirkham
Copy link
Member

Sure happy to move this forward. Will update if there are any issues.

@jakirkham
Copy link
Member

Ok, ran into an issue releasing on conda-forge. ( #419 ) Have put up PR ( #420 ) to fix it. If someone can take a look, that would be great. If not, I can push ahead with that minor fix.

@jakirkham jakirkham modified the milestones: v2.3, v2.4, v2.3.1 Mar 23, 2019
@jakirkham
Copy link
Member

Packages for 2.3.1 are now available on PyPI and conda-forge.

@rabernat
Copy link
Contributor Author

Thanks so much @jakirkham for dealing with the release friction! This is thankless but important work!

@alimanfoo
Copy link
Member

Thanks also from me, fantastic to see this go out, 🍻

@jakirkham
Copy link
Member

Thanks all for contributing! This is a very nice release 😄

@rabernat
Copy link
Contributor Author

Would anyone care to write a very short blog post about the release? This could go on Alistair's blog or we would be happy to host it on the Pangeo medium blog. Then we will have something to tweet about. :)

@alimanfoo
Copy link
Member

A blog post would be very cool. I started sketching something, contributions/comments very welcome:

https://docs.google.com/document/d/14qvMzK3HL20E63BqI3Q5EG_imoJ7h7pXhKBc_vVxnfY/edit?usp=sharing

@jakirkham you were the mastermind of this release, do you have a blogging channel you like to use?

@jakirkham
Copy link
Member

No strong preferences on where we post it. Maybe it makes sense to have a blog for the Zarr project itself?

@alimanfoo
Copy link
Member

No strong preferences on where we post it. Maybe it makes sense to have a blog for the Zarr project itself?

That would be good. Use medium (like pangeo)? Vanilla jekyll site on github pages (i.e., zarr-developers.github.io)? I think I'd prefer jekyll and github pages just because it might be a bit easier to collaborate on writing blog posts via PRs, but no strong preference.

Btw I have two working days before going on leave for 3 weeks, so have a short window in which I could help. Also very happy for someone else to run with this if it's not done before I go away.

@jakirkham
Copy link
Member

If you are comfortable setting up the framework for the repo, that would be very helpful.

@alimanfoo
Copy link
Member

I've created a new repo https://github.com/zarr-developers/zarr-developers.github.io which holds source for the blog served at https://zarr-developers.github.io. Creating a post just needs a new markdown file in the _posts folder, like the example one. If I get a chance today I'll make an initial PR to bring in the content from the google doc but am unlikely to be able to complete this before going on leave so please feel free to push forward without me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants