Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RLS: 1.1.4 #37397

Closed
simonjayhawkins opened this issue Oct 25, 2020 · 39 comments
Closed

RLS: 1.1.4 #37397

simonjayhawkins opened this issue Oct 25, 2020 · 39 comments
Labels
Release
Milestone
1.1.4

Comments

@simonjayhawkins
Copy link
Member

Tag Release

Tracking issue for the 1.1.4 release. Scheduled for release October 30, 2020

https://github.com/pandas-dev/pandas/milestone/78

Please do not remove/change milestones from these issues without a note explaining the reasoning (changing milestones doesn't trigger notification)

During issue triage, regressions from 1.0.5 onwards should be milestoned 1.1.4 in the first instance. cc @pandas-dev/pandas-triage
@simonjayhawkins simonjayhawkins added this to the 1.1.4 milestone Oct 25, 2020
@simonjayhawkins
Copy link
Member Author

release notes at https://pandas.pydata.org/pandas-docs/dev/whatsnew/v1.1.4.html

fix for #36951 not yet backported, see #36951 (comment)

@simonjayhawkins
Copy link
Member Author

wheel build status at MacPython/pandas-wheels#106

currently failing on 32bit builds and macOS

https://dev.azure.com/pandas-dev/pandas-wheels/_build/results?buildId=45995&view=logs&j=93056758-5bfb-5750-f113-e720ddefdb4c&t=5370d41f-ddc8-5ef9-ef54-c6555373fd14

@simonjayhawkins
Copy link
Member Author

other than backports, it looks like #37023 is the only open PR that is in the pipeline close to being ready.

https://github.com/pandas-dev/pandas/pulls?q=is%3Aopen+is%3Apr+milestone%3A1.1.4

@simonjayhawkins
Copy link
Member Author

fix for #36951 not yet backported, see #36951 (comment)

see #37305 (comment)

@phofl
Copy link
Member

phofl commented Oct 26, 2020

#37221? Or adding this to 1.2?

@simonjayhawkins
Copy link
Member Author

#37221? Or adding this to 1.2?

Thanks @phofl have milestoned 1.1.4 for now.

@jreback
Copy link
Contributor

jreback commented Oct 26, 2020

thanks @simonjayhawkins ping me in any blockers

@simonjayhawkins
Copy link
Member Author

currently failing on 32bit builds and macOS

for the failing 32bit test some have been addressed in #35898 and now that #35898 is backported should make it easier to address the remaining ones. #37466 (comment)

For the macOS failures. I've not yet made much progress yet, (never used a mac!) and may not get chance to circle back to this today. anyone with macOS experience is welcome to pick this up. cc @pandas-dev/pandas-core

For the actual release, have configured WSL 2 and should now be able to use the linux scripts directly, pandas-dev/pandas-release#33 (currently doing a dummy run)

@jbrockmendel
Copy link
Member

I'll take a look at the macOS builds; i'm in the same boat when it comes to the azure windows builds

@TomAugspurger
Copy link
Contributor

I added MacPython/pandas-wheels#107 for 3.9 builds with 1.1.3 and ran into the macOS build issue. Seeing if updating multibuild fixes it.

@simonjayhawkins
Copy link
Member Author

Thanks @TomAugspurger. once the macOS issue is resolved, I think only #37473 needs to be merged and backported and we should be good to release.

We had a regression in 1.1.3, #37094. There are no PRs to fix. If we are doing a 1.1.5 release this isn't necessarily issue. But if we want 1.1.4 to be the last in the 1.1.x series, we should maybe fix this before release.

I think probably best to assume a 1.1.5 release. (even if only a handful of fixes)

@jorisvandenbossche
Copy link
Member

@simonjayhawkins would you have to look into a possible fix for #37094? Otherwise I can try this evening. I agree it would be nice to fix this for 1.1.4, since it was a regression in 1.1.2->1.1.3.

@simonjayhawkins
Copy link
Member Author

I agree it would be good to fix since it is also affecting many users. Now that we are almost in a good state for release, I'll take a look this afternoon.

@simonjayhawkins
Copy link
Member Author

I think probably best to assume a 1.1.5 release. (even if only a handful of fixes)

I'll be creating a 1.1.5 milestone shortly and moving the open issues tagged 1.1.4 to 1.1.5 ( except #37094)

@simonjayhawkins
Copy link
Member Author

I'll be creating a 1.1.5 milestone shortly and moving the open issues tagged 1.1.4 to 1.1.5 ( except #37094)

OK starting this now. wheel build status at MacPython/pandas-wheels#106 is green except for test_slice_irregular_datetime_index_with_nan #37473 so good for release tomorrow.

@simonjayhawkins
Copy link
Member Author

I agree it would be good to fix since it is also affecting many users. Now that we are almost in a good state for release, I'll take a look this afternoon.

@jorisvandenbossche not sure how best to fix the root cause pandas.core.algorithms.isin (see #37094 (comment)) without possibly introducing another regression.

I was hoping to get started on the release first thing tomorrow. I won't get chance now to do a PR for #37094 this evening.

@jorisvandenbossche
Copy link
Member

@simonjayhawkins I put up a PR to fix #37094 (seems an additional dtype check to avoid that path for object dtype does the trick) -> #37499

@simonjayhawkins
Copy link
Member Author

I was going to change the MRE to put ser.isin({"foo", "bar", np.nan}) and check perf.

@simonjayhawkins
Copy link
Member Author

I was going to change the MRE to put ser.isin({"foo", "bar", np.nan}) and check perf.

I think we are taking quite a perf hit with the fix

205 ms ± 10.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) fix

9.64 ms ± 355 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) 1.0.5

@jorisvandenbossche
Copy link
Member

Can you compare with 1.1.2 ? As it might be the performance already decreased there as well (with the introduction of the isna check)

@jorisvandenbossche
Copy link
Member

Hmm, of course 1.1.2 didn't yet include the change, so it the same as 1.0.5.

Given how many people were affected by the regression, I think this is worth fixing even if it has a slowdow (certainly because the slowdown is only for a non-typical use case: checking if numeric values are in object dtype values).
Now, I slightly changed the fix, keeping that path for object dtype values. That should also solve the performance question.

@simonjayhawkins
Copy link
Member Author

just #37517 to merge and then can then start final pre-release checks

@simonjayhawkins
Copy link
Member Author

simonjayhawkins commented Oct 30, 2020 •
edited

@simonjayhawkins
Copy link
Member Author

For the actual release, have configured WSL 2 and should now be able to use the linux scripts directly, pandas-dev/pandas-release#33 (currently doing a dummy run)

Ran into a couple of issues with WSL. Memory leak and VScode server sync issue but otherwise tag locally, sdist build, pip tests, conda tests and documentation build went OK.

Will go with this environment for the release but proceed with caution. A few pros and cons vs the Docker release container. The Docker contaniner ensures a clean environment for release, but oth the WSL Ubuntu setup is more akin to to the release process used by linux and macOS users.

@simonjayhawkins
Copy link
Member Author

@jorisvandenbossche @jreback @TomAugspurger can someone with permissions restart linux py_3.9_64 on MacPython/pandas-wheels#106

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Oct 30, 2020 via email

@simonjayhawkins
Copy link
Member Author

Thanks Tom.

@simonjayhawkins
Copy link
Member Author

I'm going to change the order of tasks slightly. see #21050

have tagged locally, build the sdist and ran pip tests. conda test in progress. (pip and conda tests not really now necessary since already tested at https://github.com/simonjayhawkins/pandas-release/actions?query=workflow%3A%22Tag+Release%22 and MacPython/pandas-wheels#106)

will build documentation locally before pushing tag.

after pushing tag will upload documentation and make wheels

once wheels are built (and in the staging area) will then do the GitHub release (which will trigger notifications to users) and then upload wheels to PyPI

Hopefully, this will reduce the lag between notifications to users and wheels and docs being available.

@simonjayhawkins
Copy link
Member Author

after pushing tag will upload documentation

I'm using a different machine with different ssh keys. there will be a lag here this time while I sort that out.

@simonjayhawkins
Copy link
Member Author

docs uploaded https://pandas.pydata.org/pandas-docs/version/1.1.4/

@simonjayhawkins
Copy link
Member Author

@TomAugspurger @jreback @jorisvandenbossche can someone with permissions merge MacPython/pandas-wheels#108

@simonjayhawkins
Copy link
Member Author

@TomAugspurger @jreback @jorisvandenbossche can someone with permissions merge MacPython/pandas-wheels#108

Thanks Tom

@simonjayhawkins
Copy link
Member Author

I'll give it another 20mins and if the travis and restarted azure job on MacPython have not started will do the GitHub release and upload the build wheels so that I can start on the conda builds.

will then upload the missing wheels later.

@jreback
Copy link
Contributor

jreback commented Oct 30, 2020

@simonjayhawkins I added you as a maintainer to pandas-wheels.

@simonjayhawkins
Copy link
Member Author

Thanks Jeff.

@simonjayhawkins
Copy link
Member Author

I'll give it another 20mins and if the travis and restarted azure job on MacPython have not started

The azure job was showing queued but re-run and started immediately. The Travis job has now started.

@simonjayhawkins
Copy link
Member Author

All wheels are on PyPI (no 3.9 for aarch though)

1.1.4 docs are on stable

web site will update when ci on PR merged to master 20mins ago completes

conda-forge in progress conda-forge/pandas-feedstock#98

@jreback
Copy link
Contributor

jreback commented Oct 31, 2020

thanks @simonjayhawkins !

@jreback jreback closed this as completed Oct 31, 2020
@jorisvandenbossche
Copy link
Member

Thanks a lot Simon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Release
Projects
None yet
Milestone
1.1.4
Development

No branches or pull requests
6 participants
@jreback @jorisvandenbossche @TomAugspurger @jbrockmendel @simonjayhawkins @phofl