Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove py27 CI jobs #24942

Merged
merged 59 commits into from Mar 17, 2019

Conversation

@h-vetinari
Copy link
Contributor

commented Jan 26, 2019

The start of a looooooooong process to get rid of the py2 compat code.

I kept most of the CI jobs that used to be 2.7 at the lower end of the various supported versions, with 2x 2.7->3.5, 1x 2.7->3.6 and only one real upgrade from 2.7->3.7 (first CI job for 3.7 & windows).

@jreback @TomAugspurger

h-vetinari added some commits Jan 26, 2019

@jreback

This comment has been minimized.

Copy link
Contributor

commented Jan 26, 2019

keep in mind that we likely won't merge anything PY2 removal for a month or so.

@h-vetinari

This comment has been minimized.

Copy link
Contributor Author

commented Jan 26, 2019

keep in mind that we likely won't merge anything PY2 removal for a month or so.

Which makes a lot of sense from a backports POV, but I didn't think about that. Guess I just couldn't/can't wait. ;)

@gfyoung gfyoung added this to the 0.25.0 milestone Jan 26, 2019

@codecov

This comment has been minimized.

Copy link

commented Feb 1, 2019

Codecov Report

Merging #24942 into master will increase coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #24942      +/-   ##
==========================================
+ Coverage   92.37%   92.37%   +<.01%     
==========================================
  Files         166      166              
  Lines       52420    52388      -32     
==========================================
- Hits        48423    48396      -27     
+ Misses       3997     3992       -5
Flag Coverage Δ
#multiple 90.8% <ø> (ø) ⬆️
#single 42.9% <ø> (+0.01%) ⬆️
Impacted Files Coverage Δ
pandas/io/clipboard/clipboards.py 28.23% <0%> (-2.36%) ⬇️
pandas/core/indexes/base.py 96.32% <0%> (-0.23%) ⬇️
pandas/core/reshape/merge.py 94.4% <0%> (-0.08%) ⬇️
pandas/core/indexes/multi.py 95.59% <0%> (-0.02%) ⬇️
pandas/core/indexes/range.py 97.36% <0%> (-0.01%) ⬇️
pandas/core/indexes/datetimes.py 96.27% <0%> (-0.01%) ⬇️
pandas/core/resample.py 97.23% <0%> (-0.01%) ⬇️
pandas/core/generic.py 96.63% <0%> (-0.01%) ⬇️
pandas/core/tools/timedeltas.py 98% <0%> (ø) ⬆️
pandas/core/tools/numeric.py 100% <0%> (ø) ⬆️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bb43726...b88b87d. Read the comment docs.

@codecov

This comment has been minimized.

Copy link

commented Feb 1, 2019

Codecov Report

Merging #24942 into master will decrease coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #24942      +/-   ##
==========================================
- Coverage   91.25%   91.25%   -0.01%     
==========================================
  Files         172      172              
  Lines       52976    52976              
==========================================
- Hits        48342    48341       -1     
- Misses       4634     4635       +1
Flag Coverage Δ
#multiple 89.82% <ø> (ø) ⬆️
#single 41.74% <ø> (ø) ⬆️
Impacted Files Coverage Δ
pandas/util/testing.py 88.98% <0%> (-0.1%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a61d823...fe83f86. Read the comment docs.

@jreback jreback referenced this pull request Feb 7, 2019

Closed

[WIP] remove ci testing for python 2 #25217

1 of 4 tasks complete
@h-vetinari
Copy link
Contributor Author

left a comment

@jreback
Pushing some changes based on your review. This PR contains some xfails that I wasn't able to solve so far, as well as two changes to regexes that would otherwise fail. If the goal is a passing CI, those are unavoidable.

@@ -2182,6 +2182,8 @@ def test_write_cells_merge_styled(self, merge_cells, ext, engine):
assert xcell_b1.font == openpyxl_sty_merged
assert xcell_a2.font == openpyxl_sty_merged

@pytest.mark.xfail(PY35 and not PY36, reason='only fails on Linux?',

This comment has been minimized.

Copy link
@h-vetinari

h-vetinari Mar 14, 2019

Author Contributor

Because I wasn't able to make this test work on PY35 regardless of the versions I tried.

@@ -11,8 +11,9 @@

from pandas._libs import (
algos as libalgos, groupby as libgroupby, hashtable as ht)
from pandas.compat import PY2, lrange, range
from pandas.compat.numpy import np_array_datetime64_compat
from pandas.compat import lrange, range

This comment has been minimized.

Copy link
@h-vetinari

h-vetinari Mar 14, 2019

Author Contributor

Again, if I don't fix the regex here, the tests will fail.

- hypothesis>=3.58.0
- isort
# satisfy fastparquet deps with conda

This comment has been minimized.

Copy link
@h-vetinari

h-vetinari Mar 14, 2019

Author Contributor

Removed. Remaining jobs with fastparquet:

ci/deps\azure-windows-36.yaml:9:  - fastparquet>=0.2.1
ci/deps\travis-36-doc.yaml:9:  - fastparquet>=0.2.1
ci/deps\travis-36.yaml:10:  - fastparquet>=0.2.1
@jorisvandenbossche

This comment has been minimized.

Copy link
Member

commented Mar 14, 2019

@h-vetinari can you point to the test errors for which you needed to add the additional xfails? (or do a new build without any of them) Ideally, we should investigate each of those in more detail why they are failing instead of just xfailing them (unless you already know the reason for each of them)

@h-vetinari

This comment has been minimized.

Copy link
Contributor Author

commented Mar 14, 2019

@jorisvandenbossche
The fails are distributed over the CI runs throughout the thread, but I've reverted them in the last commit (including the regex fixes @jreback wanted me to revert), so then there's a single place to inspect the failures. Also - who knows - maybe some of them resolved themselves...

@h-vetinari

This comment has been minimized.

Copy link
Contributor Author

commented Mar 15, 2019

@jorisvandenbossche @TomAugspurger @jreback
Please check out https://travis-ci.org/pandas-dev/pandas/jobs/506535600, it fails already in the single stage with the tests I would have xfailed. When I first started this PR I played around with bumping the versions of xlrd etc (even when unpinning pymysql/xlsxwriter), but that didn't solve things either.

Considering the fact that removing support for PY35 is already being talked about in #25725, I think a small handful of xfailed tests is not a big burden. Certainly those are less trivial to fix than for example the skips that have been added for regex fails (of which there about 30).

@jreback

This comment has been minimized.

Copy link
Contributor

commented Mar 15, 2019

as i said above pls start over and JUST remove the 2.7 jobs; since things are working now this will just work. after that you can do independent PR to add new / change things.

something is breaking and certainly needs fixing
but not here

@jreback

This comment has been minimized.

Copy link
Contributor

commented Mar 15, 2019

PY35 is not being removed at this time

@h-vetinari

This comment has been minimized.

Copy link
Contributor Author

commented Mar 15, 2019

@jreback: as i said above pls start over and JUST remove the 2.7 jobs; since things are working now this will just work. after that you can do independent PR to add new / change things.

Of course it's easier to just rip things out, but I actually went to the effort here of thinking of a good distribution for the CI, updating the jobs, and making them pass again.

That's also how the last numpy bump was made. Tbh, I feel these conditions are quite arbitrary, but well... I'll delete the former PY2 jobs later today, and post the fixes in a separate PR.

PY35 is not being removed at this time

I know that. But adding 6-7 PY35-specific xfails (for non-core functionality) aren't that severe IMO, considering the point in the lifecycle.

@jreback

This comment has been minimized.

Copy link
Contributor

commented Mar 15, 2019

@h-vetinari as I have explained many many times. PRs which do lots of things take way more time to review and are much harder to merge. It is the same case here.

@h-vetinari

This comment has been minimized.

Copy link
Contributor Author

commented Mar 15, 2019

@jreback
This is green now. If only it always were as easy as deleting CI jobs, haha.

@jreback
Copy link
Contributor

left a comment

just the 1 comment

Show resolved Hide resolved doc/source/whatsnew/v0.25.0.rst
@h-vetinari
Copy link
Contributor Author

left a comment

Sorry, somehow didn't get a notification for the first comment.

Show resolved Hide resolved doc/source/whatsnew/v0.25.0.rst

@jreback jreback merged commit 707c720 into pandas-dev:master Mar 17, 2019

8 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
pandas-dev.pandas Build #20190315.8 succeeded
Details
pandas-dev.pandas (Checks_and_doc) Checks_and_doc succeeded
Details
pandas-dev.pandas (Linux py36_locale_slow) Linux py36_locale_slow succeeded
Details
pandas-dev.pandas (Linux py37_locale) Linux py37_locale succeeded
Details
pandas-dev.pandas (Linux py37_np_dev) Linux py37_np_dev succeeded
Details
pandas-dev.pandas (Windows py36_np14) Windows py36_np14 succeeded
Details
pandas-dev.pandas (macOS py35_macos) macOS py35_macos succeeded
Details
@jreback

This comment has been minimized.

Copy link
Contributor

commented Mar 17, 2019

thanks, would also appreciate a PR against : https://github.com/Macpython/pandas-wheels (the daily branch, not master).

@h-vetinari

This comment has been minimized.

Copy link
Contributor Author

commented Mar 17, 2019

thanks, would also appreciate a PR against : https://github.com/Macpython/pandas-wheels (the daily branch, not master).

This same PR or the PY35 one?

@jreback

This comment has been minimized.

Copy link
Contributor

commented Mar 17, 2019

neither. one to remove the PY2 jobs only (this is where we build wheels), and is our daily testing.

@WillAyd WillAyd referenced this pull request Mar 18, 2019

Closed

Build Failures #25762

hksonngan pushed a commit to hksonngan/pandas that referenced this pull request Mar 18, 2019

@h-vetinari h-vetinari deleted the h-vetinari:drop_py2_ci branch Mar 25, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.