Py2 deprecation #2876

honnix · 2019-12-27T13:54:51Z

Description

This consolidates all PRs for python2 deprecation: https://github.com/spotify/luigi/pulls?q=is%3Apr+label%3A%22Python2+deprecation%22+is%3Aclosed

Motivation and Context

Open a PR to make final check easier.

Have you tested this? If so, how?

honnix · 2019-12-27T13:55:33Z

cc @drowoseque . Thanks for your great contribution!

dlstadther · 2019-12-28T21:40:38Z

Thanks @drowoseque and @honnix for managing and doing this deprecation!

drowoseque · 2020-05-21T08:49:47Z

@honnix or @dlstadther could you pls resolve conflicts and merge master in this branch to keep it up-to-date?

honnix · 2020-05-21T10:43:09Z

@drowoseque Sure. We might get this merged next week. Sorry for taking so long.

* py2/py3 import error eliminated * E402 module level import not at top of file * E303 too many blank lines (3)

* snamebite removed * no minicluster * no minicluster * flake8 * branches - master * no snakebite in tox.ini

* no itervalues * no exception

* Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * six.assertCountEqual -> self.assertCountEqual * no ununsed import * E128 continuation line under-indented for visual indent

* Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no six.next * yield from instead of loop * too-many-blank-lines

* Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * six.add_metaclass -> metaclass= * six.add_metaclass -> metaclass=

* Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no from __future__ import

* no string_types * no-binary_type * Update .travis.yml * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * original * no ununsed import * Update .travis.yml * Fix params hashing (#2540) * move params freezing to separated module * freeze dict for toml parser * FIX FrozenOrderedDict name * Freeze only params of config, not all config * +docstring * [luigi.mock.MockFile] removed (#2839) * no-mock-file * MockFile removed * unused import * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * new string_types * no string_types * no-binary_type * original * no ununsed import * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * new string_types * snakebite * no-unused-import

* Update .travis.yml * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no-six-moves * del attr removed * del attr removed * exec_ removed

* 2.7 mentions removed from README and setup.py * py27 -> py37 in CONTRIBUTING.rst

* Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * abstractproperty -> property + abstractmethod * Update beam_dataflow.py * Update beam_dataflow.py

* no-if-six.py* * no unused imports * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * postgres toxenv for py3 * postgres toxenv for py3 * postgres toxenv for py3 * \n * no-unused-import * test_multiple_workers back

* iteritems usage removed * no unused imports * bmo * bmo * iteritems removed * no doc for iteritems * most_common from collections.Counter * returning tuple * postgres toxenv for py3 * original * no-unused-import * F401 'luigi.six' imported but unused * blank line

* Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no six.{u,b} * joined * Update .travis.yml * "except ImportError" removed (#2845) * py2/py3 import error eliminated * E402 module level import not at top of file * E303 too many blank lines (3) * snakebite removed (#2833) * snamebite removed * no minicluster * no minicluster * flake8 * branches - master * no snakebite in tox.ini * [luigi.six.itervalues] usage removed (#2835) * no itervalues * no exception * [luigi.six.assertCountEqual] removed (#2841) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * six.assertCountEqual -> self.assertCountEqual * no ununsed import * E128 continuation line under-indented for visual indent * [luigi.six.next] removed (#2842) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no six.next * yield from instead of loop * too-many-blank-lines * `luigi.six.add_metaclass` removed (#2843) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * six.add_metaclass -> metaclass= * six.add_metaclass -> metaclass= * no `from __future__ import ...` (#2846) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no from __future__ import * [luigi.six.string_types] and [luigi.six.binary_type] removed (#2838) * no string_types * no-binary_type * Update .travis.yml * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * original * no ununsed import * Update .travis.yml * Fix params hashing (#2540) * move params freezing to separated module * freeze dict for toml parser * FIX FrozenOrderedDict name * Freeze only params of config, not all config * +docstring * [luigi.mock.MockFile] removed (#2839) * no-mock-file * MockFile removed * unused import * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * new string_types * no string_types * no-binary_type * original * no ununsed import * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * new string_types * snakebite * no-unused-import * utf-8 in auth * no unused imports * py27 -> py37 in CONTRIBUTING.rst * E231 missing whitespace after ',' * [luigi.six.moves] removed (#2844) * Update .travis.yml * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * no-six-moves * del attr removed * del attr removed * exec_ removed * py 2.7 mentions removed from README and setup.py (#2852) * 2.7 mentions removed from README and setup.py * py27 -> py37 in CONTRIBUTING.rst * abstractproperty -> property + abstractmethod (#2847) * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * abstractproperty -> property + abstractmethod * Update beam_dataflow.py * Update beam_dataflow.py * no-if-six.py* (#2840) * no-if-six.py* * no unused imports * Update .travis.yml * no py27 in travis (#2836) * no py27 in travis * py37 * flake8 and docs back * py36 py37 postgres * py36-apache * MixedUnicodeBytes removed * unused imports removed * no-if-py3 * space * space * MixedUnicodeBytesFormat returned * postgres toxenv for py3 * postgres toxenv for py3 * postgres toxenv for py3 * \n * no-unused-import * test_multiple_workers back * F401 'luigi.six' imported but unused * test_url_without_basic_auth fix * [luigi.six.iteritems] usage removed (#2834) * iteritems usage removed * no unused imports * bmo * bmo * iteritems removed * no doc for iteritems * most_common from collections.Counter * returning tuple * postgres toxenv for py3 * original * no-unused-import * F401 'luigi.six' imported but unused * blank line * F401 'luigi.six' imported but unused * six.b('...') -> b'...' * blank line

honnix · 2020-06-01T13:33:27Z

@dlstadther @Tarrasch Could you please approve this as well? I will get it merged right after.

dlstadther

drowoseque · 2020-06-01T15:17:38Z

luigi/__meta__.py

@@ -7,5 +7,5 @@
 __author__ = 'The Luigi Authors'
 __contact__ = 'https://github.com/spotify/luigi'
 __license__ = 'Apache License 2.0'
-__version__ = '2.8.13'
+__version__ = '3.0.0b2'


@honnix shouldn't we change version here prior to merge?

Good catch. We can release 3.0.0 immediately after merging to master. So it is OK in this PR I think.

drowoseque · 2020-06-01T15:27:27Z

@honnix
btw I've found that some new code handling ImportError has arrived after merging #2845
ie

luigi/freezing.py:7
luigi/tools/deps.py:48

I think we should remove it before merging
don't you mind a pr from me with eliminating that?

honnix · 2020-06-01T15:28:23Z

@drowoseque Sure, please go ahead.

honnix · 2020-06-01T15:32:16Z

luigi/freezing.py:7 is introduced for python 3.9 #2895

Actually they both are in the same PR. @drowoseque

drowoseque · 2020-06-01T15:37:34Z

okay, then no need for removal. You can go ahead :)

honnix added Big code change Affects core Breaking change Python2 deprecation labels Dec 27, 2019

honnix requested review from dlstadther and Tarrasch as code owners December 27, 2019 13:54

drowoseque approved these changes Dec 28, 2019

View reviewed changes

honnix force-pushed the py2-deprecation branch from 66410a7 to 587b6bc Compare January 2, 2020 12:49

arteymix mentioned this pull request Jan 19, 2020

Move to Python 3 PavlidisLab/rnaseq-pipeline#34

Closed

honnix force-pushed the py2-deprecation branch from 587b6bc to 9032532 Compare January 20, 2020 08:26

honnix force-pushed the py2-deprecation branch 2 times, most recently from 14bb79f to c6cfbba Compare January 28, 2020 21:53

honnix force-pushed the py2-deprecation branch from 25890dc to f6809d2 Compare February 13, 2020 08:46

honnix and others added 13 commits June 1, 2020 14:15

Update .travis.yml

b6079b2

"except ImportError" removed (#2845)

4d90229

* py2/py3 import error eliminated * E402 module level import not at top of file * E303 too many blank lines (3)

snakebite removed (#2833)

2254d10

* snamebite removed * no minicluster * no minicluster * flake8 * branches - master * no snakebite in tox.ini

[luigi.six.itervalues] usage removed (#2835)

0b32ae8

* no itervalues * no exception

py 2.7 mentions removed from README and setup.py (#2852)

6e8ba14

* 2.7 mentions removed from README and setup.py * py27 -> py37 in CONTRIBUTING.rst

drowoseque and others added 13 commits June 1, 2020 14:16

no six text_type (#2857)

0bb926f

no six create_bound_method (#2855)

13eb444

six.BytesIO removed (#2854)

966d9ee

remove unused six (#2859)

04393b8

[luigi.six.getargspec] removed (#2856)

828bf39

class Xyz(object) -> class Xyz

481202f

blank line to force ci

fdd39c2

luigi.six removed (#2871)

ee3618c

prepare 3.0.0. beta1 release

2c19fac

tox cleanup (#2877)

26c54f8

Version 3.0.0b2

3fabb43

honnix force-pushed the py2-deprecation branch from f6809d2 to 3fabb43 Compare June 1, 2020 12:19

use metaclass directly (#2952)

3b5562d

dlstadther approved these changes Jun 1, 2020

View reviewed changes

drowoseque reviewed Jun 1, 2020

View reviewed changes

honnix merged commit 91ff2f0 into master Jun 1, 2020

honnix deleted the py2-deprecation branch June 1, 2020 20:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Py2 deprecation #2876

Py2 deprecation #2876

honnix commented Dec 27, 2019 •

edited

Loading

honnix commented Dec 27, 2019

dlstadther commented Dec 28, 2019

drowoseque commented May 21, 2020

honnix commented May 21, 2020

honnix commented Jun 1, 2020

dlstadther left a comment

drowoseque Jun 1, 2020

honnix Jun 1, 2020

drowoseque commented Jun 1, 2020

honnix commented Jun 1, 2020

honnix commented Jun 1, 2020 •

edited

Loading

drowoseque commented Jun 1, 2020

Py2 deprecation #2876

Py2 deprecation #2876

Conversation

honnix commented Dec 27, 2019 • edited Loading

Description

Motivation and Context

Have you tested this? If so, how?

honnix commented Dec 27, 2019

dlstadther commented Dec 28, 2019

drowoseque commented May 21, 2020

honnix commented May 21, 2020

honnix commented Jun 1, 2020

dlstadther left a comment

Choose a reason for hiding this comment

drowoseque Jun 1, 2020

Choose a reason for hiding this comment

honnix Jun 1, 2020

Choose a reason for hiding this comment

drowoseque commented Jun 1, 2020

honnix commented Jun 1, 2020

honnix commented Jun 1, 2020 • edited Loading

drowoseque commented Jun 1, 2020

honnix commented Dec 27, 2019 •

edited

Loading

honnix commented Jun 1, 2020 •

edited

Loading