Add pyproject.toml and move setup.py logic to setup.cfg. #268

sodul · 2023-04-25T00:05:34Z

This will bring forward compatibility with newer versions of pip.

Fixes #267

Tested with:

> python setup.py --version       
3.4.3
> python setup.py --author 
Nathan Stocks
> pip install -e .
Obtaining file:///Users/stephane/repos/clumio-oss/green
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting colorama
  Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting coverage
  Using cached coverage-7.2.3-cp311-cp311-macosx_11_0_arm64.whl (200 kB)
Collecting lxml
  Using cached lxml-4.9.2.tar.gz (3.7 MB)
  Preparing metadata (setup.py) ... done
Collecting unidecode
  Using cached Unidecode-1.3.6-py3-none-any.whl (235 kB)
Building wheels for collected packages: green, lxml
  Building editable for green (pyproject.toml) ... done
  Created wheel for green: filename=green-3.4.3-0.editable-py2.py3-none-any.whl size=4971 sha256=dc61fbe5a4931e0c3a4a220bc568fb01d184a40598298d695b63b03bbf12ec08
  Stored in directory: /private/var/folders/z7/tq_3ftkd2wv74hzkjgvv2kcr0000gp/T/pip-ephem-wheel-cache-nq2x7xv5/wheels/4e/b3/dd/1fc3d18f5288acd1b0e6b3c3ec768fc284216a15db23d7f697
  Building wheel for lxml (setup.py) ... done
  Created wheel for lxml: filename=lxml-4.9.2-cp311-cp311-macosx_13_0_arm64.whl size=1699824 sha256=1f4e75d5a054367b416904ad5bf51ea2058229152ff8c9b36a97e3a8d003ef5d
  Stored in directory: /Users/stephane/Library/Caches/pip/wheels/fb/5b/f7/0a27880b4a007daeff53a196d01901627f640392b7e76e76e5
Successfully built green lxml
Installing collected packages: unidecode, lxml, coverage, colorama, green
Successfully installed colorama-0.4.6 coverage-7.2.3 green-3.4.3 lxml-4.9.2 unidecode-1.3.6

[notice] A new release of pip is available: 23.0.1 -> 23.1.1
[notice] To update, run: python -m pip install --upgrade pip
> green green
..........................................ssssssss.................................................................s..............................................................................................s.....................s

Skipped green.test.test_djangorunner.TestDjangoRunner.test_check_default_verbosity - Django is not installed

Skipped green.test.test_djangorunner.TestDjangoRunner.test_check_verbosity_argument_recognised - Django is not installed

Skipped green.test.test_djangorunner.TestDjangoRunner.test_run_coverage - Django is not installed

Skipped green.test.test_djangorunner.TestDjangoRunner.test_run_noTests - Django is not installed

Skipped green.test.test_djangorunner.TestDjangoRunner.test_run_testsWithBadInput - Django is not installed

Skipped green.test.test_djangorunner.TestDjangoRunner.test_run_testsWithLabel - Django is not installed

Skipped green.test.test_djangorunner.TestDjangoRunner.test_run_testsWithoutLabel - Django is not installed

Skipped green.test.test_djangorunner.TestDjangoRunner.test_run_with_verbosity_flag - Django is not installed

Skipped green.test.test_output.TestGreenStream.testDisableWindowsFalse - Colorama won't strip ANSI unless running on Windows

Skipped green.test.test_runner.TestProcesses.test_uncaughtException - testtools must be installed to run this test.

Skipped green.test.test_windows.TestWindows.test_colorOutput - This test is for windows-specific behavior.

Ran 233 tests in 4.972s using 10 processes

OK (passes=222, skips=11)

sodul · 2023-04-25T00:12:14Z

I do not have much prior experience with pyproject.toml and setup.cfg. I had to make some concessions on things that setup.py was able to perform, especially with the version string, but things seem to work overall.

I was able to test with python 3.11 on macOS and 3.10 through docker:

> docker run -it --entrypoint=/bin/bash -v $(pwd):/green python:3.11
root@04a02e05cae2:/# cd green/
root@04a02e05cae2:/green# pip install -e .
Obtaining file:///green
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting colorama
  Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting coverage
  Downloading coverage-7.2.3-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (231 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 231.8/231.8 kB 2.6 MB/s eta 0:00:00
Collecting lxml
  Downloading lxml-4.9.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (7.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.2/7.2 MB 12.1 MB/s eta 0:00:00
Collecting unidecode
  Downloading Unidecode-1.3.6-py3-none-any.whl (235 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 235.9/235.9 kB 19.3 MB/s eta 0:00:00
Building wheels for collected packages: green
  Building editable for green (pyproject.toml) ... done
  Created wheel for green: filename=green-3.4.3-0.editable-py2.py3-none-any.whl size=4949 sha256=0e24d4950924f6c8719cfc45ffffd210670fd0b2583a8ab82ac995a21a1d1d02
  Stored in directory: /tmp/pip-ephem-wheel-cache-byr4_9c5/wheels/b5/b4/5f/4a5d30b042601dc195ad2dc33e34811f748dbd440573f8a7fc
Successfully built green
Installing collected packages: unidecode, lxml, coverage, colorama, green
Successfully installed colorama-0.4.6 coverage-7.2.3 green-3.4.3 lxml-4.9.2 unidecode-1.3.6
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip available: 22.3.1 -> 23.1.1
[notice] To update, run: pip install --upgrade pip

sodul · 2023-04-25T00:26:23Z

Tested with the python:3.x containers by running pip install -e .; green green:

3.11: ok
3.10: ok
3.9: ok
3.8: ok
3.7: ok
3.6: error
3.5: error

The error is:

    File "/tmp/pip-build-env-z1stumkg/overlay/lib/python3.6/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 102, in __init__
      req = REQUIREMENT.parseString(requirement_string)
    File "/tmp/pip-build-env-z1stumkg/overlay/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1654, in parseString
      raise exc
    File "/tmp/pip-build-env-z1stumkg/overlay/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1644, in parseString
      loc, tokens = self._parse( instring, 0 )
    File "/tmp/pip-build-env-z1stumkg/overlay/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
      loc,tokens = self.parseImpl( instring, preloc, doActions )
    File "/tmp/pip-build-env-z1stumkg/overlay/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3417, in parseImpl
      loc, exprtokens = e._parse( instring, loc, doActions )
    File "/tmp/pip-build-env-z1stumkg/overlay/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1406, in _parseNoCache
      loc,tokens = self.parseImpl( instring, preloc, doActions )
    File "/tmp/pip-build-env-z1stumkg/overlay/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3205, in parseImpl
      raise ParseException(instring, loc, self.errmsg, self)
  pkg_resources._vendor.pyparsing.ParseException: Expected stringEnd (at char 4), (line:1, col:5)

Since 3.5 and 3.6 are officially deprecated, let me know if you want me to ensure support for them.

sodul · 2023-04-25T00:39:12Z

I was able to install with 3.5 and 3.6 by hardcoding the install_requires value.

Unfortunately I can't run green on itself with 3.5 as the test code has 3.6 syntax, f-strings did not exist yet in 3.5.

> docker run -it --entrypoint=/bin/bash -v $(pwd):/green python:3.5
root@9f2547817a80:/# cd green/
root@9f2547817a80:/green# pip install -e .
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Obtaining file:///green
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Collecting unidecode
  Downloading Unidecode-1.3.6-py3-none-any.whl (235 kB)
     |████████████████████████████████| 235 kB 1.4 MB/s 
Collecting lxml
  Downloading lxml-4.9.2-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (5.7 MB)
     |████████████████████████████████| 5.7 MB 1.7 MB/s 
Collecting coverage
  Downloading coverage-5.5-cp35-cp35m-manylinux2010_x86_64.whl (240 kB)
     |████████████████████████████████| 240 kB 19.1 MB/s 
Collecting colorama
  Downloading colorama-0.4.5-py2.py3-none-any.whl (16 kB)
Installing collected packages: unidecode, lxml, coverage, colorama, green
  Running setup.py develop for green
Successfully installed colorama-0.4.5 coverage-5.5 green lxml-4.9.2 unidecode-1.3.6
WARNING: You are using pip version 20.2.3; however, version 20.3.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
root@9f2547817a80:/green# green green
Traceback (most recent call last):
  File "/usr/local/bin/green", line 33, in <module>
    sys.exit(load_entry_point('green', 'console_scripts', 'green')())
  File "/green/green/cmdline.py", line 89, in main
    return _main(argv, testing)
  File "/green/green/cmdline.py", line 68, in _main
    result = run(test_suite, stream, args, testing)
  File "/green/green/runner.py", line 94, in run
    parallel_targets = toParallelTargets(suite, args.targets)
  File "/green/green/loader.py", line 348, in toParallelTargets
    proto_test_list = toProtoTestList(suite)
  File "/green/green/loader.py", line 332, in toProtoTestList
    toProtoTestList(i, test_list, doing_completions)
  File "/green/green/loader.py", line 324, in toProtoTestList
    getattr(suite, exception_method.strip())()
  File "/green/green/loader.py", line 80, in testFailure
    raise ImportError(message)
ImportError: Failed to import green.test.test_integration computed from filename /green/green/test/test_integration.py
Traceback (most recent call last):
  File "/green/green/loader.py", line 54, in loadFromModuleFilename
    __import__(dotted_module)
  File "/green/green/test/test_integration.py", line 29
    fh = open(os.path.join(sub_tmpdir, f"test_finalizer{i}.py"), "w")
                                                             ^
SyntaxError: invalid syntax

sodul · 2023-04-25T00:44:24Z

I was able to run green green with python 3.5 by changing the syntax ingreen/test/test_integration.py:

        for i in range(multiprocessing.cpu_count() * 2):
            fh = open(os.path.join(sub_tmpdir, "test_finalizer%d.py" % i), "w")
            fh.write(
                dedent(
                    """
                import unittest
                class Pass%d(unittest.TestCase):
                    def test_pass%d(self):
                        pass
                def msg():
                    print("finalizer worked")
                """ % (i, i)
                )
            )
            fh.close()

Let me know how you want to handle this:

explicitly drop support for python 3.5, and I'll update setup.cfg to reflect that.
keep support for 3.5 and patch the test.

CleanCut

Let me know how you want to handle this:

explicitly drop support for python 3.5, and I'll update setup.cfg to reflect that.

keep support for 3.5 and patch the test.

Let's go ahead and drop support for Python 3.5.

If you will enable the Allow edits from maintainers setting on this PR, I can help fix things up.

setup.cfg

This will bring forward compatibility with newer versions of pip. Drop support for Python 3.5 and older. Removed versioned entrypoints, this is not longer relevant now that Python 2 is long deprecated. Fixes CleanCut#267

setup.cfg

README.md

sodul · 2023-05-06T04:45:00Z

I'm going to run find . -name '*.py' -exec pyupgrade --py36-plus {} + for this PR. It is catching a lot of python 2 leftovers, and will update many strings to f-string, amongst other goodies that will make the code more readable.

If you are not against it I'm considering adding mypy to the CI pipeline and start adding type annotation, but in a separate PR. I've found mypy to be very valuable in finding a lot of hidden bugs in our own codebase.

Run black.

sodul · 2023-05-06T05:51:44Z

The CI check cannot run python 3.6 on Ubuntu Latest. Considering 3.6 has been deprecated for over a year that's not surprising. I can skip 3.6 on Ubuntu only since it passes on macOS and Windows. Note that 3.7 will be deprecated at the end of June:

https://devguide.python.org/versions/

In case you are wondering about some of the changes decided by pyupgrade, they are well documented: https://github.com/asottile/pyupgrade

coveralls · 2023-05-06T06:50:09Z

Coverage: 100.0%. Remained the same when pulling 802cd19 on clumio-oss:pyproject.toml into f021ba4 on CleanCut:main.

CleanCut

LGTM!

CleanCut requested changes Apr 26, 2023

View reviewed changes

setup.cfg Outdated Show resolved Hide resolved

sodul requested a review from CleanCut April 27, 2023 00:10

Add pyproject.toml and move setup.py logic to setup.cfg.

d93775b

This will bring forward compatibility with newer versions of pip. Drop support for Python 3.5 and older. Removed versioned entrypoints, this is not longer relevant now that Python 2 is long deprecated. Fixes CleanCut#267

sodul force-pushed the pyproject.toml branch from 1be091d to d93775b Compare May 5, 2023 22:28

CleanCut reviewed May 5, 2023

View reviewed changes

setup.cfg Outdated Show resolved Hide resolved

README.md Show resolved Hide resolved

sodul added 3 commits May 5, 2023 21:46

Minor cleanup of readme and comments.

ea21e3a

Run find . -name '*.py' -exec pyupgrade --py36-plus {} +.

8757652

Add python 3.6 and 3.7 to the CI checks.

7a834bc

Run black.

sodul requested a review from CleanCut May 6, 2023 06:04

Exclude CI checks for Python 3.6 on Ubuntu-latest.

802cd19

CleanCut approved these changes May 8, 2023

View reviewed changes

CleanCut merged commit 2cd780e into CleanCut:main May 8, 2023
17 checks passed

sodul deleted the pyproject.toml branch May 9, 2023 00:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pyproject.toml and move setup.py logic to setup.cfg. #268

Add pyproject.toml and move setup.py logic to setup.cfg. #268

sodul commented Apr 25, 2023

sodul commented Apr 25, 2023

sodul commented Apr 25, 2023

sodul commented Apr 25, 2023 •

edited

sodul commented Apr 25, 2023

CleanCut left a comment

sodul commented May 6, 2023

sodul commented May 6, 2023 •

edited

coveralls commented May 6, 2023

CleanCut left a comment

Add pyproject.toml and move setup.py logic to setup.cfg. #268

Add pyproject.toml and move setup.py logic to setup.cfg. #268

Conversation

sodul commented Apr 25, 2023

sodul commented Apr 25, 2023

sodul commented Apr 25, 2023

sodul commented Apr 25, 2023 • edited

sodul commented Apr 25, 2023

CleanCut left a comment

Choose a reason for hiding this comment

sodul commented May 6, 2023

sodul commented May 6, 2023 • edited

coveralls commented May 6, 2023

CleanCut left a comment

Choose a reason for hiding this comment

sodul commented Apr 25, 2023 •

edited

sodul commented May 6, 2023 •

edited