Development Setup fails #2532

DimitriPapadopoulos · 2022-10-16T13:06:02Z

You can install required dependencies for development by running the following within a checkout of the codespell source:
pip install -e ".[dev]"

This command fails for me on Ubuntu 22.04:

Detailed log

$ pip3 install --user -e ".[dev]"
Obtaining file:///my/path/codespell
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
WARNING: unknown 2.2.2.dev70+g055cb2a9 does not provide the extra 'dev'
Installing collected packages: UNKNOWN
  Running setup.py develop for UNKNOWN
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [32 lines of output]
        running develop
        /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        WARNING: The user site-packages directory is disabled.
        /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        error: can't create or remove files in install directory
        
        The following error occurred while trying to add or remove files in the
        installation directory:
        
            [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-33461.write-test'
        
        The installation directory you specified (via --install-dir, --prefix, or
        the distutils default setting) was:
        
            /usr/local/lib/python3.10/dist-packages/
        
        Perhaps your account does not have write access to this directory?  If the
        installation directory is a system-owned directory, you may need to sign in
        as the administrator or "root" account.  If you do not have administrative
        access to this machine, you may wish to choose a different installation
        directory, preferably one that is listed in your PYTHONPATH environment
        variable.
        
        For information on other options, you may wish to consult the
        documentation at:
        
          https://setuptools.pypa.io/en/latest/deprecated/easy_install.html
        
        Please make the appropriate changes for your system and try again.
        
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [32 lines of output]
    running develop
    /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    WARNING: The user site-packages directory is disabled.
    /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    error: can't create or remove files in install directory
    
    The following error occurred while trying to add or remove files in the
    installation directory:
    
        [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-33461.write-test'
    
    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:
    
        /usr/local/lib/python3.10/dist-packages/
    
    Perhaps your account does not have write access to this directory?  If the
    installation directory is a system-owned directory, you may need to sign in
    as the administrator or "root" account.  If you do not have administrative
    access to this machine, you may wish to choose a different installation
    directory, preferably one that is listed in your PYTHONPATH environment
    variable.
    
    For information on other options, you may wish to consult the
    documentation at:
    
      https://setuptools.pypa.io/en/latest/deprecated/easy_install.html
    
    Please make the appropriate changes for your system and try again.
    
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
$

First, I had to manually install aspell to run make check without a warning:

sudo apt install libaspell-dev
pip3 install --user aspell-python-py3

Then, I had to investigate the errors...

The text was updated successfully, but these errors were encountered:

DimitriPapadopoulos · 2022-10-16T13:12:05Z

Installing from GitHub fails the same way:

Detailed log

$ pip3 install --user git+https://github.com/codespell-project/codespell.git
Collecting git+https://github.com/codespell-project/codespell.git
  Cloning https://github.com/codespell-project/codespell.git to /tmp/pip-req-build-2cm03rxk
  Running command git clone --filter=blob:none --quiet https://github.com/codespell-project/codespell.git /tmp/pip-req-build-2cm03rxk
  Resolved https://github.com/codespell-project/codespell.git to commit 055cb2a9bc1e98a9800a352c53c38a50faf62dc3
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: UNKNOWN
  Building wheel for UNKNOWN (pyproject.toml) ... done
  Created wheel for UNKNOWN: filename=UNKNOWN-2.2.2-py3-none-any.whl size=7972 sha256=3d705eaff3dc72bb7b05bc50134eff6f0d8da205f441334d2761b4472621f13c
  Stored in directory: /tmp/pip-ephem-wheel-cache-ju_qo01r/wheels/80/e9/c1/0a51b174dbeeca916646da713d485fb2536d6ebdfce7300a1a
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
Successfully installed UNKNOWN-2.2.2
$

DimitriPapadopoulos · 2022-10-16T13:13:27Z

Might be related to the addition of pyproject.toml by #2523.

Saransh-cpp · 2022-10-16T15:18:20Z

@DimitriPapadopoulos, I hope you are using the latest version of setuptools. This is not a problem with codespell. See pypa/setuptools#3269 for more information.

DimitriPapadopoulos · 2022-10-16T16:04:32Z

If a recent version of setuptools is required, it should be installed automatically, an error should be emitted if this version is not available, or the requirement should be documented in the installation instructions.

I have version 59.6.0 of setuptools, the one bundled with Ubuntu 22.04:

$ dpkg -l python3-setuptools
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name               Version      Architecture Description
+++-==================-============-============-=================================
ii  python3-setuptools 59.6.0-1.2   all          Python3 Distutils Enhancements
$ 
$ python3
Python 3.10.6 (main, Aug 10 2022, 11:40:04) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.__version__
'59.6.0'
>>>

File pyrpojrct.toml tells me this version is good enough:

codespell/pyproject.toml

Lines 48 to 49 in 055cb2a

    
           [build-system] 
        
           requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2", "wheel"]

Upgrading to a more recent of setuptools does not help:

Installation of latest setuptools and relevant log

$ pip3 install -U --user setuptools
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (59.6.0)
Collecting setuptools
  Using cached setuptools-65.5.0-py3-none-any.whl (1.2 MB)
Installing collected packages: setuptools
Successfully installed setuptools-65.5.0
$ 
$ python3
Python 3.10.6 (main, Aug 10 2022, 11:40:04) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.__version__
'65.5.0'
>>> 
$ 
$ pip3 install -e ".[dev]"
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///my/path/codespell
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
WARNING: unknown 2.2.2.dev70+g055cb2a9 does not provide the extra 'dev'
Installing collected packages: UNKNOWN
  Attempting uninstall: UNKNOWN
    Found existing installation: UNKNOWN 2.2.2
    Uninstalling UNKNOWN-2.2.2:
      Successfully uninstalled UNKNOWN-2.2.2
  Running setup.py develop for UNKNOWN
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [32 lines of output]
        running develop
        /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        WARNING: The user site-packages directory is disabled.
        /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        error: can't create or remove files in install directory
        
        The following error occurred while trying to add or remove files in the
        installation directory:
        
            [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-50719.write-test'
        
        The installation directory you specified (via --install-dir, --prefix, or
        the distutils default setting) was:
        
            /usr/local/lib/python3.10/dist-packages/
        
        Perhaps your account does not have write access to this directory?  If the
        installation directory is a system-owned directory, you may need to sign in
        as the administrator or "root" account.  If you do not have administrative
        access to this machine, you may wish to choose a different installation
        directory, preferably one that is listed in your PYTHONPATH environment
        variable.
        
        For information on other options, you may wish to consult the
        documentation at:
        
          https://setuptools.pypa.io/en/latest/deprecated/easy_install.html
        
        Please make the appropriate changes for your system and try again.
        
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
  Rolling back uninstall of UNKNOWN
  Moving to /home/dimitri/.local/lib/python3.10/site-packages/UNKNOWN-2.2.2.dist-info/
   from /home/dimitri/.local/lib/python3.10/site-packages/~NKNOWN-2.2.2.dist-info
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [32 lines of output]
    running develop
    /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    WARNING: The user site-packages directory is disabled.
    /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    error: can't create or remove files in install directory
    
    The following error occurred while trying to add or remove files in the
    installation directory:
    
        [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-50719.write-test'
    
    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:
    
        /usr/local/lib/python3.10/dist-packages/
    
    Perhaps your account does not have write access to this directory?  If the
    installation directory is a system-owned directory, you may need to sign in
    as the administrator or "root" account.  If you do not have administrative
    access to this machine, you may wish to choose a different installation
    directory, preferably one that is listed in your PYTHONPATH environment
    variable.
    
    For information on other options, you may wish to consult the
    documentation at:
    
      https://setuptools.pypa.io/en/latest/deprecated/easy_install.html
    
    Please make the appropriate changes for your system and try again.
    
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
$

@Saransh-cpp Why exactly do you think this error is related to the version of setuptools?

DimitriPapadopoulos · 2022-10-16T16:39:37Z

Ah, removing the system python3-setuptools DEB package does help.

Deletion of system setuptools and relevant log

$ sudo apt purge python3-setuptools 
[...]
$ 
$ python3
Python 3.10.6 (main, Aug 10 2022, 11:40:04) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.__version__
'65.5.0'
>>> 
$ 
$ pip3 install -e ".[dev]"
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///my/path/codespell
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting pytest-cov
  Using cached pytest_cov-4.0.0-py3-none-any.whl (21 kB)
Collecting pytest-dependency
  Using cached pytest-dependency-0.5.1.tar.gz (27 kB)
  Preparing metadata (setup.py) ... done
Collecting flake8
  Using cached flake8-5.0.4-py2.py3-none-any.whl (61 kB)
Requirement already satisfied: tomli in /usr/lib/python3/dist-packages (from codespell==2.2.2.dev70+g055cb2a9) (1.2.2)
[...]
Successfully built codespell pytest-dependency
Installing collected packages: iniconfig, pyflakes, pycodestyle, py, pep517, mccabe, codespell, pytest, flake8, build, pytest-dependency, pytest-cov, check-manifest
Successfully installed build-0.8.0 check-manifest-0.48 codespell-2.2.2.dev70+g055cb2a9 flake8-5.0.4 iniconfig-1.1.1 mccabe-0.7.0 pep517-0.13.0 py-1.11.0 pycodestyle-2.9.1 pyflakes-2.5.0 pytest-7.1.3 pytest-cov-4.0.0 pytest-dependency-0.5.1
$

Indeed, the system setuptools installed by DEB package python3-setuptools hides the one installed by pip3 under ~/.local/lib/python3.10/site-packages.

The question remains, which minimal version of setuptools is required:

codespell/pyproject.toml

Lines 48 to 49 in 055cb2a

    
           [build-system] 
        
           requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2", "wheel"]

It appears a vanilla installation of setuptools version 59.6.0 works quite well, so I suspect the problem lies in the DEB package python3-setuptools provided by Ubuntu.

Saransh-cpp · 2022-10-16T18:38:21Z

Technically, setuptools should have a lower limit of 64.0.0 as codespell uses a pyproject.toml based build. See https://twitter.com/juanluisback/status/1557734536586625025?t=rzVtT-91AwzSOvBHanCJ9g&s=19.

DimitriPapadopoulos · 2022-10-16T19:17:01Z

I guess we need to change:

requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2", "wheel"]

to:

requires = ["setuptools>=64", "setuptools_scm[toml]>=6.2", "wheel"]

I'm not sure what setuptools_scm[toml]>=6.2 means at this point, and whether it need to be updated too.

Saransh-cpp · 2022-10-17T06:34:31Z

It should stay as it is - #2523 (comment)

DimitriPapadopoulos · 2022-10-17T06:52:21Z

#2523 (comment) is about keeping setup.py. How does it relate with the version of setuptools or setuptools_scm[toml]?

Saransh-cpp · 2022-10-17T07:21:42Z

They want to maintain legacy builds, which means they might also want to support older versions of setuptools for this legacy compatibility.

DimitriPapadopoulos · 2022-10-17T07:36:05Z

Ah, I get it. But the version of setuptools and legacy compatibility are two different things in this context, aren't they?

I kept it for legacy purposes / compat with existing calls like python setup.py check and python setup.py develop

The idea of legacy compatibility in this case is to support running setup.py, not to support old versions of setuptools. Are these related?

Saransh-cpp · 2022-10-17T18:24:47Z

The idea of legacy compatibility in this case is to support running setup.py, not to support old versions of setuptools. Are these related?

Ah, maybe, maybe not. Technically we shouldn't even be using setuptools as a frontend (all the python setup.py ... commands) as that is unmaintained (a lot of long threads on this are available in setuptools' repository).

If they do end up removing setup.py then it would be an absolute necessity to upgrade to setuptools>=64.0.0, but right now I think it is a choice 😄

DimitriPapadopoulos · 2022-10-17T19:17:10Z

I see. I guess we can leave it as it is for now, and revisit when the need arises.

This was referenced Oct 16, 2022

Migrate to pyproject.toml + cleanup zarr-developers/zarr-python#1158

Merged

MAINT: Update versioneer 0.19 → 0.26 numpy/numpy#22438

Merged

This was referenced Oct 21, 2022

Broken package name (since commit [754b9a52f7bb83d8d356050a14caebc6fbaccc85]) #2547

Closed

MAINT: Use pyproject and setuptools_scm #2523

Merged

Document required setuptools version #2560

Merged

larsoner closed this as completed in #2560 Oct 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Development Setup fails #2532

Development Setup fails #2532

DimitriPapadopoulos commented Oct 16, 2022 •

edited

DimitriPapadopoulos commented Oct 16, 2022

DimitriPapadopoulos commented Oct 16, 2022

Saransh-cpp commented Oct 16, 2022

DimitriPapadopoulos commented Oct 16, 2022 •

edited

DimitriPapadopoulos commented Oct 16, 2022 •

edited

Saransh-cpp commented Oct 16, 2022

DimitriPapadopoulos commented Oct 16, 2022 •

edited

Saransh-cpp commented Oct 17, 2022

DimitriPapadopoulos commented Oct 17, 2022

Saransh-cpp commented Oct 17, 2022

DimitriPapadopoulos commented Oct 17, 2022

Saransh-cpp commented Oct 17, 2022

DimitriPapadopoulos commented Oct 17, 2022

Development Setup fails #2532

Development Setup fails #2532

Comments

DimitriPapadopoulos commented Oct 16, 2022 • edited

DimitriPapadopoulos commented Oct 16, 2022

DimitriPapadopoulos commented Oct 16, 2022

Saransh-cpp commented Oct 16, 2022

DimitriPapadopoulos commented Oct 16, 2022 • edited

DimitriPapadopoulos commented Oct 16, 2022 • edited

Saransh-cpp commented Oct 16, 2022

DimitriPapadopoulos commented Oct 16, 2022 • edited

Saransh-cpp commented Oct 17, 2022

DimitriPapadopoulos commented Oct 17, 2022

Saransh-cpp commented Oct 17, 2022

DimitriPapadopoulos commented Oct 17, 2022

Saransh-cpp commented Oct 17, 2022

DimitriPapadopoulos commented Oct 17, 2022

DimitriPapadopoulos commented Oct 16, 2022 •

edited

DimitriPapadopoulos commented Oct 16, 2022 •

edited

DimitriPapadopoulos commented Oct 16, 2022 •

edited

DimitriPapadopoulos commented Oct 16, 2022 •

edited