Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Poetry Install Bug #8286

Closed
NickDeBeenSAE opened this issue Aug 9, 2023 · 19 comments
Closed

Poetry Install Bug #8286

NickDeBeenSAE opened this issue Aug 9, 2023 · 19 comments
Labels
status/external-issue Issue is caused by external project (platform, dep, etc)

Comments

@NickDeBeenSAE
Copy link

NickDeBeenSAE commented Aug 9, 2023

Good afternoon from Fedora 38, and Australia as a result.

I am a freelance programmer, but I am about to go into a Diploma of Game Development.

[nickdebeen@fedora Downloads]$ ls
gpt4all
[nickdebeen@fedora Downloads]$ cd gpt4all/gpt4all-bindings/python
[nickdebeen@fedora python]$ pip3 install -e .
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/nickdebeen/Downloads/gpt4all/gpt4all-bindings/python
  Preparing metadata (setup.py) ... done
Requirement already satisfied: requests in /usr/lib/python3.11/site-packages (from gpt4all==1.0.8) (2.28.2)
Requirement already satisfied: tqdm in /home/nickdebeen/.local/lib/python3.11/site-packages (from gpt4all==1.0.8) (4.65.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/lib/python3.11/site-packages (from requests->gpt4all==1.0.8) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.11/site-packages (from requests->gpt4all==1.0.8) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.11/site-packages (from requests->gpt4all==1.0.8) (1.26.16)
Installing collected packages: gpt4all
  Attempting uninstall: gpt4all
    Found existing installation: gpt4all 1.0.8
    Uninstalling gpt4all-1.0.8:
      Successfully uninstalled gpt4all-1.0.8
  Running setup.py develop for gpt4all
Successfully installed gpt4all-1.0.8
[nickdebeen@fedora python]$ poetry install

Poetry could not find a pyproject.toml file in /home/nickdebeen/Downloads/gpt4all/gpt4all-bindings/python or its parents
[nickdebeen@fedora python]$ cd..
bash: cd..: command not found...
cd [nickdebeen@fedora python]$ cd ..
[nickdebeen@fedora gpt4all-bindings]$ cd ..
[nickdebeen@fedora gpt4all]$ poetry install

Poetry could not find a pyproject.toml file in /home/nickdebeen/Downloads/gpt4all or its parents
[nickdebeen@fedora gpt4all]$ poetry init

This command will guide you through creating your pyproject.toml config.

Package name [gpt4all]:  
Version [0.1.0]:  
Description []:  
Author [None, n to skip]:  
expected string or bytes-like object, got 'NoneType'
Author [None, n to skip]:  n
License []:  
Compatible Python versions [^3.11]:  

Would you like to define your main dependencies interactively? (yes/no) [yes] 
You can specify a package in the following forms:
  - A single name (requests): this will search for matches on PyPI
  - A name and a constraint (requests@^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - A url (https://example.com/packages/my-package-0.1.0.tar.gz)

Package to add or search for (leave blank to skip): 

Would you like to define your development dependencies interactively? (yes/no) [yes] 
Package to add or search for (leave blank to skip): 

Generated file

[tool.poetry]
name = "gpt4all"
version = "0.1.0"
description = ""
authors = ["Your Name "]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"


Do you confirm generation? (yes/no) [yes] 
[nickdebeen@fedora gpt4all]$ poetry install
The currently activated Python version 3.11.4 is not supported by the project (>=3.11, <=3.11).
Trying to find and use a compatible version. 

Poetry was unable to find a compatible version. If you have one, you can explicitly use it via the "env use" command.
[nickdebeen@fedora gpt4all]$ poetry install
Creating virtualenv genossgpt-1KB5LGh4-py3.11 in /home/nickdebeen/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file
[nickdebeen@fedora gpt4all]$ poetry run genosgpt
Command not found: genosgpt
[nickdebeen@fedora gpt4all]$ poetry run genossgpt
Command not found: genossgpt
[nickdebeen@fedora gpt4all]$ poetry run genossgpt
Command not found: genossgpt
[nickdebeen@fedora gpt4all]$ poetry run genossgpt
Warning: 'genossgpt' is an entry point defined in pyproject.toml, but it's not installed as a script. You may get improper `sys.argv[0]`.

The support to run uninstalled scripts will be removed in a future release.

Run `poetry install` to resolve and get rid of this message.


No file/folder found for package genossgpt
[nickdebeen@fedora gpt4all]$ poetry install
Installing dependencies from lock file
[nickdebeen@fedora gpt4all]$ poetry run genossgpt
Warning: 'genossgpt' is an entry point defined in pyproject.toml, but it's not installed as a script. You may get improper `sys.argv[0]`.

The support to run uninstalled scripts will be removed in a future release.

Run `poetry install` to resolve and get rid of this message.


No file/folder found for package genossgpt
[nickdebeen@fedora gpt4all]$ poetry run genossgpt
Warning: 'genossgpt' is an entry point defined in pyproject.toml, but it's not installed as a script. You may get improper `sys.argv[0]`.

The support to run uninstalled scripts will be removed in a future release.

Run `poetry install` to resolve and get rid of this message.


No file/folder found for package genossgpt
[nickdebeen@fedora gpt4all]$ poetry install
Installing dependencies from lock file
[nickdebeen@fedora gpt4all]$ poetry init
A pyproject.toml file with a poetry section already exists.
[nickdebeen@fedora gpt4all]$ poetry shell
Spawning shell within /home/nickdebeen/.cache/pypoetry/virtualenvs/genossgpt-1KB5LGh4-py3.11
[nickdebeen@fedora gpt4all]$ . /home/nickdebeen/.cache/pypoetry/virtualenvs/genossgpt-1KB5LGh4-py3.11/bin/activate
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ peotry install
bash: peotry: command not found...
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry install
Installing dependencies from lock file
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry run genossgpt
Warning: 'genossgpt' is an entry point defined in pyproject.toml, but it's not installed as a script. You may get improper `sys.argv[0]`.

The support to run uninstalled scripts will be removed in a future release.

Run `poetry install` to resolve and get rid of this message.


No file/folder found for package genossgpt
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry self update
Updating Poetry version ...

Using version ^1.5.1 for poetry

Updating dependencies
Resolving dependencies... (21.7s)

Package operations: 1 install, 47 updates, 0 removals

  • Updating pycparser (2.20 /usr/lib/python3.11/site-packages -> 2.21): Failed

  CalledProcessError

  Command '['/usr/bin/python3', '-m', 'pip', 'uninstall', 'pycparser', '-y']' returned non-zero exit status 2.

  at /usr/lib64/python3.11/subprocess.py:571 in run
       567│             # We don't call process.wait() as .__exit__ does that for us.
       568│             raise
       569│         retcode = process.poll()
       570│         if check and retcode:
    →  571│             raise CalledProcessError(retcode, process.args,
       572│                                      output=stdout, stderr=stderr)
       573│     return CompletedProcess(process.args, retcode, stdout, stderr)
       574│ 
       575│ 

The following error occurred when trying to handle this error:


  EnvCommandError

  Command ['/usr/bin/python3', '-m', 'pip', 'uninstall', 'pycparser', '-y'] errored with the following return code 2
  
  Output:
  Found existing installation: pycparser 2.20
  Uninstalling pycparser-2.20:
  ERROR: Exception:
  Traceback (most recent call last):
    File "/usr/lib64/python3.11/shutil.py", line 825, in move
      os.rename(src, real_dst)
  OSError: [Errno 18] Invalid cross-device link: '/usr/lib/python3.11/site-packages/pycparser' -> '/tmp/pip-uninstall-zgit0ktr'
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/usr/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
      status = run_func(*args)
               ^^^^^^^^^^^^^^^
    File "/usr/lib/python3.11/site-packages/pip/_internal/commands/uninstall.py", line 98, in run
      uninstall_pathset = req.uninstall(
                          ^^^^^^^^^^^^^^
    File "/usr/lib/python3.11/site-packages/pip/_internal/req/req_install.py", line 660, in uninstall
      uninstalled_pathset.remove(auto_confirm, verbose)
    File "/usr/lib/python3.11/site-packages/pip/_internal/req/req_uninstall.py", line 373, in remove
      moved.stash(path)
    File "/usr/lib/python3.11/site-packages/pip/_internal/req/req_uninstall.py", line 271, in stash
      renames(path, new_path)
    File "/usr/lib/python3.11/site-packages/pip/_internal/utils/misc.py", line 311, in renames
      shutil.move(old, new)
    File "/usr/lib64/python3.11/shutil.py", line 843, in move
      rmtree(src)
    File "/usr/lib64/python3.11/shutil.py", line 732, in rmtree
      _rmtree_safe_fd(fd, path, onerror)
    File "/usr/lib64/python3.11/shutil.py", line 660, in _rmtree_safe_fd
      _rmtree_safe_fd(dirfd, fullname, onerror)
    File "/usr/lib64/python3.11/shutil.py", line 683, in _rmtree_safe_fd
      onerror(os.unlink, fullname, sys.exc_info())
    File "/usr/lib64/python3.11/shutil.py", line 681, in _rmtree_safe_fd
      os.unlink(entry.name, dir_fd=topfd)
  PermissionError: [Errno 13] Permission denied: '__init__.cpython-311.opt-1.pyc'
  

  at ~/.local/lib/python3.11/site-packages/poetry/utils/env.py:1525 in _run
      1521│                 output = ""
      1522│             else:
      1523│                 output = subprocess.check_output(cmd, stderr=stderr, env=env, **kwargs)
      1524│         except CalledProcessError as e:
    → 1525│             raise EnvCommandError(e, input=input_)
      1526│ 
      1527│         return decode(output)
      1528│ 
      1529│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ sudo poetry self update
[sudo] password for nickdebeen: 
sudo: poetry: command not found
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ exit
exit
[nickdebeen@fedora gpt4all]$ sudo pip uninstall pycparser
[sudo] password for nickdebeen: 
Found existing installation: pycparser 2.20
Uninstalling pycparser-2.20:
  Would remove:
    /usr/lib/python3.11/site-packages/pycparser
    /usr/lib/python3.11/site-packages/pycparser-2.20-py3.11.egg-info
Proceed (Y/n)? y
  Successfully uninstalled pycparser-2.20
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[nickdebeen@fedora gpt4all]$ poetry activate

The command "activate" does not exist.
[nickdebeen@fedora gpt4all]$ poetry shell
Spawning shell within /home/nickdebeen/.cache/pypoetry/virtualenvs/genossgpt-1KB5LGh4-py3.11
[nickdebeen@fedora gpt4all]$ . /home/nickdebeen/.cache/pypoetry/virtualenvs/genossgpt-1KB5LGh4-py3.11/bin/activate
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ pip3 uninstall pycparser
WARNING: Skipping pycparser as it is not installed.
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry install --directory /home/nickdebeen/Downloads/gpt4all
Installing dependencies from lock file
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry build
Building GenossGPT (0.0.1)

No file/folder found for package genossgpt
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry build /home/nickdebeen/local_models

No arguments expected for "build" command, got "/home/nickdebeen/local_models"
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry run poetry --version
Poetry (version 1.5.1)
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry build /home/nickdebeen/local_models

No arguments expected for "build" command, got "/home/nickdebeen/local_models"
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry install --directory /home/nickdebeen/Downloads/gpt4all
Installing dependencies from lock file
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry cache clear -all pypi

The option "a" does not exist
(genossgpt-py3.11) [nickdebeen@fedora gpt4all]$ poetry cache clear pypi

Add the --all option if you want to clear all pypi caches
@NickDeBeenSAE NickDeBeenSAE added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Aug 9, 2023
@NickDeBeenSAE
Copy link
Author

NickDeBeenSAE commented Aug 9, 2023

As you can see, I found quite a significant amount of bugs with Poetry.

I am trying to build the GenossGPT model GPT4All.

I can't if Poetry, even in seen virtual environment fails because of the bugs seen.

Bug #1:

Poetry requires the --all flag to be set when clearing cache.

It clearly says that the -a function does not exist as you can see.

That is a bug, as it did previously.

Bug #2:

Poetry's install command fails to find and install the dependencies required to run GenossGPT.

In the output printed above, it says it is installing dependencies from its .lock file, but there are none listed.

See attached my .lock file for your viewing needs:

# This file is automatically @generated by Poetry 1.5.1 and should not be changed by hand.
package = []

[metadata]
lock-version = "2.0"
python-versions = "^3.11"
content-hash = "81b2fa642d7f2d1219cf80112ace12d689d053d81be7f7addb98144d56fc0fb2"

[tool.poetry]
name = "GenossGPT"
version = "0.0.1"
description = "An LLVM that utilises the GPT4.0 Architecture."
authors = ["Nickolas De Been"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"

[tool.poetry.scripts]
genossgpt = "genossgpt.main:main_function"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

@dimbleby
Copy link
Contributor

dimbleby commented Aug 9, 2023

(1) you typed -all where you meant --all
(2) it looks as though you're using a virtual environment that pulls in system-wide packages. This is unwise: per your stack poetry wants to own the environment in which it is installing and in this example it is trying to uninstall something that it does not have permission to uninstall. Use a new virtual environment.

There are no bugs here, please refer to the documentation.

@NickDeBeenSAE
Copy link
Author

Well, I am going to counter and say that it doesn't work outside the virtual environmemt either.

The virtual environment Poetry was in, is its own.

@dimbleby
Copy link
Contributor

dimbleby commented Aug 9, 2023

you haven't told us how you installed poetry but I guess to get into this it was not by a supported method. Uninstall it, and start over but following the docs.

you haven't shared your pyproject.toml so if there is a problem with that no-one else can do anything about it

@NickDeBeenSAE
Copy link
Author

I followed the guide to install Poetry in the first place.

I have shared my .toml.

You didn't look.

@dimbleby
Copy link
Contributor

dimbleby commented Aug 9, 2023

do you mean this?

[tool.poetry]
name = "gpt4all"
version = "0.1.0"
description = ""
authors = ["Your Name "]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

if so the answer is clear: you haven't added any dependencies. Of course poetry doesn't install anything.

But then that pyproject.toml would not produce the lockfile that you show - gpt4all is not the same project name as GenossGPT. So it's entirely unclear what you're doing.

Oh, later you provide a blob of text that is a pyproject.toml appended to a lockfile - is that what you mean? That also contains no dependencies, so same answer: of course poetry doesn't find any dependencies if you haven't told it about any dependencies.

I suspect this is your first time with poetry and you don't know what you're doing. That's fine, that's how we all start out. But please: don't raise an issue declaring that you have found lots of bugs - first read and follow the docs.

Please close.

@NickDeBeenSAE
Copy link
Author

No, I meant this:

[tool.poetry]
name = "GenossGPT"
version = "0.0.1"
description = "An LLVM that utilises the GPT4.0 Architecture."
authors = ["Nickolas De Been"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"

[tool.poetry.scripts]
genossgpt = "genossgpt.main:main_function"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

@dimbleby
Copy link
Contributor

dimbleby commented Aug 9, 2023

As I say - same answer. You haven't provided any dependencies, poetry doesn't know about dependencies, there is no bug. Please close.

@NickDeBeenSAE
Copy link
Author

Oh, later you provide a blob of text that is a pyproject.toml appended to a lockfile - is that what you mean? That also contains no dependencies, so same answer: of course poetry doesn't find any dependencies if you haven't told it about any dependencies.

This is not the case, as GitHub does not support the uploading of .lock and .toml files.

@NickDeBeenSAE
Copy link
Author

NickDeBeenSAE commented Aug 9, 2023

As I say - same answer. You haven't provided any dependencies, poetry doesn't know about dependencies, there is no bug. Please close.

Wrong, because I went with the defaults that Poetry requested.

I simply pressed enter on all questions.

Poetry failed to ask for dependencies, otherwise I would have seen them.

@NickDeBeenSAE
Copy link
Author

But then that pyproject.toml would not produce the lockfile that you show - gpt4all is not the same project name as GenossGPT. So it's entirely unclear what you're doing.

GenossGPT is made from the GPT4All repository:

https://github.com/OpenGenerativeAI/GenossGPT

@dimbleby
Copy link
Contributor

dimbleby commented Aug 9, 2023

Let's not bicker about your formatting or your ability to enter dependencies:

Would you like to define your main dependencies interactively? (yes/no) [yes] 
....
Package to add or search for (leave blank to skip): 

Would you like to define your development dependencies interactively? (yes/no) [yes] 
Package to add or search for (leave blank to skip): 

bottom line: you haven't added any dependencies and that's where you are. Please close.

@NickDeBeenSAE
Copy link
Author

NickDeBeenSAE commented Aug 9, 2023

Let's not bicker about your formatting or your ability to enter dependencies:

Package to add or search for (leave blank to skip): 

Would you like to define your development dependencies interactively? (yes/no) [yes] 
Package to add or search for (leave blank to skip): 

bottom line: you haven't added any dependencies and that's where you are. Please close.

Arguing is what you think I'm doing.

I am doing what I know and see.

And this is following instructions on the repositories page.

It said nothing about dependencies that are required.

I haven't seen you respond with any dependencies yet, so rather then trying to create an argument as you think I am doing, which is an accusation at best, print the dependencies required.

And this is with a please and a thank you on top.

@NickDeBeenSAE
Copy link
Author

NickDeBeenSAE commented Aug 9, 2023

I think I have come to a thought as to why Poetry failed to install any dependencies:

https://github.com/OpenGenerativeAI/GenossGPT

The GenossGPT or OpenGenerativeAI page in this case (both are the same though) also failed to list any dependencies as Poetry isn't there's to manage the repository of in the first place.

@NickDeBeenSAE
Copy link
Author

you haven't told us how you installed poetry but I guess to get into this it was not by a supported method. Uninstall it, and start over but following the docs.

Poetry was installed via curl.

Following the "docs" instructions.

@MarcelWilson
Copy link

It said nothing about dependencies that are required.

It does...
https://python-poetry.org/docs/basic-usage/#specifying-dependencies

I think I have come to a thought as to why Poetry failed to install any dependencies:

because you didn't list any in the pyproject.toml. If your project needs genoss then you need to declare it as a dependency.

Poetry failed to ask for dependencies, otherwise I would have seen them.

That's not how poetry works.

@NickDeBeenSAE
Copy link
Author

Well, I just gave up on it.

If the OpenGenerativeAI and GPT4All project's both fail list even a single dependency, Poetry is not going to build nor install anything.

@dimbleby
Copy link
Contributor

dimbleby commented Sep 7, 2023

please close

@radoering radoering added status/external-issue Issue is caused by external project (platform, dep, etc) and removed status/triage This issue needs to be triaged kind/bug Something isn't working as expected labels Sep 7, 2023
@radoering radoering closed this as not planned Won't fix, can't repro, duplicate, stale Sep 7, 2023
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
status/external-issue Issue is caused by external project (platform, dep, etc)
Projects
None yet
Development

No branches or pull requests

4 participants