Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UserWarning: Calling a list() method without specifying get_all=True or iterator=True will return a maximum of 20 items #117

Closed
hashworks opened this issue Oct 30, 2023 · 3 comments · Fixed by #119

Comments

@hashworks
Copy link

Describe the bug
Running gitlabber produces a warning that doesn't affect the functionality.

To Reproduce
gitlabber -r .

/usr/bin/gitlabber:33: UserWarning: Calling a `list()` method without specifying `get_all=True` or `iterator=True` will return a maximum of 20 items. Your query returned 20 of 30 items. See https://python-gitlab.readthedocs.io/en/v4.0.0/api-usage.html#pagination for more details. If this was done intentionally, then this warning can be supressed by adding the argument `get_all=False` to the `list()` call. (python-gitlab: /usr/lib/python3.11/site-packages/gitlab/client.py:956)
  sys.exit(load_entry_point('gitlabber==1.1.9', 'console_scripts', 'gitlabber')())
* loading tree: 55.6%|████████████████████████████████████████████▍                                   | 20/36, group=common/usr/bin/gitlabber:33: UserWarning: Calling a `list()` method without specifying `get_all=True` or `iterator=True` will return a maximum of 20 items. Your query returned 20 of 41 items. See https://python-gitlab.readthedocs.io/en/v4.0.0/api-usage.html#pagination for more details. If this was done intentionally, then this warning can be supressed by adding the argument `get_all=False` to the `list()` call. (python-gitlab: /usr/lib/python3.11/site-packages/gitlab/client.py:956)
  sys.exit(load_entry_point('gitlabber==1.1.9', 'console_scripts', 'gitlabber')())
* loading tree: 81.0%|████████████████████████████████████████████████████████████████▊               | 64/79, group=playbooks/usr/bin/gitlabber:33: UserWarning: Calling a `list()` method without specifying `get_all=True` or `iterator=True` will return a maximum of 20 items. Your query returned 20 of 34 items. See https://python-gitlab.readthedocs.io/en/v4.0.0/api-usage.html#pagination for more details. If this was done intentionally, then this warning can be supressed by adding the argument `get_all=False` to the `list()` call. (python-gitlab: /usr/lib/python3.11/site-packages/gitlab/client.py:956)
  sys.exit(load_entry_point('gitlabber==1.1.9', 'console_scripts', 'gitlabber')())
* loading tree: 85.9%|████████████████████████████████████████████████████████████████████▋           | 85/99, group=roles/usr/bin/gitlabber:33: UserWarning: Calling a `list()` method without specifying `get_all=True` or `iterator=True` will return a maximum of 20 items. Your query returned 20 of 24 items. See https://python-gitlab.readthedocs.io/en/v4.0.0/api-usage.html#pagination for more details. If this was done intentionally, then this warning can be supressed by adding the argument `get_all=False` to the `list()` call. (python-gitlab: /usr/lib/python3.11/site-packages/gitlab/client.py:956)
  sys.exit(load_entry_point('gitlabber==1.1.9', 'console_scripts', 'gitlabber')())

Expected behavior
No warnings appear.

Versions (please complete the following information):

  • OS: Arch Linux
  • Python Version: 3.11.5
  • Gitlabber version: 1.1.9

Additional context
Dep python-gitlab is on version 4.0.0.

@flolauck
Copy link
Contributor

flolauck commented Nov 7, 2023

In my case this bug actually does affect functionality.

When I run this on a group with 34 repos:

gitlabber -t $GITLAB_READ_API_TOKEN -r -n path -a exclude -u https://internal.domain.com -i '/project/group**' -m http .

only the first 20 are downloaded.

Versions:

  • OS: Ubuntu 22.04.2 LTS
  • Python Version: 3.9.18
  • Gitlabber version: 1.1.9
  • python-gitlab: 4.1.1

The problem is this https://python-gitlab.readthedocs.io/en/v4.1.1/api-usage.html#pagination

As a workaround I changed the three places where list() is called in gitlab_tree.py to include get_all=True as parameter in my local installation of gitlabber.

I took the liberty to create a pull request: #119

@chrisdlangton
Copy link

If this was a repo clone issues, adding to say it is also happening with pip or pipx install:

gitlabber:8: UserWarning: Calling a `list()` method without specifying `get_all=True` or `iterator=True` will return a maximum of 20 items. Your query returned 20 of 38 items. See https://python-gitlab.readthedocs.io/en/v4.2.0/api-usage.html#pagination for more details. If this was done intentionally, then this warning can be supressed by adding the argument `get_all=False` to the `list()` call. (python-gitlab: /Users/christopher.langton/.local/pipx/venvs/gitlabber/lib/python3.12/site-packages/gitlab/client.py:956)
  sys.exit(main())

@Ramorous
Copy link

Fixed Here: #119

@ezbz ezbz closed this as completed in #119 Jun 30, 2024
ezbz added a commit that referenced this issue Jul 2, 2024
….com causing warning in the logs

- Add E2E test for large groups (>21 subgroups) and large subgroups (>21 projects) to ensure get_all parameter works
- Fixes #117
- Fixes #128
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants