eclean: output InvalidDepstring info in findPackages #43

dol-sen · 2024-02-05T02:47:28Z

Add try/except pair to _deps_equal() calls to output relavent details causing the exception in order to aid the user to fix the issue.

Bug: https://bugs.gentoo.org/923439

zmedico · 2024-02-05T04:06:12Z

pym/gentoolkit/eclean/search.py

+                    file=sys.stderr,
+                )
+                print(pp.error(er))
+                exit(1)


I suppose if the binpkg was at fault here then we might behave as though _deps_equal returned False and just delete binpkg with possibly a warning message. Invalid ebuild deps being fatal seems reasonable though.

Yeah, a bad binpkg should get deleted. I figured this would be a good start to at least find out what was broken on his 2 systems. We can leave this PR open for awhile until we here back from the Stuart.

Plus I need to create some errors to properly test it.

srcshelton · 2024-02-05T19:19:53Z

I think there's a mistake in the update where pp.error() is called:

$ sudo eclean -p packages --changed-deps
 * Building file list for packages cleaning...
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/search.py", line 663, in findPackages
    if _deps_equal(
       ^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/search.py", line 536, in _deps_equal
    deps_b = use_reduce(deps_b, uselist=uselist, eapi=eapi_b, token_class=Atom)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/dep/__init__.py", line 942, in use_reduce
    result = _use_reduce_cached(
             ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/dep/__init__.py", line 633, in _use_reduce_cached
    raise InvalidDependString(
portage.exception.InvalidDependString: expected: dependency string, got: ')', token 65

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.11/eclean", line 41, in <module>
    main()
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/cli.py", line 737, in main
    doAction(action, options, exclude=exclude, output=output)
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/cli.py", line 565, in doAction
    clean_me, invalids = findPackages(
                         ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/search.py", line 674, in findPackages
    pp.error("findPackages", "InvalidDependString found for: %s" % cpv),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: error() takes 1 positional argument but 2 were given

dol-sen · 2024-02-05T19:42:37Z

Yeah, that was my bad :(
OK, fixed and re-pushed. git pull -f to re-test

srcshelton · 2024-02-06T09:47:26Z

Ah - now this is useful (although is the "another exception" exception expected?)

$ sudo eclean -p packages --changed-deps
 * Building file list for packages cleaning...
!!! findPackages: InvalidDependString found for: app-editors/vim-8.2.4586

!!! findPackages: deps_binpkg: >=app-eselect/eselect-vi-1.1 >=sys-libs/ncurses-5.2-r2:0/6= ~app-editors/vim-core-8.2.4586 !<app-editors/vim-core-8.2.4328-r1 

!!! findPackages: deps_ebuild: >=app-eselect/eselect-vi-1.1 >=sys-libs/ncurses-5.2-r2:0= nls? ( virtual/libintl ) acl? ( kernel_linux? ( sys-apps/acl ) ) crypt? ( dev-libs/libsodium:= ) cscope? ( dev-util/cscope ) gpm? ( >=sys-libs/gpm-1.19.3 ) lua? ( lua_single_target_luajit? ( dev-lang/luajit:= ) lua_single_target_lua5-1? ( dev-lang/lua:5.1 ) lua_single_target_lua5-1? ( dev-lang/lua:5.1[deprecated] ) ) ~app-editors/vim-core-8.2.4586 !<app-editors/vim-core-8.2.4328-r1 vim-pager? ( app-editors/vim-core[-minimal] ) perl? ( dev-lang/perl:= ) python? ( python_single_target_python3_10? ( dev-lang/python:3.10[threads(+)] ) ) racket? ( dev-scheme/racket ) ruby? ( || ( ) ) selinux? ( sys-libs/libselinux ) sound? ( media-libs/libcanberra ) tcl? ( dev-lang/tcl:0= ) X? ( x11-libs/libXt ) 

Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/search.py", line 663, in findPackages
    if _deps_equal(
       ^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/search.py", line 536, in _deps_equal
    deps_b = use_reduce(deps_b, uselist=uselist, eapi=eapi_b, token_class=Atom)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/dep/__init__.py", line 942, in use_reduce
    result = _use_reduce_cached(
             ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/dep/__init__.py", line 633, in _use_reduce_cached
    raise InvalidDependString(
portage.exception.InvalidDependString: expected: dependency string, got: ')', token 65

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.11/eclean", line 41, in <module>
    main()
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/cli.py", line 737, in main
    doAction(action, options, exclude=exclude, output=output)
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/cli.py", line 565, in doAction
    clean_me, invalids = findPackages(
                         ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gentoolkit/eclean/search.py", line 685, in findPackages
    print(pp.error(er))
          ^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gentoolkit/pprinter.py", line 69, in error
    return output.red("!!! ") + string + "\n"
           ~~~~~~~~~~~~~~~~~~~^~~~~~~~
TypeError: can only concatenate str (not "InvalidDependString") to str

… is looks as if the issue is an empty Ruby dependency - so I guess my follow-on questions are:

Should an empty dependency such as this be a full-blown error, or just a warning, or be silently accepted?
Can this legitimately happen on platforms (such as this one, ARM64) which aren't as broadly keyworded as others, where there genuinely is no unmasked ruby package at any time?

thesamesam · 2024-02-06T23:53:23Z

In ::gentoo, it would be a CI error for that to happen. The issue is that with --changed-deps, presumably app-editors/vim-8.2.4586 from a local repository got re-evaluated (to compare the dependencies), and the eclass no longer supports old Rubys.

For cases like this, we should probably warn-and-move-on, as the user may not want to lose their binpkg built against old Ruby?

srcshelton · 2024-02-07T08:25:29Z

In ::gentoo, it would be a CI error for that to happen. The issue is that with --changed-deps, presumably app-editors/vim-8.2.4586 from a local repository got re-evaluated (to compare the dependencies), and the eclass no longer supports old Rubys.

In this case, even though there was an overlay which contained an old vim ebuild (now removed!), it's also the case that the binhost isn't regularly cleaned, and this binary package was built a long time ago - so is this a situation with could occur even with a completely standard setup?

For cases like this, we should probably warn-and-move-on, as the user may not want to lose their binpkg built against old Ruby?

I agree with this!

Add try/except pair to _deps_equal() to output relavent details causing the exception in order to aid the user to fix the issue. Mark binpkg dep failures as a non match for possible deletion. Make the ebuild dep failure a warning only, return True to save the binpkg. Add parameter docstring info Bug: https://bugs.gentoo.org/923439 Signed-off-by: Brian Dolbec <dolsen@gentoo.org>

Signed-off-by: Brian Dolbec <dolsen@gentoo.org>

dol-sen · 2024-02-18T02:21:08Z

Pushed to primary gentoo repo

dol-sen force-pushed the master branch 2 times, most recently from bc81ea9 to 69e7b7b Compare February 5, 2024 03:28

zmedico reviewed Feb 5, 2024

View reviewed changes

dol-sen force-pushed the master branch from 69e7b7b to 8b81d8d Compare February 5, 2024 19:40

dol-sen force-pushed the master branch 2 times, most recently from 136f26a to d61304e Compare February 18, 2024 01:11

dol-sen added 3 commits February 17, 2024 17:38

Initial tests for _deps_equal()

844abd2

Signed-off-by: Brian Dolbec <dolsen@gentoo.org>

enalyze/rebuild.py: black cleanup

2bb8f0e

Signed-off-by: Brian Dolbec <dolsen@gentoo.org>

eclean/search.py: Fix find_packages docstring for invalid_paths

e3e2ff5

Signed-off-by: Brian Dolbec <dolsen@gentoo.org>

dol-sen force-pushed the master branch from d61304e to e3e2ff5 Compare February 18, 2024 01:40

thesamesam approved these changes Feb 18, 2024

View reviewed changes

dol-sen closed this Feb 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

eclean: output InvalidDepstring info in findPackages #43

eclean: output InvalidDepstring info in findPackages #43

dol-sen commented Feb 5, 2024

zmedico Feb 5, 2024

dol-sen Feb 5, 2024

srcshelton commented Feb 5, 2024 •

edited

dol-sen commented Feb 5, 2024

srcshelton commented Feb 6, 2024

thesamesam commented Feb 6, 2024

srcshelton commented Feb 7, 2024

dol-sen commented Feb 18, 2024

eclean: output InvalidDepstring info in findPackages #43

eclean: output InvalidDepstring info in findPackages #43

Conversation

dol-sen commented Feb 5, 2024

zmedico Feb 5, 2024

Choose a reason for hiding this comment

dol-sen Feb 5, 2024

Choose a reason for hiding this comment

srcshelton commented Feb 5, 2024 • edited

dol-sen commented Feb 5, 2024

srcshelton commented Feb 6, 2024

thesamesam commented Feb 6, 2024

srcshelton commented Feb 7, 2024

dol-sen commented Feb 18, 2024

srcshelton commented Feb 5, 2024 •

edited