Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure to parse some repos #1325

Closed
fanboimsft opened this issue Apr 4, 2023 · 5 comments
Closed

Failure to parse some repos #1325

fanboimsft opened this issue Apr 4, 2023 · 5 comments

Comments

@fanboimsft
Copy link

I have installed all the listed dependencies but still get an error while parsing some repos:

Apr 04 06:46:59 parsing arch
Apr 04 06:46:59 ERROR: failed: unsupported callable
Apr 04 06:47:27 parsing ataraxia
Apr 04 06:47:27 ERROR: failed: unsupported callable

It doesn't happen with every repo, for example AUR works fine:

Apr 04 06:47:13 parsing aur
Apr 04 06:47:26 done
Apr 04 06:47:30 parsing calculate
Apr 04 06:47:30 done

I don't see any other errors in the logs

@AMDmi3
Copy link
Member

AMDmi3 commented Apr 4, 2023

Try with --fatal to get a backtrace. Which python version do you have?

@fanboimsft
Copy link
Author

fanboimsft commented Apr 4, 2023

I have Python 3.11.2

--fatal log:

Traceback (most recent call last):
  File "/usr/lib/python3.11/inspect.py", line 1369, in getfullargspec
    sig = _signature_from_callable(func,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/inspect.py", line 2519, in _signature_from_callable
    return _signature_from_builtin(sigcls, obj,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/inspect.py", line 2326, in _signature_from_builtin
    raise ValueError("no signature found for builtin {!r}".format(func))
ValueError: no signature found for builtin <slot wrapper '__init__' of 'object' objects>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/new/root/repology-updater/./repology-update.py", line 410, in <module>
    sys.exit(main())
             ^^^^^^
  File "/new/root/repology-updater/./repology-update.py", line 387, in main
    process_repositories(env)
  File "/new/root/repology-updater/./repology-update.py", line 190, in process_repositories
    env.get_repo_processor().parse([reponame], transformer=transformer, maintainermgr=maintainermgr, logger=runlogger)
  File "/new/root/repology-updater/repology/repoproc/__init__.py", line 250, in parse
    self._parse(repository, transformer, maintainermgr, logger)
  File "/new/root/repology-updater/repology/repoproc/__init__.py", line 226, in _parse
    serializer.serialize(self._iter_parse_all_sources(repository, transformer, maintainermgr, logger))
  File "/new/root/repology-updater/repology/repoproc/serialization.py", line 59, in serialize
    for package in packages:
  File "/new/root/repology-updater/repology/repoproc/__init__.py", line 191, in _iter_parse_all_sources
    yield from self._iter_parse_source(repository, source, transformer, maintainermgr, logger.get_indented())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/new/root/repology-updater/repology/repoproc/__init__.py", line 173, in _iter_parse_source
    self.parser_factory.spawn_with_known_args(
  File "/new/root/repology-updater/repology/moduleutils.py", line 73, in spawn_with_known_args
    filtered_kwargs = {
                      ^
  File "/new/root/repology-updater/repology/moduleutils.py", line 74, in <dictcomp>
    key: value for key, value in kwargs.items() if key in inspect.getfullargspec(class_.__init__).args
                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/inspect.py", line 1379, in getfullargspec
    raise TypeError('unsupported callable') from ex
TypeError: unsupported callable

log.txt

@AMDmi3
Copy link
Member

AMDmi3 commented Apr 4, 2023

Please provide more information about your system.

Also, does this work for you?

import inspect
inspect.getfullargspec(object.__init__)

@fanboimsft
Copy link
Author

It's a custom fork of KISS Linux, recompiling python without the "--without-doc-strings" flag fixed the issue.

Thanks for helping me figure this out!

AMDmi3 added a commit that referenced this issue Apr 7, 2023
Historically, we've had fetcher and parser args mixed up in
the same config dict like this:

    fetcher: FooFetcher
    parser: BarParser
    some_fetcher_arg: 34
    some_parser_arg: 12

so we've had to check which arguments parser/fetcher constructor
actually understands and pass only accepted args when spawning a
class. Currently, parser and fetcher specifications are separate:

    fetcher:
      class: FooFetcher
      some_fetcher_arg: 34
    parser:
      class: BarParser
      some parser_arg: 12

and, apart from the class name, (should) only contain valid constructor
arguments, thus inspect.getfullargspec machinery is no longer needed,
so remove it, especially given that it's known to break in some
cases (#1325).

This also makes parser/fetcher argument handling stricter, which
has revealed one problem with amazon linux repo config, specifying
args which are not really supported by the parser.
@AMDmi3
Copy link
Member

AMDmi3 commented Apr 7, 2023

I doubt --without-doc-strings was the cause, as in the python source it looks like it just effectively replaces builtin docstrings with empty strings, so it shouldn't introduce any semantic changes. But it could be some other custom flag.
Regardless, I've removed the problematic code which is no longer needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants