fix: DefinedNamespace: fixed handling of control attributes. #1825

aucampia · 2022-04-15T15:18:22Z

Summary of changes

This patch changes DefinedNamespace to always raise AttributeError for
the name _NS and other "control attributes" from __getattr__, and to
also not consider them as part of the __dir__.

Without doign this inspect.signature recurses
infinitely when inspecting rdflib.namespace.DefinedNamespace and dir
results in an AttributeError.

One situation in which this occurs is when sphinx autodoc is generating
documentation from type hints:

WARNING: error while formatting signature for rdflib.namespace.DefinedNamespace: Handler <function record_typehints at 0x7fbf2696dd40> for event 'autodoc-process-signature' threw an exception (exception: maximum recursion depth exceeded while calling a Python object)

Also:

Changed DefinedNamespace.__repr__ to use repr for formatting
the URI string instead of quoting by hand. This probably has no real effect,
as the namespace shoudl not have a double or single quote in it, but
it is still correct to use repr.

The main reason for this patch is to eliminate a warning for sphinx
which is blocking the build since sphinx.fail_on_warning was enabled.

There is also two cases in DefinedNamespaceMeta where an undefined
method is being called, these were causing type errors before, but I
added notes now to indicate they are real bugs, because if they are
reached they just result in AttributeError exceptions. This should be
fixed sometime, not sure what the reasoning behind it was.

Checklist

Checked that there aren't other open pull requests for
the same change.
Added tests for any changes that have a runtime impact.
Checked that all tests and type checking passes.
Considered granting push permissions to the PR branch,
so maintainers can fix minor issues and keep your PR up to date.

aucampia · 2022-04-15T15:20:24Z

This, together with #1823, is needed to make docs build pass on master.

aucampia · 2022-04-15T15:22:03Z

Stack trace sample:

Traceback (most recent call last):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/.venv/lib64/python3.9/site-packages/sphinx/ext/autodoc/typehints.py", line 30, in record_typehints
    sig = inspect.signature(obj, type_aliases=app.config.autodoc_type_aliases)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/.venv/lib64/python3.9/site-packages/sphinx/util/inspect.py", line 637, in signature
    signature = inspect.signature(subject, follow_wrapped=follow_wrapped)
  File "/usr/lib64/python3.9/inspect.py", line 3113, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
  File "/usr/lib64/python3.9/inspect.py", line 2862, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
  File "/usr/lib64/python3.9/inspect.py", line 2295, in _signature_from_callable
    partialmethod = obj._partialmethod
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 235, in __contains__
    if item_str.startswith(str(cls._NS)):
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 219, in __getattr__
    return cls.__getitem__(name)
  File "/home/iwana/sw/d/github.com/iafork/rdflib.cleanish/rdflib/namespace/__init__.py", line 208, in __getitem__
    if (cls._warn or cls._fail) and name not in cls:

aucampia · 2022-04-15T17:49:45Z

I will actually add one more test to verify that inspect.signature on the namespace returns.

aucampia · 2022-04-15T18:19:53Z

I think there are deeper issues here, adding more tests.

aucampia · 2022-04-15T18:51:14Z

Okay, this fix is slightly wrong, reworking it.

aucampia · 2022-04-15T19:21:11Z

Okay it seems DefinedNamespace is severely broken, most of the super() stuff just will never run :/

aucampia · 2022-04-15T23:05:59Z

The remaining warning from sphinx is fixed in #1823.

ghost

Trickier than it looked at first.

aucampia · 2022-04-17T17:36:24Z

Trickier than it looked at first.

Yes, also went down a couple of dead ends 😅 and fixed a couple of none bugs first - thanks for the review.

This patch changes `DefinedNamespace` to always raise `AttributeError` for the name `_NS` and other "control attributes" from `__getattr__`, and to also not consider them as part of the `__dir__`. Without doign this `inspect.signature` recurses infinitely when inspecting `rdflib.namespace.DefinedNamespace` and `dir` results in an `AttributeError`. One situation in which this occurs is when sphinx autodoc is generating documentation from type hints: ``` WARNING: error while formatting signature for rdflib.namespace.DefinedNamespace: Handler <function record_typehints at 0x7fbf2696dd40> for event 'autodoc-process-signature' threw an exception (exception: maximum recursion depth exceeded while calling a Python object) ``` Also: - Changed `DefinedNamespace.__repr__` to use repr for formatting the URI string instead of quoting by hand. This probably has no real effect, as the namespace shoudl not have a double or single quote in it, but it is still correct to use repr. The main reason for this patch is to eliminate a warning for sphinx which is blocking the build since `sphinx.fail_on_warning` was enabled. There is also two cases in `DefinedNamespaceMeta` where an undefined method is being called, these were causing type errors before, but I added notes now to indicate they are real bugs, because if they are reached they just result in `AttributeError` exceptions. This should be fixed sometime, not sure what the reasoning behind it was.

aucampia · 2022-04-18T21:23:03Z

I will merge this tomorrow morning (CET) if there is no further feedback as this is blocking docs build and preview.

aucampia requested review from nicholascar, ashleysommer, a user and white-gecko April 15, 2022 15:18

aucampia force-pushed the iwana-20220415T1648-defined_namespace_partialmethod branch from 8200b21 to 637e106 Compare April 15, 2022 15:19

aucampia mentioned this pull request Apr 15, 2022

docs: change term reference to italicized #1823

Merged

3 tasks

aucampia marked this pull request as draft April 15, 2022 18:19

aucampia force-pushed the iwana-20220415T1648-defined_namespace_partialmethod branch 2 times, most recently from f4be36b to d0cbbfb Compare April 15, 2022 21:50

aucampia changed the title ~~fix: DefinedNamespace: always raise AttributeError for _partialmethod~~ fix: DefinedNamespace: always raise AttributeError for _NS from __getattr__ Apr 15, 2022

aucampia force-pushed the iwana-20220415T1648-defined_namespace_partialmethod branch 2 times, most recently from e36f2b4 to 0a5957e Compare April 15, 2022 22:02

aucampia changed the title ~~fix: DefinedNamespace: always raise AttributeError for _NS from __getattr__~~ fix: DefinedNamespace: fixed handling of _NS attribute Apr 15, 2022

aucampia force-pushed the iwana-20220415T1648-defined_namespace_partialmethod branch 4 times, most recently from 42fd6e9 to fc7267b Compare April 15, 2022 23:01

aucampia changed the title ~~fix: DefinedNamespace: fixed handling of _NS attribute~~ fix: DefinedNamespace: fixed handling of control attributes. Apr 15, 2022

aucampia requested a review from hsolbrig April 15, 2022 23:05

aucampia marked this pull request as ready for review April 15, 2022 23:06

aucampia added the fix Fixes an issue label Apr 15, 2022

aucampia force-pushed the iwana-20220415T1648-defined_namespace_partialmethod branch from fc7267b to 64ac2ff Compare April 16, 2022 08:24

ghost approved these changes Apr 17, 2022

View reviewed changes

aucampia force-pushed the iwana-20220415T1648-defined_namespace_partialmethod branch from 64ac2ff to fdc6479 Compare April 18, 2022 12:50

aucampia mentioned this pull request Apr 18, 2022

docs: switch to sphinx-autodoc-typehints #1854

Merged

3 tasks

aucampia merged commit e30e386 into RDFLib:master Apr 19, 2022

aucampia deleted the iwana-20220415T1648-defined_namespace_partialmethod branch May 7, 2022 09:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: DefinedNamespace: fixed handling of control attributes. #1825

fix: DefinedNamespace: fixed handling of control attributes. #1825

aucampia commented Apr 15, 2022 •

edited

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

ghost left a comment

aucampia commented Apr 17, 2022

aucampia commented Apr 18, 2022

fix: DefinedNamespace: fixed handling of control attributes. #1825

fix: DefinedNamespace: fixed handling of control attributes. #1825

Conversation

aucampia commented Apr 15, 2022 • edited

Summary of changes

Checklist

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

aucampia commented Apr 15, 2022

ghost left a comment

Choose a reason for hiding this comment

aucampia commented Apr 17, 2022

aucampia commented Apr 18, 2022

aucampia commented Apr 15, 2022 •

edited