Documentation for __getattr__

[PaulDavis]

BPO	8722
Nosy	@terryjreedy, @ncoghlan, @merwok, @csabella
PRs	bpo-8722: Document __getattr__ behavior with AttributeError in property #4754 [3.6] bpo-8722: Document __getattr__ behavior with AttributeError in property (GH-4754) #5542 [3.7] bpo-8722: Document __getattr__ behavior with AttributeError in property (GH-4754) #5543
Files	example.py: Short example

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-02-05.03:27:03.936>
created_at = <Date 2010-05-15.04:10:57.779>
labels = ['interpreter-core', '3.8', 'type-feature', '3.7', 'docs']
title = 'Documentation for __getattr__'
updated_at = <Date 2018-02-05.03:27:03.935>
user = 'https://bugs.python.org/PaulDavis'

bugs.python.org fields:

activity = <Date 2018-02-05.03:27:03.935>
actor = 'ncoghlan'
assignee = 'docs@python'
closed = True
closed_date = <Date 2018-02-05.03:27:03.936>
closer = 'ncoghlan'
components = ['Documentation', 'Interpreter Core']
creation = <Date 2010-05-15.04:10:57.779>
creator = 'Paul.Davis'
dependencies = []
files = ['17349']
hgrepos = []
issue_num = 8722
keywords = ['patch', 'needs review']
message_count = 16.0
messages = ['105790', '105791', '113192', '113193', '113212', '307831', '307853', '307876', '307877', '307894', '307907', '311535', '311633', '311637', '311638', '311640']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'ncoghlan', 'dstanek', 'eric.araujo', 'docs@python', 'Paul.Davis', 'cheryl.sabella']
pr_nums = ['4754', '5542', '5543']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue8722'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[PaulDavis]

The docs for __getattr__ in the object model section could be more specific on the behavior when a @Property raises an AttributeError and there is a custom __getattr__ defined. Specifically, it wasn't exactly clear that __getattr__ would be invoked after a @Property was found and evaluated.

The attached script demonstrates the issue on OS X 10.6, Python 2.6.1

I'm thinking something along the lines of:

If the attribute search encounters an AttributeError (perhaps due to a @Property raising the error) the search is considered a failure and __getattr__ is invoked.

[PaulDavis]

I should mention, in example.py, it wasn't immediately clear that "print f.bing" would actually print instead of raising the AttributeError. As in, I had a property raising an unexpected error that happend to be an AttributeError. If its not an attribute error, the error is not swallowed. I can see the obvious argument for "AttributeError means not found", it just wasn't immediately obvious what was going on.

[terryjreedy]

The problem with changing 2.7 docs is that object access is different for old- and new-style properties. Does your example work if you remove 'object'? (IE, can old style classes have properties?)

For new-style classes, the example behavior is clear if you 1. know that object has a .__getattribute__ method inherited by everything when not overriden and 2. read the doc for that which says that __getattr__ is called whenever a __getattribute__ call raises AttributeError, which it does here by passing through the .get error.

For 3.x, I think in 3.3.2. Customizing attribute access,
object.__getattr__(self, name)
"Called when an attribute lookup has not found the attribute in the usual places (i.e. it is not an instance attribute nor is it found in the class tree for self). name is the attribute name. "

might be replaced by

"Called when self.__getattribute__(name) raise AttributeError because name is not an instance attribute, not found in the class tree for self, or is a property attribute whose .get() method raises AttributeError."

But this does not work for 2.7.

[terryjreedy]

/raise/raises/

I am pretty sure that when __getattribute__ is bypassed, so is __getattr__.

[merwok]

Old-style classes can’t have descriptors, hence no properties, static methods, class methods or super.

[terryjreedy]

Cheryl, thank you for reviving this, as it is still needed. A slightly revised example better illustrates the claim in the doc revision about when __getattr__ is called.

class Foo(object):

    def __init__(self):
        self.foo = 1
        self.data = {"bing": 4}

    def __getattr__(self, name):
        print(f'Getting {name}')
        return self.data.get(name)

    @property
    def bar(self):
        return 3

    @property
    def bing(self):
        raise AttributeError("blarg")

f = Foo()
print('foo', f.foo)
print('__str__', f.__str__)
print('bar', f.bar)
print('bing', f.bing)
f.__getattribute__('bing')

# prints
foo 1
__str__ <method-wrapper '__str__' of Foo object at 0x0000016712378128>
bar 3
Getting bing
bing 4
Traceback (most recent call last):
  File "F:\Python\a\tem2.py", line 24, in <module>
    f.__getattribute__('bing')
  File "F:\Python\a\tem2.py", line 17, in bing
    raise AttributeError("blarg")
AttributeError: blarg

[csabella]

Terry,

Thanks for clarifying with this example. I hadn't tried this when I was playing with the other example. I guess getattribute might be defined by a class, but generally wouldn't be called directly, so the use of getattr and getattribute and the raising of AttributeError is more for an attributeref (https://docs.python.org/3/reference/expressions.html#attribute-references) usage?

[terryjreedy]

Before testing, let alone documenting, the status quo, I would like to be sure that suppressing the exception is truly the intended behavior. Is there a way to get an annotated listing from git (given which patch, and therefore which person, is responsible for each line)? I will try asking on pydev.

Calling __getattr__ on property failure is a behavior of __getattribute__, not of the property, and I would expect object.__getattribute__ to be tested wherever object is, but I have not found such tests. If we do add a test, the best model in test_desc.py looks like `def test_module_subclasses(self):`. The test class would only need __getattr__ and the faulty property.

class Foo(object):
    def __getattr__(self, name):
        print(f'Getattr {name}')
        return True
    @property
    def bing(self):
        print('Property bing')
        raise AttributeError("blarg")

f = Foo()
print(f.bing)

#prints (which would be the log list in a test)
Property bing
Getattr bing
True

[terryjreedy]

The behavior and doc for __setattr__ and __delattr__ should also be checked.

[csabella]

> Is there a way to get an annotated listing from git (given which patch, and therefore which person, is responsible for each line)?

Which source did you want to look at? In github, if you go into any source, you can click on a line and it gives an option for 'git blame'. That shows the last commit change for each line. You can then click an icon to see a previous commit, etc. For the .rst sources, it's a little different and there is a Blame button at the top of the source that will bring up the same view (commit annotations to the left of the source) as right-clicking.

I had posted about git blame a few months ago on core mentorship and Carol Willing mentioned another tool to get all the changes by line. Here was her post:

Thanks for passing along the tip for others. You may also find the npm package git-guilt useful as it will display all the contributors to a particular line's history. https://www.npmjs.com/package/git-guilt https://www.npmjs.com/package/git-guilt

[terryjreedy]

Thanks. I normally look at source in my local clone with an editor. I found 'view blame' and 'view blame prior' on github.

[terryjreedy]

Nick, this is about better documenting the behavior of __get(set/del)attr__ in 3.x it relations to AttributeError in a property. I think I understand what it does and think the patch is correct. Could you either review or suggest someone else who better understands core behavior like this?

[ncoghlan]

New changeset d1f3181 by Nick Coghlan (Cheryl Sabella) in branch 'master':
bpo-8722: Document __getattr__ behavior with AttributeError in property (GH-4754)
d1f3181

[ncoghlan]

New changeset a8c25d1 by Nick Coghlan (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-8722: Document __getattr__ behavior with AttributeError in property (GH-5542)
a8c25d1

[ncoghlan]

New changeset fea0a12 by Nick Coghlan (Miss Islington (bot)) in branch '3.7':
[3.7] bpo-8722: Document __getattr__ behavior with AttributeError in property (GH-5543)
fea0a12

[ncoghlan]

Thanks for the patch Cheryl, and for the reviews Terry!