Metaclasses under Python 3

[gilch]

When pondering #850, it occurred to me that I don't know how to set a metaclass in Hy. (It doesn't come up much.) I mean in Python2 we can do it with the old syntax (__metaclass__ = Meta in the class body), which is straightforward in Hy. But this syntax isn't allowed in Python3. After reading the docs, I don't think Hy can do this. That or the docs need some updating. It's an issue either way.

[zackmdavis]

Good observation; I agree that this is currently impossible. With the Python 3 metaclass-definition syntax, the metaclass keyword argument ends up in the keywords attribute of ast.ClassDef, which is unconditionally empty in the classdefs we produce.

in [2]: class the_trivial_metaclass(type): pass
in [3]: ast.parse("class Foo(metaclass=the_trivial_metaclass): pass").body[0].keywords[0].value.id
Out[3]: 'the_trivial_metaclass'

[gilch]

I guess the most obvious syntax would be.

(defclass Foo [Parent :metaclass Meta :spam "spam"]
...)

[algernon]

How about this:

(defclass Foo [Parent1 Parent2 [:meta MyMeta]])
  ...

The reason for the extra brackets is that this makes it stand out a bit more, which makes it easier (for the human eye) to notice, especially when one adds more parent classes after:

(defclass Foo [P1 [:meta Meta] P2]
  ...)

vs

(defclass Foo [P1 :meta Meta P2]
  ...)

Negligible difference, mind you, but I believe bracket-grouping these kinds of things is a useful thing. Also, this way I can just iterate over the members of the parent list, and if I find a list, I'll handle that specially. If we had it inlined, I'd have to check each element and sometimes pop one off, sometimes two. With a bracket, we pop one off every time, and maybe treat it specially. Again, not much of a difference, but still.

Since this is not likely to be used all that much, the extra brackets shouldn't be much of an issue to type.

[gilch]

No, you can't put the args after the kwargs in a Python class definition, because it uses the same grammar rule as a function call. Why would we want to allow this in Hy? This would get even more confusing when you use more keywords. Let's just be consistent with Hy's function call syntax and do it the same way.

[gilch]

Maybe I should explain myself better. Metaclasses are really not that hard, but they're perhaps an arcane topic.

In Python, a class definition is just syntactic sugar for a function call:

X = type('X', (object,), dict(a=1))
# same as
class X(object):
    a = 1

Like everything else in Python, classes are themselves objects, and they are returned by calls to type() (the 3-arg version; 1-arg just names the class).

Metaclasses just let you change which function you're calling. Here we've changed the default metaclass--from type, to my_meta:

>>> def my_meta(name,bases,namespace,**kwargs):
    print("in my_meta!")
    print(name,bases,namespace,kwargs)
    return "Not even a class!"

>>> class foo(metaclass=my_meta):pass

in my_meta!
foo () {'__module__': '__main__', '__qualname__': 'foo'} {}
>>> foo
'Not even a class!'
>>> class foo(str, int, spam="spam", metaclass=my_meta):pass

in my_meta!
foo (<class 'str'>, <class 'int'>) {'__module__': '__main__', '__qualname__': 'foo'} {'spam': 'spam'}
>>> # mixing order not allowed!
>>> class foo(str, metaclass=my_meta, int):pass
SyntaxError: non-keyword arg after keyword arg

See, just a function call. There is one more trick with the __prepare__ attribute:

>>> my_meta.__prepare__ = my_prepare_func

This lets you call an additional function to pre-populate the class attributes before the class body statements are executed. This function also gets the same kwargs that the metaclass gets.

[algernon]

Hrm. I brooded over this last night, and your reasoning makes sense. I'll prep a PR with a (defclass Foo [a b :meta m] ...) syntax.

[gilch]

No, you can't put the args after the kwargs in a Python class definition, because it uses the same grammar rule as a function call. Why would we want to allow this in Hy?

To answer my own question, it appears that Hy actually does allow positional arguments after keyword arguments.

=> (defn test [arg &kwargs kwargs])
def test(arg, **kwargs):
    pass
=> (test :foo "bar" "before")
test('before', foo='bar')

Even though this is expressly forbidden in Python:

>>> def test(arg, **kwargs):
    pass

>>> test(foo="bar", "before")
SyntaxError: non-keyword arg after keyword arg
>>> 

I wonder if this was intentional. This strange behavior is potentially useful given the tail-threading macro ->>, but also potentially confusing.

The helper function responsible for handling kwargs in function calls begins on line 445 in compiler.py. This might be a helpful reference for kwarg support in defclass.