New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for keyword-only arguments on Python 3+ #281

Closed
wants to merge 3 commits into
base: master
from

Conversation

Projects
None yet
8 participants
@malinoff

malinoff commented Nov 1, 2017

Pull Request Check List

  • Added tests for changed code.
  • New features have been added to our Hypothesis testing strategy.
  • Updated documentation for changed code.
  • Documentation in .rst files is written using semantic newlines.
  • Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives.
  • Changes (and possible deprecations) have news fragments in changelog.d.

Please, correct me if I checked something wrong.

@malinoff malinoff force-pushed the malinoff:issue-106 branch from c6ec91c to a2dcab3 Nov 1, 2017

@malinoff malinoff changed the title from Issue 106 to Added support for keyword-only arguments on Python 3+ Nov 1, 2017

@malinoff

This comment has been minimized.

malinoff commented Nov 1, 2017

@hynek I decided to go with a dedicated attr.ib() argument for the sake of simplicity: I don't like to overload init with more options than a simple on/off switch.
But if you think this is a bad API, let's discuss a better approach.

@hynek

This comment has been minimized.

Member

hynek commented Nov 1, 2017

Wow, thanks for tackling this!

Before I dive deeper and start nitpicking:

  • please make it kw_only it’s more readable and I’d like to set the precedent of not smashing words together German-style. :)
  • There was an example of how to solve this problem for Python 2: #106 (comment) and I think it’s worth pursuing. No need to make the feature Python 3-only and it should be rather easy to add? If you run into any problems implementing it, we can always back out again.
@codecov

This comment has been minimized.

codecov bot commented Nov 1, 2017

Codecov Report

Merging #281 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@          Coverage Diff          @@
##           master   #281   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           9      9           
  Lines         660    679   +19     
  Branches      138    144    +6     
=====================================
+ Hits          660    679   +19
Impacted Files Coverage Δ
src/attr/_make.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1bb8fbd...af1a70a. Read the comment docs.

@python-attrs python-attrs deleted a comment from codecov bot Nov 1, 2017

@malinoff

This comment has been minimized.

malinoff commented Nov 1, 2017

  • please make it kw_only

Done.

  • There was an example of how to solve this problem for Python 2: #106 (comment) and I think it’s worth pursuing. No need to make the feature Python 3-only and it should be rather easy to add? If you run into any problems implementing it, we can always back out again.

Or we could make Python 3 transition slightly faster :) With no jokes, although I'm against of supporting such unique Python 3 features on Python 2 I'll try to implement this fallback anyway.

@hynek

This comment has been minimized.

Member

hynek commented Nov 1, 2017

Ohhh…so you implemented it only on a per-attribute basis instead of per-class?

@hynek

This comment has been minimized.

Member

hynek commented Nov 1, 2017

(JFTR I have zero qualms leaving Python 2 behind in 2017 if the cost exceeds a certain threshold)

@malinoff

This comment has been minimized.

malinoff commented Nov 1, 2017

Ohhh…so you implemented it only on a per-attribute basis instead of per-class?

Yes. In my opinion, if we have a per-attribute way of specifying keyword-only attributes making a whole class keyword-only is easy:

from functools import partial
kw_attrib = partial(attr.ib, kw_only=True)
@attr.s
class Foo:
    bar = kw_attrib()
    baz = kw_attrib()

But maybe an additional attr.s option (kw_only) is worth implementing. I don't know, I feel no evil with partial approach.

@malinoff malinoff force-pushed the malinoff:issue-106 branch 3 times, most recently from dd7e95b to 57c5b0e Nov 1, 2017

@malinoff

This comment has been minimized.

malinoff commented Nov 1, 2017

@hynek regarding Python 2 emulation of keyword-only arguments, I've found https://github.com/pasztorpisti/kwonly-args.
Although it works to some extent, it still doesn't exactly match Python 3 behavior:

Python 2.7.14 (default, Oct  8 2017, 18:34:38)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from kwonly_args import first_kwonly_arg, KWONLY_REQUIRED
>>> @first_kwonly_arg('b')
... def foo(a, b=KWONLY_REQUIRED, c=KWONLY_REQUIRED, d=4):
...   pass
...
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/malinoff/Projects/opensource/attrs/venv27/lib/python2.7/site-packages/kwonly_args/__init__.py", line 108, in wrapper
    ', '.join(sorted(missing_kwonly_args))))
TypeError: foo() missing 2 keyword-only argument(s): b, c
>>> foo(1, 2, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/malinoff/Projects/opensource/attrs/venv27/lib/python2.7/site-packages/kwonly_args/__init__.py", line 108, in wrapper
    ', '.join(sorted(missing_kwonly_args))))
TypeError: foo() missing 2 keyword-only argument(s): b, c

vs

Python 3.6.1 (default, May 29 2017, 09:11:03)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def foo(a, *, b, c, d=4):
...   pass
...
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo() missing 1 required positional argument: 'a'
>>> foo(1, 2, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo() takes 1 positional argument but 3 were given

And the implementation is scary enough for me not wanting to translate it into the list of lines used to generate the __init__ body. Maybe somebody else has courage to do so.

What do you think about adding this as a Python 3 only feature for now?

@malinoff malinoff force-pushed the malinoff:issue-106 branch from 57c5b0e to af1a70a Nov 2, 2017

@malinoff

This comment has been minimized.

malinoff commented Nov 2, 2017

Rebased onto master.

@malinoff

This comment has been minimized.

malinoff commented Nov 2, 2017

@Tinche since you've been pretty active in the referenced issues, I'd like to hear your opinion too.

@hynek

This comment has been minimized.

Member

hynek commented Nov 4, 2017

As I’ve said, I don’t mind leaving Python 2 behind if it means too much work. @Tinche is currently having a life-or-death battle with a manflu, but this won’t make 17.3 anyways, so no pressure. :)

@leezu

This comment has been minimized.

leezu commented Nov 10, 2017

@malinoff Implementing the additional @attr.s option would be make this work nicely with @attr.s(auto_attribs=True), or am I overlooking something? I.e. it would remove the need for attr.ib calls..

Quoting an usage example from the docs

>>> import typing
>>> @attr.s(auto_attribs=True)
... class AutoC:
...     cls_var: typing.ClassVar[int] = 5  # this one is ignored
...     l: typing.List[int] = attr.Factory(list)
...     x: int = 1
...     foo: str = attr.ib(
...          default="every attrib needs a type if auto_attribs=True"
...     )
...     bar: typing.Any = None
@malinoff

This comment has been minimized.

malinoff commented Nov 10, 2017

@leezu having only an attr.s option locks you between def foo(a, b, c=3): and def foo(*, a, b, c=3):.
Having an attr.ib option gives you enough flexibility: def foo(a, *b, c=3): and allows to implement the corresponding attr.s option for short-cut and convenience.

Also, I've opened this PR before 17.3 have been released, so I couldn't take auto_attribs into account at that time.

@hynek @Tinche, since 17.3 was cut off, could you please take a closer took at this PR?

@RonnyPfannschmidt

This comment has been minimized.

RonnyPfannschmidt commented Nov 10, 2017

would it be acceptable for python2 support to use a **kw_somewhat_unreadable_123 name combined with a funcsigs2 signature object that would return the complete signature for correctness ?

(given that popping from the dict + asserting can work, but the signature would suck

@leezu

This comment has been minimized.

leezu commented Nov 11, 2017

@malinoff essentially I would like to avoid writing out attr.ib for every attribute (based on auto_attribs=True), but still be able to use keyword-only arguments.

Having an attr.ib option gives you enough flexibility: def foo(a, *b, c=3): and allows to implement the corresponding attr.s option for short-cut and convenience.

Do you mean the kw_attrib partial? It doesn't allow avoiding attr.ib (or well, now kw_attrib).

You can argue that using keyword-only arguments is an advanced use-case, so it should require the "more complex" attr.ib syntax instead of the type hint based syntax with auto_attribs=True. I think otherwise, but please let me know if you disagree.

having only an attr.s option locks you between def foo(a, b, c=3): and def foo(*, a, b, c=3):

What about not "only an attr.s option" but an "shortcut attr.s option" which defaults to def foo(*, a, b, c=3) and more complex use-cases can be implemented via the corresponding attr.ib calls?

@malinoff

This comment has been minimized.

malinoff commented Nov 11, 2017

@leezu

What about not "only an attr.s option" but an "shortcut attr.s option" which defaults to def foo(*, a, b, c=3) and more complex use-cases can be implemented via the corresponding attr.ib calls?

That's what I mean in my previous reply.

@hynek

This comment has been minimized.

Member

hynek commented Dec 8, 2017

So to come back to this: IIRC this would be the first option that exists only on the attributes but not for the whole class?

I think we should really allow both: in attr.ib to None (=default) and add an option to attr.s. If you don’t want to do it here, it’s fine: we can finish this one up but block 17.4. until the second part is done. At this moment, this has to be rebased.

@ronlut

This comment has been minimized.

ronlut commented Dec 21, 2017

Sorry for bursting into your conversation, but I hope it's related enough.
Will this PR (or combined with anything else already implemented) allow me to use attrs for something similar to this use case:

class A:
    def __init__(self, **kwargs):
        for key in kwargs:
            setattr(self, key, kwargs[key])

I guess it's more related to your plan to add an option to attr.s and not the attr.ib one.

@veltzerdoron

This comment has been minimized.

veltzerdoron commented Dec 24, 2017

I see this is not implemented in the code yet, will it be a part of version 17.4?

@hynek

This comment has been minimized.

Member

hynek commented Dec 24, 2017

Unlikely, 17.4 is gonna be a mostly (overdue) bug fix release that should drop next week if nothing comes up.

@hynek hynek referenced this pull request Jan 28, 2018

Closed

keyword-only attrs #335

@alanhdu

This comment has been minimized.

alanhdu commented Mar 25, 2018

@malinoff Is there anything else that needs to happen with this PR? I'm happy to take over the branch to try to get it in if you don't have the time).

(Although for my own use-case, I just need the class-level kw_only flag, but not the attr.ib level flag).

@asvetlov

This comment has been minimized.

asvetlov commented Mar 26, 2018

@malinoff

This comment has been minimized.

malinoff commented Mar 26, 2018

I'm okay to finish the PR by myself, I just don't understand what else should be implemented/fixed.

@alanhdu

This comment has been minimized.

alanhdu commented Apr 2, 2018

@hynek Would you mind commenting on what else needs to happen here (other than resolving the merge conflicts)? Is this blocked on the design issue of the whether this should also be an attr.s option?

(FWIW @malinoff I would definitely like the attr.s option so this works nicely with auto_attribs).

@alanhdu

This comment has been minimized.

alanhdu commented Apr 19, 2018

Sorry to be a bother, but bump @hynek about what more needs to happen here.

@hynek

This comment has been minimized.

Member

hynek commented Apr 21, 2018

I’m not gonna review the concrete code now, here’s the general requirements for this to be merged:

  • Python 2 is completely optional for new features if it would be too kludgy to implement. I cannot stress this enough.
  • There needs to be a global option like @attr.s(kw_only=True).

So it should work like this:

@attr.s(kw_only=True)
class C:
    x = attr.ib()
    y = attr.ib()

-> __init__(self, *, x, y)

@attr.s
class C:
    x = attr.ib()
    y = attr.ib(kw_only=True)

-> __init__(self, x, *, y)

I think it’s fair to expect that a non-kw_only attribute must not come after a kw_only attribute.

Am I miss something?

I’m sorry for not being very responsive.

(Please disregard my comments from before about setting attr.ib's kw_only to None – that would be inconsistent with the rest of attrs. It should be False of course.)

asford added a commit to uw-ipd/attrs that referenced this pull request Jul 24, 2018

hynek added a commit that referenced this pull request Aug 11, 2018

Added support for keyword-only arguments on Python 3+ [rebase] (#411)
* Added support for keyword-only attributes. Closes #106, and closes #38

(Rebases #281)

Co-authored-by: Alex Ford <fordas@uw.edu>

* Add `attr.s`-level `kw_only` flag.

Add `kw_only` flag to `attr.s` decorator, indicating that all class
attributes should be keyword-only in __init__.

Minor updates to internal interface of `Attribute` to support
evolution of attributes to `kw_only` in class factory.

Expand examples with `attr.s` level kw_only.

* Add `kw_only` to type stubs.

* Update changelog for rebased PR.

Hear ye, hear ye. A duplicate PR is born.

* Tidy docs from review.

* Tidy code from review.

* Add explicit tests of PY2 kw_only SyntaxError behavior.

* Add `PythonToOldError`, raise for kw_only on PY2.

* `Attribute._evolve` to `Attribute._assoc`.
@hynek

This comment has been minimized.

Member

hynek commented Aug 11, 2018

fixed by #411

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment