New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shorter syntax for Optional[...] #429

Open
JukkaL opened this Issue May 11, 2017 · 16 comments

Comments

Projects
None yet
9 participants
@JukkaL
Contributor

JukkaL commented May 11, 2017

Multiple people have suggested a shorter syntax to replace Optional[...]. The current syntax will become more inconvenient if we don't infer optional types from None default values (see #275).

Here are various options that I remember seeing proposed:

  1. x? or ?x

This would require new Python syntax.

Hack uses ?x. TypeScript uses x? for names, not types, if I've understood things correctly.

  1. x | None

This would require support for | for all type objects, which would not be backward compatible.

The | operator is used by TypeScript.

  1. {x}

Probably not an option as this is too cryptic. Suggested in #151 .

  1. from typing import Optional as O or from typing import Optional as Opt

This is not actually a new syntax and works currently. These are arguably inelegant.

@kirbyfan64

This comment has been minimized.

Show comment
Hide comment
@kirbyfan64

kirbyfan64 May 11, 2017

Contributor

FWIW most modern languages that support optional/non-nullable types use ? (e.g. Crystal, VerboseScriptTypeScript, Swift, Dart, etc.).

Contributor

kirbyfan64 commented May 11, 2017

FWIW most modern languages that support optional/non-nullable types use ? (e.g. Crystal, VerboseScriptTypeScript, Swift, Dart, etc.).

@dmoisset

This comment has been minimized.

Show comment
Hide comment
@dmoisset

dmoisset May 12, 2017

One alternative in the line of x|None (i.e. improving the syntax for Union rather than Optional) and taking from 3, would be to make: [T1, T2, T3] a shorthand for Union[T1, T2, T3]; so instead of Optional[T] you could use [T, None]

The same could be applied with tuples of types (round brackets) or sets of types (curly brackets).

dmoisset commented May 12, 2017

One alternative in the line of x|None (i.e. improving the syntax for Union rather than Optional) and taking from 3, would be to make: [T1, T2, T3] a shorthand for Union[T1, T2, T3]; so instead of Optional[T] you could use [T, None]

The same could be applied with tuples of types (round brackets) or sets of types (curly brackets).

@markshannon

This comment has been minimized.

Show comment
Hide comment
@markshannon

markshannon May 12, 2017

I don't like the list form [T1, T2] as that implies a record type to me. The set form {T1, T2} seems more natural for a union.

markshannon commented May 12, 2017

I don't like the list form [T1, T2] as that implies a record type to me. The set form {T1, T2} seems more natural for a union.

@markshannon

This comment has been minimized.

Show comment
Hide comment
@markshannon

markshannon May 12, 2017

So here are the options proposed so far (that I am aware of).

def foo(x:Optional[int]=None): ... Status quo -- works out of the box
def foo(x:{int}=None): ... A bit cryptic -- works out of the box
def foo(x:int|None=None):... Involves major modification on int
def foo(x:int or None=None):... Works out of the box -- Loses or None when evaluated
def foo(x:{int, None}=None):... Works out of the box
def foo(x:[int, None]=None):... Works out of the box -- [int,None] != [None,int]?

Note that all but the first two are general to any union type and not specific to a union with NoneType.

markshannon commented May 12, 2017

So here are the options proposed so far (that I am aware of).

def foo(x:Optional[int]=None): ... Status quo -- works out of the box
def foo(x:{int}=None): ... A bit cryptic -- works out of the box
def foo(x:int|None=None):... Involves major modification on int
def foo(x:int or None=None):... Works out of the box -- Loses or None when evaluated
def foo(x:{int, None}=None):... Works out of the box
def foo(x:[int, None]=None):... Works out of the box -- [int,None] != [None,int]?

Note that all but the first two are general to any union type and not specific to a union with NoneType.

@ilevkivskyi

This comment has been minimized.

Show comment
Hide comment
@ilevkivskyi

ilevkivskyi May 12, 2017

Collaborator

x? or ?x for variables is not sufficient, since optional types can appear inside other types (like Callable[..., Optional[int]] or List[Optional[str]]). Also I think that with int? syntax (like in Swift) it is easy to miss the ? at the end. I don't like the [T1, T2] syntax either, since it looks ugly in Callable, for example Callable[[[int, None]], [int, None]].

In general I don't think the "verbosity" is a problem, I have worked with --strict-optional for some time I didn't notice this. I have rather noticed that people often use None where it can be avoided (PEP 526 partially solves this problem).

Collaborator

ilevkivskyi commented May 12, 2017

x? or ?x for variables is not sufficient, since optional types can appear inside other types (like Callable[..., Optional[int]] or List[Optional[str]]). Also I think that with int? syntax (like in Swift) it is easy to miss the ? at the end. I don't like the [T1, T2] syntax either, since it looks ugly in Callable, for example Callable[[[int, None]], [int, None]].

In general I don't think the "verbosity" is a problem, I have worked with --strict-optional for some time I didn't notice this. I have rather noticed that people often use None where it can be avoided (PEP 526 partially solves this problem).

@ilinum

This comment has been minimized.

Show comment
Hide comment
@ilinum

ilinum May 26, 2017

I can't seem to see where the x? and ?x is not sufficient for expressing any optional type. If you want to have a List[Optional[str]] it would just be List[str?]. If you want Optional[List[str]], it will be List[str]?.

Here are my two cents on the issue: the current syntax is a little more explicit, so it's harder to miss. However, if we go for a less verbose syntax, people will get used to it pretty fast and will not miss the optional types.

At the same time, I think it's okay to keep the current syntax. If you're just starting to use types, then the explicit syntax is much easier to understand whereas any other syntax takes more time getting used to.

ilinum commented May 26, 2017

I can't seem to see where the x? and ?x is not sufficient for expressing any optional type. If you want to have a List[Optional[str]] it would just be List[str?]. If you want Optional[List[str]], it will be List[str]?.

Here are my two cents on the issue: the current syntax is a little more explicit, so it's harder to miss. However, if we go for a less verbose syntax, people will get used to it pretty fast and will not miss the optional types.

At the same time, I think it's okay to keep the current syntax. If you're just starting to use types, then the explicit syntax is much easier to understand whereas any other syntax takes more time getting used to.

@ambv

This comment has been minimized.

Show comment
Hide comment
@ambv

ambv Jun 20, 2017

Contributor

I'm +1 on either str? or ?str. Very short and obvious.

Contributor

ambv commented Jun 20, 2017

I'm +1 on either str? or ?str. Very short and obvious.

@gvanrossum

This comment has been minimized.

Show comment
Hide comment
@gvanrossum

gvanrossum Jun 20, 2017

Member
Member

gvanrossum commented Jun 20, 2017

@ethanhs

This comment has been minimized.

Show comment
Hide comment
@ethanhs

ethanhs Jun 21, 2017

Contributor

I am -1 on either ?str or str?. One of the great things about Python's design in my mind is that it doesn't use !bool_var. not bool_var may be more explicit, but for many reasons, its harder to miss and much more readable. It is very good at almost phrase like constructions, eg if not check: ... instead of if !check, which can be easily missed when reading or writing.

I have a few ideas to throw out (listing Jukka's first as I like Opt, and as a reference):

def foo(a: Optional[str]): ...
def foo(a: Opt[str]): ...
def foo(a: Non[str]): ...
def foo(a: Null[str]): ...
def foo(a: Maybe[str]): ...

I think of all of these Opt seems promising, but I don't think it is exactly there. The other one that seems interesting to me is Maybe as it is shorter, but not as much as other suggestions, however it has the advantage of being intuitive, memorable, easy to understand, and shorter than Optional.

Contributor

ethanhs commented Jun 21, 2017

I am -1 on either ?str or str?. One of the great things about Python's design in my mind is that it doesn't use !bool_var. not bool_var may be more explicit, but for many reasons, its harder to miss and much more readable. It is very good at almost phrase like constructions, eg if not check: ... instead of if !check, which can be easily missed when reading or writing.

I have a few ideas to throw out (listing Jukka's first as I like Opt, and as a reference):

def foo(a: Optional[str]): ...
def foo(a: Opt[str]): ...
def foo(a: Non[str]): ...
def foo(a: Null[str]): ...
def foo(a: Maybe[str]): ...

I think of all of these Opt seems promising, but I don't think it is exactly there. The other one that seems interesting to me is Maybe as it is shorter, but not as much as other suggestions, however it has the advantage of being intuitive, memorable, easy to understand, and shorter than Optional.

@JukkaL

This comment has been minimized.

Show comment
Hide comment
@JukkaL

JukkaL Jun 21, 2017

Contributor

@ethanhs Maybe[str] was one of the original ideas floated around before we chose Optional, but it was rejected because if was considered too clever, I think.

Contributor

JukkaL commented Jun 21, 2017

@ethanhs Maybe[str] was one of the original ideas floated around before we chose Optional, but it was rejected because if was considered too clever, I think.

@markshannon

This comment has been minimized.

Show comment
Hide comment
@markshannon

markshannon Jun 21, 2017

What is actually wrong with Optional[str]? And where is it a problem?

I think we ought to establish what problem we are solving before we attempt to solve it.

markshannon commented Jun 21, 2017

What is actually wrong with Optional[str]? And where is it a problem?

I think we ought to establish what problem we are solving before we attempt to solve it.

@JukkaL

This comment has been minimized.

Show comment
Hide comment
@JukkaL

JukkaL Jun 21, 2017

Contributor

My main problem with Optional[str] is that more complex types often are quite verbose. Here is an example function return type from a recent mypy PR:

Optional[Callable[[MethodSigContext], CallableType]]

If the return type would be Optional as well, it would look like this:

Optional[Callable[[MethodSigContext], Optional[CallableType]]]

This doesn't look very readable to me, though part of this because of Callable. Using type aliases can help, but it can also make code hard to read, as you need to find the definition of the alias to understand the type.

Here's the same type using type? syntax, for example:

Callable[[MethodSigContext], CallableType?]?

I don't like this, since it's hard to see that the Callable[...] value is optional. The final ? is next to ], so I have to find the matching [ to see which type is getting modified by the ?.

Here's with ?type:

?Callable[[MethodSigContext], ?CallableType]

This is pretty readable, but the ? prefix doesn't look very clean or Pythonic, I'd say. Here the ? is associated with name of the type instead of the closing square bracket, which arguably makes this easier to read than the previous example.

Here's the example using | None:

Callable[[MethodSigContext], CallableType | None] | None

It doesn't strike me as a significant win over Optional[...] in terms of readability.

Contributor

JukkaL commented Jun 21, 2017

My main problem with Optional[str] is that more complex types often are quite verbose. Here is an example function return type from a recent mypy PR:

Optional[Callable[[MethodSigContext], CallableType]]

If the return type would be Optional as well, it would look like this:

Optional[Callable[[MethodSigContext], Optional[CallableType]]]

This doesn't look very readable to me, though part of this because of Callable. Using type aliases can help, but it can also make code hard to read, as you need to find the definition of the alias to understand the type.

Here's the same type using type? syntax, for example:

Callable[[MethodSigContext], CallableType?]?

I don't like this, since it's hard to see that the Callable[...] value is optional. The final ? is next to ], so I have to find the matching [ to see which type is getting modified by the ?.

Here's with ?type:

?Callable[[MethodSigContext], ?CallableType]

This is pretty readable, but the ? prefix doesn't look very clean or Pythonic, I'd say. Here the ? is associated with name of the type instead of the closing square bracket, which arguably makes this easier to read than the previous example.

Here's the example using | None:

Callable[[MethodSigContext], CallableType | None] | None

It doesn't strike me as a significant win over Optional[...] in terms of readability.

@dmoisset

This comment has been minimized.

Show comment
Hide comment
@dmoisset

dmoisset Jun 21, 2017

The verbosity (and nesting of brackets) is one of the drawbacks, but I also like the chance to remove the confusion on Optional vs optional arguments (that may have a default or None but something else). That gives us two unrelated but slightly overlapping concepts use the same name, so users are confused (I have had to clarify this thing to every user I have taught to use mypy).

Something that could be done from a styleguide standpoint is use the | operator but recommend to put None first. So @JukkaL 's example would be:

None | Callable[[MethodSigContext], None | CallableType]

which I think makes it more visible which parts are nullable, and IMO is more readable then the current syntax given the removed nesting

dmoisset commented Jun 21, 2017

The verbosity (and nesting of brackets) is one of the drawbacks, but I also like the chance to remove the confusion on Optional vs optional arguments (that may have a default or None but something else). That gives us two unrelated but slightly overlapping concepts use the same name, so users are confused (I have had to clarify this thing to every user I have taught to use mypy).

Something that could be done from a styleguide standpoint is use the | operator but recommend to put None first. So @JukkaL 's example would be:

None | Callable[[MethodSigContext], None | CallableType]

which I think makes it more visible which parts are nullable, and IMO is more readable then the current syntax given the removed nesting

@markshannon

This comment has been minimized.

Show comment
Hide comment
@markshannon

markshannon Jun 21, 2017

It seems unavoidable that complex types will be lengthy. But longer does not necessarily mean harder to read.
Names like Optional and Callable may take a bit more space, but they are highly readable (at least for a native English speaker). ? requires me to mentally translate to "Optional".

I don't see that any of the suggestions are significantly more readable, and most aren't really any shorter apart from ?, and that is far too big a language change for a fairly unimportant feature.

markshannon commented Jun 21, 2017

It seems unavoidable that complex types will be lengthy. But longer does not necessarily mean harder to read.
Names like Optional and Callable may take a bit more space, but they are highly readable (at least for a native English speaker). ? requires me to mentally translate to "Optional".

I don't see that any of the suggestions are significantly more readable, and most aren't really any shorter apart from ?, and that is far too big a language change for a fairly unimportant feature.

@markshannon

This comment has been minimized.

Show comment
Hide comment
@markshannon

markshannon Jun 21, 2017

Regarding @dmoisset point that Optional is confusing w.r.t. to optional arguments, wouldn't a simple rename suffice?

from typing import Optional as NoneOr
x : NoneOr[str]

markshannon commented Jun 21, 2017

Regarding @dmoisset point that Optional is confusing w.r.t. to optional arguments, wouldn't a simple rename suffice?

from typing import Optional as NoneOr
x : NoneOr[str]
@dmoisset

This comment has been minimized.

Show comment
Hide comment
@dmoisset

dmoisset Jun 21, 2017

@markshannon I could do renames like that in my own code, but "personal" approaches are not useful on annotations, because those are essentially placed on interfaces. I would make my annotations harder to read for third parties using my library unless I'm using a "standard" alias, so that's why it makes sense to discuss this change as part of the standard.

dmoisset commented Jun 21, 2017

@markshannon I could do renames like that in my own code, but "personal" approaches are not useful on annotations, because those are essentially placed on interfaces. I would make my annotations harder to read for third parties using my library unless I'm using a "standard" alias, so that's why it makes sense to discuss this change as part of the standard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment