runtime(python): highlight ellipsis literals

[jparise]

The ellipsis literal (...) can be used in multiple contexts:

• Placeholders: class Foo: ...
• Containers: Tuple[int, ...]
• Assignments: x = ...

This is a trickier pattern to match because we can't rely on keyword boundaries, so we instead look for exactly three dots (...).

This does mean that we will match the ... portion of x...x, which isn't valid Python syntax, but I think that's an acceptable trade-off that avoids making this pattern much more complex.

Ref: https://docs.python.org/3/library/constants.html#Ellipsis

[jparise]

@zvezdan with this, I believe we now cover all of the builtin constants.

[zvezdan]

This does mean that we will match the ... portion of x...x, which isn't valid Python syntax, but I think that's an acceptable trade-off that avoids making this pattern much more complex.

It can be used in doctest expected result line(s) to indicate anything in between.
See this example from the documentation, specifically the part

"""Some description.

>>> exec(s)  #doctest: +ELLIPSIS
-3.21716034272e-0...7
"""

This, unfortunately, shows why the matching needs to be more complex.
In an example like this:

"""A doctest

>>> class MyClass:
...     def __init__(self, x):
...         try:
...             self.a = c
...         except ValueError:
...             self.b = c

"""

now shows all the continuation lines as ellipsis and they should be the same highlight as a regular code and >>>.

[jparise]

This does mean that we will match the ... portion of x...x, which isn't valid Python syntax, but I think that's an acceptable trade-off that avoids making this pattern much more complex.

It can be used in doctest expected result line(s) to indicate anything in between. See this example from the documentation, specifically the part

"""Some description.

>>> exec(s)  #doctest: +ELLIPSIS
-3.21716034272e-0...7
"""

Ah, I wasn't aware of +ELLIPSIS! To confirm, the ... is not matched/highlighted with my proposed change, but you're suggesting it should be, correct?

This, unfortunately, shows why the matching needs to be more complex. In an example like this:

"""A doctest

>>> class MyClass:
...     def __init__(self, x):
...         try:
...             self.a = c
...         except ValueError:
...             self.b = c

"""

now shows all the continuation lines as ellipsis and they should be the same highlight as a regular code and >>>.

Yes, this is definitely flawed. I'll work on a solution to this case.

[jparise]

"""A doctest

>>> class MyClass:
...     def __init__(self, x):
...         try:
...             self.a = c
...         except ValueError:
...             self.b = c

"""

now shows all the continuation lines as ellipsis and they should be the same highlight as a regular code and >>>.

@zvezdan, I believe I've addressed this in the most recent commit. I took this approach:

1. Moved the ellipsis literal matching into a distinct pythonEllipsis group
2. Linked that group to pythonBuiltin
3. Added a doctest-aware pythonEllipsis that is used in pythonDoctest contexts

[dkearns]

This won't match if it follows another syntax match. E.g. type(...) if the parens are syntax matches. You might want to use a lookahead as well for symmetry but it's not required.

Suggested change

	syn match pythonEllipsis "\%(^\|[^.]\)\zs\.\.\.\ze\%([^.]\|$\)" display
	syn match pythonBuiltin "\.\@1<!.\.\.\ze\%([^.]\|$\)" display

[jparise]

That's a very nice suggestion. Thanks! I applied it to the latest revision.

(It still needs to be pythonEllipsis due to my most recent change for doctest awareness.)

[dkearns]

Sorry I didn't intend to suggest renaming the group. I think that may have been the name in an earlier draft and it was changed underneath my other open comment window.

[jparise]

This does mean that we will match the ... portion of x...x, which isn't valid Python syntax, but I think that's an acceptable trade-off that avoids making this pattern much more complex.

It can be used in doctest expected result line(s) to indicate anything in between. See this example from the documentation, specifically the part

"""Some description.

>>> exec(s)  #doctest: +ELLIPSIS
-3.21716034272e-0...7
"""

The latest revision now correctly highlights ... for doctest values like this. (It's always on and isn't conditional on +ELLIPSIS.) I also added a few test covers to cover these examples.

[chrisbra]

thanks