More Literal Types...

[jdunkerley]

We currently have literal support for Number and Text.

We have previously spoken about adding ``` (backtick) to be identified as an expression (so we can make it explicit rather than implied).

Following a discussion with @GregoryTravis and @radeusgd, it would be very useful to follow JavaScript's lead and support a regex literal. They use / to identify this. We can then retire all of the use_regex options as we would get a Regex.Pattern. It can also then be used to do syntax highlighting within the IDE (as would also be the case with backticks above).

Additionally, a literal format for date and time objects would be a great addition to a data product. I appreciate not many general purpose languages allow this but for our use case this would be fantastic. Something like:

• d'2021-03-02' ==> Date
• t'12:00:00' ==> Time_Of_Day
• dt'2021-03-02 12:00:00' ==> Date_Time
  (Any Enso supported literal Date/Time format (ISO+))

cc: @sylwiabr @wdanilo @JaroslavTulach @hubertp

[radeusgd]

While the backtick for expression syntax and likely regex will need special handling anyway, as for the date-time literals, I'd suggest to consider what Scala does: essentially allowing to define a 'special function' for handling string literals (there it's main usage is string interpolation, but that is another topic).

So one can define d'2021-03-02' just by defining some special form of named conversion Text.literal_d (text : Text) -> Date = Date.parse ... (the actual syntax would need some thought).

Essentially the idea is to allow users to define custom string literals, identified by their prefix - i.e. foo'abc' will pass 'abc' to some function identified by the name foo.

I'm not 100% convinced this is right for Enso as it may make the code less readable if we have 100s of literals, so maybe just the few provided for date-time are enough for us. But I thought it may be at least worth considering as it offers some flexibility.

[radeusgd]

One of nice usecases of the custom String interpolation in Scala is allowing for very nicely defining prepared statements in SQL, where one can do sql"SELECT * FROM x WHERE a = ${my_variable}" and thanks to the magic of custom sql string interpolation, this creates a statement that is not prone to SQL injection - because it is not doing a regular string interpolation, but actually constructing a prepared statement with a hole under the hood: SELECT * FROM x WHERE a = ? and filling the ? with the value.

Of course we want our Database libraries to be good enough to not require writing custom SQL, but for some advanced usecases the users may have to write custom queries and then amenities like this could prove very useful.

[wdanilo]

@jdunkerley how these literals are better than writing

foo = date "2012-01-03"
bar = time "12:00"
baz = date "2012-01-03 12:01"

We can even make it more special and return a correct type based on the string, like:

foo = time "2012-01-03"
bar = time "12:00"
baz = time "2012-01-03 12:01"

TBH I really hate prefixes to literals like d, t and dt. They are obscure and hard to remember. Instead of them we can simply expose functions that do what we want. Maybe I don't see some use case here – if so, I'd love to know it.

[GregoryTravis]

@JaroslavTulach Improved concise syntax is a benefit you get from a custom parser, and the improvement works equally well in functional and non-functional languages. Improved performance is not the goal.

[wdanilo]

@GregoryTravis I don't understand your point. Could you give an example (preferably in a pseudo-code) where literal would make the code better / easier to understand that by using just a function?

[GregoryTravis]

You could write r"\r\d" instead of r"\r\d". It's a small difference, but regexes are already hard to write.

[jdunkerley]

A similar approach to Date can work, if we have Regex.new (I think it is compile at the moment but could match the others) have a special text box where the input is like regexr.com we would be able to work with that.

text1.find_all (Regex.new "\b[a-z]")

This avoids the literals - the only thing I don't like is that it leads to more extra code on nodes versus the smaller

text1.find_all /\b[a-z]/

which feels a natural syntax for Regex.

[wdanilo]

Oh, regarding regexes, I'm very OK with adding their support to the parser. Unlike literals for date, a special syntax for regex is very common and very widely used, so I think this is a good idea! Sorry for not being clear about it earlier.

[JaroslavTulach]

support a regex literal

Regex is usually problematic because of all the escape characters it needs and that clash with escape characters used by C-like languages.

use_regex: If true, the term is used as a regular expression.

Why don't you type the term argument as Text | Regex and remove use_regex instead?

syntax highlighting within the IDE

Enso already provides support for foreign functions. Rather than complicating lexing, parsing, Tree, IR, engine & IDE I'd rather hide support for regular expressions to foreign regex blocks which are a natural and (relatively) easily supportable embeddings of strange syntaxes into Enso. I do realize I am looking at the problem from implementation perspective, however.

[jdunkerley]

indeed the intention is exactly to take the Text | Regex.

As you said we should embrace functions so perhaps just using to_regex will work. Not sure we could syntax highlighting there but we could do with Regex.compile.

[JaroslavTulach]

using to_regex will work.

It'd rather be to Regex, to be in line with

• Use .to Text instead of .to_text and Text.from #5129

[jdunkerley]

at the moment to conversions like that don't work so currently we have to_xxx

[JaroslavTulach]

at the moment to conversions like that don't work

What is the problem with to conversions? I am not aware of any related open issue:

• to conversion is not working for builtin types like Text or Number #4879 seems to be closed as resolved

I am not aware of any other problem (and I did the evaluation at the end of last year myself). I have to assume that to conversions like that do work!

[jdunkerley]

No IDE support and I don't think it can discover all the conversions at the moment. Will have another experiment when have some time.

[jdunkerley]

The motivation behind this is to make user experience trivial.

Let's just consider the user wanting to enter a date constant. I imagine them going to the plus button and then choosing a "Date literal" from the list.

This should insert something like Date.new 2023 6 9. This would then need to be edited and chosen using a single calendar picker item.

The thoughts around having a literal like date"2023-06-09" were that this would be easier for the IDE to identify and switch to a bespoke widget, but we could achieve this on the whole function.

The same experience is also needed for the other literal types. If I am entering a regex, I would love the regexr style experience where it syntaxes highlights and helps explain.

As @JaroslavTulach, we can us the functional language style for this but would need to think clearly how this was widget-able.

This is also very true for Vector, Map and Table literals which again really need nice UI experiences.

[wdanilo]

As you mentioned, James, there will be a ton of such examples, including vectors, tables, maps, etc. Creating a special vexing literals for every such example is rather not feasible.

I don't fully understand the problems with widgets that you mentioned. Our current design is that we have an association between types and widgets. So if GUI gets information that "this fragment of code is of type date", then it will display the date widget. The same applies to other widgets. In such a case, widgets should just work for Date.new 2023 6 9 or Date.from "2023-06-09" by default (I believe we should allow for string-based date insertion method). Even if we allow 10 different ways to create a date, all of these expressions would result in the "Date" type, which should be communicated to IDE and IDE will display the right widget – this is way better solution then relying on some kind of special lexing form.

Of course, here we need to talk about one additional issue. For now, we have an agreement that if the IDE sees a function usage, like my_var.foo, we get info about its signature and expected types, so we can display all widgets there. I don't think we have a similar agreement regarding e.g. lambda bodies. For example, given my_var2 = my_var . map (t -> Date.new 2023 6 8), I think GUI does not get AST info right now, that the code span Date.new 2023 6 8 resolves to a Date type. Of course, we don't need it now, as there are no widgets inside of lambdas, but if we want to show widgets there, the only thing that GUI would need is a type information associated with an AST span.

How does it sound?

[jdunkerley]

The problem I meant was currently we display widgets for arguments.

So in this case Date.new has 3 integer arguments, as currently envisioned we would get 3 sliders (year, month, day) with maybe some cleverness to link days to month.

If the IDE can indeed recognize a Date input function and become a calendar picker that would work (though this would seem to be a large distance away). Additionally, how would we identify input dates vs calculations (I guess an attribute of some form).

The literal idea seemed like it would be a trivial item for the IDE lock onto and provide and input for.

Happy if we can identify all generator function and make work that way.

The lambda AST is a bigger issue and already comes regularly for the advanced users - that being said I would push for enhancing the grouped functions a lot before we push on the lambda as they are much simpler for our target users.

[wdanilo]

So, here are a few things to note:

1. If we have an expression like v2 = v1.fn (Date.new 2023 1 2) then the whole Date.new 2023 1 2 is an argument and becomes a widget (like a date picker).
2. If the node is Date.new 2023 1 2 then we should have in GUI a way to discover that the whole node is actually of a type Date and show it as a widget instead of widgets for its fields.

I don't see a bigger difference between points 1 and 2. CC @Frizi is it doable?

Regarding when such a date-chasing widget can be implemented, I believe it's sooner than later. We can glue some HTML pop-up date widget there to do it.

Regarding literals – they would cause a lot of trouble. For example, why a literal would show a widget, why other fors of the code would not? If I write Date.new 2023 1 2 instead of using a special literal, would I not see the widget? That is confusing. And what's worse, this would be the same story with all sort of widgets, as geo-maps, etc. In the end, we would end having tons of different literals and all users would need to remember to use them instead of using normal functions in Enso. Anyway, I believe we should simply recognize the types of variables in GUI and display widgets for them.

Regarding lambdas and grouped functions - 100% agree, lets do it this way :)

[jdunkerley]

Happy to recognise Date.new as the "literal" format and have the widget defined.
I think it is fine to only attach to the Date.new.

Happy to say we don't do literals for date/time types then as that makes sense. The goal would then be we have widgets for Date.new, Time_Of_Day.new and Date_Time.new. Additionally, these would show as specific entries in the CB much like number and text do.

[jdunkerley]

So to summarise.

We will add a new literal syntax for Regex (for example, /\d+/) as well as the previously planned back tick for expressions. Eventually, the IDE should be able to provide syntax colouring for these but that is a longer term task.

The IDE will recognise Date.new, Date_Time.new and Time_Of_Day.new and provide specific widgets to allow easier entry.

One area we may want to examine again later is string interpolation but that is another topic.