Component: Language and Interpreter

[StachuDotNet]

This issue is to organize work related to Darklang's language and interpreter.

Most of these tasks are ready to be worked on, but may require some brief discussion up first.
Interpreter work can be challenging, but fun and rewarding!

Numbers

• support infinite-precision integers, and probably make them the default (see old Infinite precision integers should be available, and possibly the default #3279)
• deal with JSON and integer size. Problem: when we coerce integers to/from JSON, many JSON impls do not support ints larger than 63 bits (i.e. Twitter API has "id" and "id_str" fields because sometimes the "id" is bigger than 53 bits)
  • relevant strategies to consider:
    • integer conversion into JSON should use a string if appropriate
    • integer conversion from JSON should always be typed, and so if there's an int it can be parsed from a stringified integer literal if appropriate
• Make numeric operators reasonable
  • Float operators are not yet supported
    • Dark's operators (+, -, *, etc) work on integers. In Dark v1, we use Float::+ instead, which doesn't suck but isn't great
    • Non-solution: We speculated that we could use the editor to simply hide the Float:: part. However, that doesn't allow polymorphism, you can't have a library that takes numbers of any kind and (for example) sums them.
    • Solution: use traits to support reuse of common operators for different types
• Remove Inf and NaN where possible (see Remove Inf and NaN where possible #3306)

Strings and Chars

• review how well we handle escape characters
• allow inserting unicode chars via escaping (see allow inserting unicode chars via escaping #3288)
• string length is currently determined in O(n) time. This value should be caches as part of the string. Using a better string impl would help solve this.
• Review support of escaping
  • Ensure common escapes work: \n, \r, \t
  • escapes that are needed for the text to work: \\ ,\'
  • allow a specific byte: \xhh (hex escaping)
  • could possibly allow octal escaping too
  • escape sequences should be clear to the user (a different color)
  • escape sequences should have a clear doc explaining how it works and what the user is looking at

match expressions

• union pattern -- multiple patterns, one RHS (match x with | X | Y _ | Z -> ...)
• additional patterns -- see What are additional LetPattern and MatchPattern cases that'd be useful? #4714

Collections (Lists, Dicts, etc.)

• (worth discussion) Interpreter should error when a record or dict literal has the same key twice (see old Interpreter should error when a record or dict literal has the same key twice #2533)
• fix: dictionaries always use strings for keys
  • Add support for dictionaries which have keys of other types
• Support Sets
  • Dark should have a set type, including a literal syntax.
• implement lists/arrays using immutable vectors
  • our implementation of Lists uses f# lists, which are slow. The same is true on the client, which uses rescript lists which are also slow.
  • however, arrays -- a natural thing to use instead -- are not immutable.
  • find an immutable vector type, benchmark operations with it, and switch over to it.
• support list cons operators (single element and combing 2 elements)

UUIDs

• Uuid shouldn't be a special type
  • UUID is a special type in the runtime, not a type defined using the type system. We'd like to get rid of special types and this is one of them.
    • Add a Uuid type to the standard library, that is perhaps an alias for a uint128, or maybe uses bytes to define it.
    • convert all builtin code to use it
• Generate different versions of UUIDs
  • We have a function to generate UUIDs. We should have different functions to generate UUIDs of v1-v6, as they all have different uses.

Other (unorganized)

• Add an id to all lambdas in StdLib, ensure errors link properly in client
• Support versioned infix operators / hide latest version (see Support versioned infix operators / hide latest version #2673)
• given a generic record type type OptionRecordWrapper<'a> = { opt: Option<'a> }, I'd expect to be able to provide the type args when creating the record with OptionRecordWrapper<Int> { opt = None } but can't
• records as constants
• do we have full/proper currying support? if not, that
• Support RegEx (see Support Regex #3415)
• defining a local fn within a fn or expr
• Traits (see Traits #3451)
• Support imperative algorithms / mutable values
  • Dark should support a mutable value so that users can write imperative algorithms when functional ones are too difficult to write (which is often)
• References in ASTs should be by IDs rather than Name Reference DBs, UserTypes, and UserFunctions in ASTs by ID rather than Name #3964
  • DBs, UserTypes, and UserFunctions
  • Functions, DBs and types are all referenced in the AST by name. Instead, we should refer to them by TLID. This means they won't go out of sync, they're faster to look up, and they're less likely or impossible to accidentally refer to a non-existing one. This will also mean that renaming them won't require changes to other toplevels (a key way these go out of sync)
• Decide how to handle arithmetic errors (e.g. division by 0)
  • see discussion in Decide how to handle arithmetic errors (e.g. division by 0) #3282
• allow functions to take optional params that have default values to be used if corresponding args not passed in
• we need non-string references
  • secrets
  • dbs
  • functions
  • http endpoints