ALL STILL IN THE BRAINSTORMING STAGE - SUBJECT TO CHANGE!
Implementation can be part of TScript34, even of TScript34-P0019. Want a name that is separate because it could be embedded in other projects.
See also JavaCommandRunner36 (JCR36), which can run individual commands and is based by some similar ideas.
- [X] Come up with name
- TOGSOFT-whatever is fine
- “TOGSOFT-41” or “TScript41”
- [X] Make a repo for it
- [/] Document syntax
- Sketched out below
- [ ] Write some test cases
Simplified from TCL or Bash:
- No `uplevel`
- Simplified tokenization rules
‘Everything is a string’ because out in the ‘real world’ byte sequences are the lowest common denominator. So let’s make ‘how to interpret this string’ explicit and conversions very easy. Interpretations can be cached.
(Actually, everything is a stream of bytes. But finite, pre-generated byte streams are the building blocks for scripts.)
Functions act as if they launch processes.
Process can do I/O and return an exit code.
Normal ‘result’ of a procuss is what it writes to stdout
($(SOME-COMMAND FOO BAR) represents ‘the output of running SOME-COMMAND FOO BAR’).
I’d like the syntax and semantics to be cleanly seperable.
e.g. "foo ${bar}" and (cat data:,foo env:bar) may be equivalent.
Long, unambiguous names for built-in commands. Can be aliased, similar to in TS34 or JCR36.
The language is relatively ‘stateless’; variables (which are equivalent to environment variables) do not change value within a block. They can only be overridden for a particular scope.
$#REST is syntactic sugar for “the rest of the block, as a string”
Like with shell scripting, environment variables and commands are separate namespaces.
8** Everything’s a string, but strings have structure
TODO: Talk about concatenated/encoded strings being a basic building block;
e.g. how the expression --foo="bar baz" can be considered to represent a single
string, --foo=bar baz, but underneath it may be represented as the concatenation
of --, foo, =, and bar baz, each of which are connected back to the place
that they were defined (or maybe multiple places, e.g. in --foo=$some_var,
the $some_var part of the resulting value could be linked to multiple
source locations: the place where $some_var was referenced, the place
where it was defined, etc.
Ideally these string pieces could track
their source even when encoded/decoded, so something like
foo="hello, world!" print-source-locations-of $(base64-encode $foo)
could indicate both the place in the source where the value was encoded,
and the place where the encoded value was originally set.
To do this, each source location would include some information like
type SourceLocation = {
(fields for file, line number, etc)
derivedFrom : {
source: SourceLocation # source of the value
encodings: Encoding[] # encodings that were applied to the value
}
}
Or maybe that’s overthinking it. If values are always represented as expressions, and expressions have source locations, then it’s as simple as
type ValueExpression = {
sourceLocation : SourceLocation,
type : "Expression"|"Concatenate"|"Base64Encode"|"Base64Decode"|...
...
}
where ‘Expression’ expressions simply reference another expression, so you can track ~foo=Hello do { bar=$foo do baz=$bar echo ~
(do
TODO: Define data structures that are needed to describe programs, then define one or more encodings.
TODO: A general way to
’#’ followed directly by non-whitespace is a tokenizer/parser directive.
’#’ followed by whitespace starts a line comment. ‘#’ can be used after non-comment tokens.
Newline can be considered a token, equivalent to ‘;’
' token means, for statement-parsing purposes, skip all following newline and ‘#’ tokens, with the effect of continuing the preceding statement.
Tokens can be barewords, double-quoted or brace-enclosed strings, or variable or sub-expression references (started with ‘$’).
grumble grumble, something about different contexts. ‘list-like strings’ are the common format used by commands, scripts being a series of those, separated by unescaped newlines.
Reserved syntax
$foo$*foo- the following expression is to be expanded rather than be treated as a single word
"foo"- double-quotes enclose text to be treated as a single item
{foo {bar {baz}}}- braces are nestable strings with no escape codes
(foo (bar baz))- parentheses are nestable strings that tokenize
their contents using the common ‘list-like string’ tokenization rules;
they are similar to curly braces but can contain unbalanced parentheses
within substrings –
(foo ")")evaluates to the stringfoo ")".
‘Everything is a string’ in a lnguage that often needs to to talk about lists and expressions means that it’s important to standardize the encoding of those things!
In attempt to be quote-semantics-compatible with other languages in this family, guillaments work similarly to braces, with slightly different semantics for each type.
Double angle quotes are equivalent to braces.
No single-quoted strings. Normally I like to reserve them to have the same syntax but different semantics to double-quoted strings, but this language, being based on shell syntax, uses ‘$’ to reference things by name.
Directional double quotes would be like regular double quotes, but nestable. This could lead to surprises abot where escaping happens, so I will not support it, for now.
# is also used as the prefix for some special variables.
$#REST is the rest of the script, which is useful when you want a
new context but don’t want a new level of indentation in your script.
The built-in ‘cat’ command, http://ns.nuke24.net/TScript41/Commands/Cat, expects arguments to be URIs, and will resolve them.
Only arguments that are unambiguously not URIs, including Windows-style paths with a single drive letter where the scheme would normally be, will be treated as file paths.
#lang tscript41
#{
A block comment. Really a variation on '{' where the token is skipped over.
#{
Block comments can be nested.
}
}
#(
A block comment using list-like string tokenizatoin rules
)
http://ns.nuke24.net/TScript41/Commands/WithCommandAliases {
{echo http://ns.nuke24.net/TScript41/Commands/Echo}
{with-aliases http://ns.nuke24.net/TScript41/Commands/WithCommandAliases}
{with-env http://ns.nuke24.net/TScript41/Commands/WithEnv}
} $#REST
# Double-quoted strings allow for variable/expression substituion and backslash escapes:
"foo ${var} $(expression) \n"
# brace-enclosed strings are nestable but have no escaping mechanism,
# similar to those in Tcl.
{literal {nestable} quoting style}
# Different sub-expressions:
echo $(expression) # can use $ outside of quoted strings; they become a single argument
echo $variable # don't need '{}' outside of quoted strings
echo $*variable # '$*' instead of '$' means 'splat' the variable or expression result.
echo $*{variable} # Same
echo $*(echo foo bar baz) # equivalent to `echo foo bar baz`
# The braces in '${...}' have the same parsing semantics
# as when used for literal strings; everything between is
# literally part of the variable name:
echo ${foo {bar} baz} # References the variable called 'foo {bar} baz'
# Sh-like '=' operator is syntactic sugar for 'WithEnv',
# to set the values of variables as seen by a sub-command
foo="bar" baz="quux" {echo $foo}
foo="bar" $#REST # with foo=bar, execute the rest of the current block