dynamic import in REPL + implicit mini notation + patternable aliases

[debrisapron]

I want to import code from e.g. jsdelivr in the REPL but I get an Unexpected token "import" message. Does the JS parser the REPL uses not support import? This would be super helpful for prototyping plugins for Strudel without the whole ceremony of forking the repo, creating a dev environment etc etc.

[felixroos]

imports are not supported (yet?). We use shift-ast, which supports import, but for that we need a custom implementation of loading scripts on the fly, which we do not have yet.

There was a discussion recently about loading scripts: #153 (search loadScript) Here is an example of loadScript: https://strudel.tidalcycles.org/?TO3z8wnvE2p3 it's just an eval so no full blown module importer. I haven't yet dug into how a module loader could work client side (yet), any help / tips are welcome

In case you want to do the ceremony, here is a setup guide: https://github.com/tidalcycles/strudel/blob/main/CONTRIBUTING.md#project-setup

[debrisapron]

I've actually done basically this exact work in another REPL-type project, would you want a PR?

[felixroos]

yes please! which types of modules did you manage to load with it?

[debrisapron]

Any valid native JS module works fine. Like you could do something like import * as strudel from "https://cdn.jsdelivr.net/gh/tidalcycles/strudel/main/packages/core/index.mjs" and you should be good to go

[felixroos]

that would be nice! feel free to ask any questions that may arise

[debrisapron]

OK so after further investigation this is not really possible with the current REPL. At the moment, the shapeshifter.mjs module in @strudel/eval makes a lot of mutations to the AST which, while I'm sure it makes for a fast & sexy live-coding experience, breaks a lot of javascript features. For example, currently all string literals are run through mini, which obviously breaks any kind of import function, because URLs are strings.

Off the top of my head, here are a few possible solutions:

1. Get rid of the autominification magic completely and just use a tagged template literal, e.g. something like:

m`[B3@2 D4] [A3@2 [G3 A3]] [B3@2 D4] [A3]`.transpose(1).whatever().blah()

This is still pretty terse & doesn't break JS semantics.

2. If we really really want to keep the magic, maybe we could run a check on the string to see if it's valid mini notation or not, and only minify it if it is. This will have weird side-effects when the user does something with a string that happens to be valid mini, e.g. console.log("my favorite band is " + "abba") but it will still work most of the time. Another problem is this means we can't display syntax errors to the user because bad syntax will just mean it doesn't get minified. This is actually a terrible idea, but imma leave it here anyway.

3. Have two separate REPL modes, "livecoding" mode with all the weird substitutions & whatnot and "compliant" mode, which is just plain JS without any magic.

4. Go the JSX route and add new syntax. For example, if we minified everything between &: and a dot:

&:[B3@2 D4] [A3@2 [G3 A3]] [B3@2 D4] [A3].transpose(1).whatever().blah()

Personally my vote would be for option 1 because I'm a JS dork and don't like people tampering with the crystalline perfection of the One True Language, but I realize this may be a minority view.

[felixroos]

You can still use single quotes for ordinary strings, so you can still write every possible javascript program!
Even though your points assume strings are not possible, I still want to answer them:

1. we already had the literal thing and it's nice, but still a bit annoying / unreadable when you have a lot. also backticks are a pain to write on some keyboards
2. most strings are valid mini notation so not much to win here
3. this would be a good thing to have. it could work with a magic comment like // strudel disable-sugar to skip the shapeshifter
4. Having invalid javascript to start with creates some problems: no syntax highlighting + another string manipulation step. Also, your example snippet is not really shorter.. (edit: just noticed it's the same length) If we go down that route i would say a full blown DSL makes more sense
   4.1 it could be interesting to try using actual JSX

Another path that can work is to parse strings with mini by default in all strudel functions, so the string magic is not needed. I am not a huge fan of this idea because you cannot use all these characters: * / : < > [ ] { } | and space, as they are used by the mini language. So you cannot use 'C minor', because you'll get seq('C', 'minor'). It can be worked around by adding replacements for special chars, like _ for space etc.. but this can get confusing quickly

[yaxu]

FWIW I think it makes a lot of sense to have easily tokenisable chord labels. I haven't seen the convention with spaces before and find it difficult to read. I think it would be great if strudel's chord labels would match Tidal's, especially if that involved improving Tidal's representation of chords.

[felixroos]

I was talking about scales, not about chords. Using spaces for scales is quite common, like 'C minor', 'G mixolydian' etc.. The names are referencing tonaljs scales, where even the scale names itself can have spaces.

Chords are currently implemented in the form of voicing dictionaries, using the chord-voicings package. The chord symbols are in theory freely configurable although the configuration is not (yet) exposed through strudels voicings API. Having them freely configurable gives the user the freedom to use any spelling.

But of course there should be a good standard set, tbh I do not really like using chord symbol names that are constrained to use the format of variable names (like tidal does afaik). One widely used format for chord symbols is this (check out the ireal forum, there are thousands of chord changes using that set of symbols, at least if you know how to decrypt the links). It also pretty closely resembles the way many musicians would write down the chord on a lead sheet. But this does not mean we still can support the chord symbol set of tidal (for tidalists that are already familiar with those).

All of the above might be worth a separate discussion, so coming back to the question if mini notation should be parsed at all times (eliminating the need to shapeshift double quoted strings):

• The fact that JS has two different notations for strings enables us to use one for mini notation, while having the option to freely use any plain old string anywhere in the program
• in case you want to use strudel without shape shifting, you can always use the existing m`` format.
• if this really turns into something people want, we could add a config option / magic comment to turn default mini parsing on

In a broader sense, this is not only about being able to use scale names with spaces, but about having the freedom to use strings freely for anything (not only music). It just feels like too big of a cost having to work around the limitations of the mini notation (aka don't use any of * / : < > [ ] { } | ' ' in your strings).

edit: @yaxu what do you mean by "easily tokenisable" and why is it that important?

[yaxu]

Ah sorry I get mixed up between all these different collections of notes ! I guess I mean then that it would be nice to have scale names that don't have whitespace in so that they can be patterned with the mininotation. If all arguments are patterns then things are a bit simpler.

[debrisapron]

OK I didn't realize that the string substitution was only for double quotes! This is a great example of where knowing JS is actually a bit of a disadvantage with this app LOL.

So I think we agree that adding a desugaring directive of some kind is a good way to go. How about /* strudel-strict */? This is inspired by the eslint standard where ignore directives in // comments apply only to the next line and directives in /* comments apply to the whole file, so it leaves open the possibility of single-line directives in the future.

[debrisapron]

Although I admit the combination of "strudel" and "sugar" is rather pleasing...

[felixroos]

No worries, in the end, scales and chords mean the same thing.. My argument is still that we would introduce a big limitation by parsing all strings. Let me make my point clear with some examples:

I would rather write

cat('C minor pentatonic', 'F minor pentatonic')
than
"<C'minor'pentatonic F'minor'pentatonic>"

Also, what about slash chords:

If everything is parsed, writing seq('E-/C') is not possible, so should we write seq('E-\/C') ?

What if I want to make flashy html visuals, which I could do now with:

randcat('<strong>hello</strong>', '<i>italic</i>','<marquee>scroll!!</marquee>')
vs
'\<strong\>hello\<\/strong\> | \<i\>italic\<\/i\> \<marquee\>scroll!!\<\/marquee\>'

What if I want to use a microtonal ratios similar to xenpaper:

seq('1/1', '9/8', '5/4', '7\12')
vs
'1\/1 9\/8 5\/4 7\\12' ?

What if I want to use any alternative DSL inside a pattern function?
How do we escape the special characters?

[felixroos]

So I think we agree that adding a desugaring directive of some kind is a good way to go. How about /* strudel-strict */? This is inspired by the eslint standard where ignore directives in // comments apply only to the next line and directives in /* comments apply to the whole file, so it leaves open the possibility of single-line directives in the future.

Good point with single line rules..
There are already the (eslint inspired) magic comments strudel disable-highlighting, strudel hide-header, strudel hide-console, so I would propose:

• strudel no-sugar
• strudel sugar-free
• strudel explicit-mini <- my favorite

I also like the sugar reference but maybe that category is too broad, because the syntax sugar involves more than just double quote to mini, and I think each thing should be controllable separately. There could also be an option that disables everything, but e.g. top level await and highlighting are desirable in most cases

[felixroos]

speaking of, we could also add strudel always-mini or strudel implicit-mini to parse strings at all times

[felixroos]

explicit mini functionality is already be implemented in #99 (minifyStrings flag)

[yaxu]

You could do:

randcat(pure('<strong>hello</strong>'), pure('<i>italic</i>'),pure('<marquee>scroll!!</marquee>'))

perhaps we could make 'pure' versions of all the functions

const randcatpure = function (...args) {return randcat(args.map(pure))}

[felixroos]

hm then we would need pure versions of each function and each method..
I still don't understand why the current behavior is problematic at all.. The rule double quotes = patterns, single quotes = strings is quickly learned, while needing to remember special escape symbols + workaround functions is not really handy.

Using strudel without the shapeshifter as it works now requires you to use just 1 more character (m) for mini patterns, which is not that bad. If we really wanted to trim down that character for that use case, we could add some flag to the core package, that will run each reified string through mini. But I also don't really understand why the shapeshifter would be a problem, if things can also be turned on and off within the eval package.

[felixroos]

one thing that the *pure functions cannot do: seq('C/G', "<C F>") would need seqpure('C/G', m('<C F>'))

[felixroos]

we could also flip things and add *mini functions like randcatmini seqmini .. but that still adds a lot. to be consistent, we'd also need .fastmini .slowmini etc... (same goes for the pure versions)

[yaxu]

Yes I think the "/' difference is fine too and don't really think making pure versions of everything is the answer, just thinking aloud.

I still think it would be better if labels for scales etc had simple enough identifiers to be able to be patterned with the mini-notation though.

[felixroos]

I still think it would be better if labels for scales etc had simple enough identifiers to be able to be patterned with the mini-notation though.

Totally agree, though I'd say we should not take away the more conventional but less patternable aliases

[debrisapron]

OK well it seems like the directive is on it's way so I'm gonna leave that tangent alone. And now that I know I can use single-quotes for normal non-magic strings, I think I have a simple fix for the import issue...

[yaxu]

Totally agree, though I'd say we should not take away the more conventional but less patternable aliases

I don't see the value of them really - if an alias is less patternable, then that creates a barrier to change.. You have to edit the label if you later decide you want to pattern it. Their presence in examples makes strudel look less like a system for patterning and more like a conventional step sequencer.

[felixroos]

ok here's an another idea: what if the mini notation had a high precedence character that marks a 'parse free zone', like:

"0 2 4".scale("'C major' ['D dorian' 'G mixolydian']")

With that, you can use any character you like inside ' ... '
The other examples from above could then all be patterned in a readable way:

"'<strong>hello</strong>' | '<i>italic</i>' | '<marquee>scroll!!</marquee>'"

"'1/1' '9/8' '5/4' '7\12'"

With that, we won't need to rewrite any data that needs to be patterned. I also like how the single quotes still mean 'ordinary string' wether inside or outside double quotes

edit: it's funny to think about strings in programming languages as "a high precedence character that marks a 'parse free zone'"

[yaxu]

Yes that makes sense. However single quotes are used to separate parts of a chord label in tidal's mininotation e.g. c'minor'ii - I think we really need to standardise on chord labels.

[felixroos]

Hm I cannot think of other symbols that make sense for strings other than double quotes, backticks and single quotes.

• Double quotes are problematic because they are already used to delimit mini notation itself. The only way to use them would be to flip the meaning of double and single quotes, using single quotes for mini notation, like '"<strong>hello</strong>" | "<i>italic</i>" | "<marquee>scroll!!</marquee>"' This seems like too big of a change..
• Backticks are the only way to do multiline strings in javascript + used in tagged template literal calls, so they would need to be escaped in those cases + they are somewhat annoying to type on some european keyboard layouts

I think we really need to standardise on chord labels.

I suppose it would not be an option to change the ' for chord labels to another symbol (or an alias)?
Or just spaces? With a generic string notation, chords could be written like 'c minor ii' (same character count)
But of course, that would break compatibility with tidal, at least if tidal does not change it too, which would be a breaking change.

So far I have no other ideas, only a strong motivation to somehow get strings into the mini notation, because:

• patterned values do not need any transformation to fit inside mini notation (see examples above)
• we could then really parse every string as mini by default, not needing shape shift double quotes only, because we can express any value without limits

[yaxu]

Yes changing tidal is definitely an option! It's good to save up breaking changes and break them all at once with a 'major' release, but we haven't done that for a while. @polymorphicengine has been working on chord parsing recently and might have opinions to share on this.

Chords as quoted strings would mean you can't pattern the different aspects of a chord, currently in tidal you can do "c'<major minor>". So while I can see strong motivation for freeing up ' for quoting strings in the mininotation, I don't think chords are necessarily a good use case for that.

Mininotation has list/vararg syntax delimited by :, like sound "bd:<4 5>". IIRC it's a bit hacky and has some limitations that could do with fixing (ref tidalcycles/Tidal#233). But I think it could make sense to switch chords to use this as well note "c:<major minor>" (which leads to the question whether sound "supersaw:c7" etc should work.. It doesn't atm)

[felixroos]

with #502 scales can now be written as "C:major" using ":" instead of space. As proposed in #504 colons could be used to add patternable custom behavior to any function. So wouldn't it make sense then to use ":" also for chords when this is ready?

[yaxu]

Yep and #504 is done now

[felixroos]

just realized that implicit minification is already pretty simple to do:

https://strudel.tidalcycles.org?lNxLnMcpieR3

[felixroos]

@debrisapron imports are now live https://strudel.tidalcycles.org/?7DsklCccTr-k 🎉