Reimplementing event highlighting in tidal-listener

[Zalastax]

I have been looking into how to implement event highlighting in mindofmatthew/text.management#33. I have understood from yaxu's explanation how deltaMetadata can be used to offset the position information included in the OSC events. But I believe that all ways that this can be currently used would not be satisfactory. I believe part of the solution is to add some meta ID, separate from the pattern ID, for editors to know which place in the code that is being referred to. Another aspect is to make tidal-listener inject deltaMetadata automatically, which used to be done in deltaMini.

One way to use deltaMetadata would be to calculate the absolute position of the mininotation string within the editor. This would make highlighting easy in the editor: just highlight the position included in the OSC event. As long as we never modify existing lines and only add new code, this would actually work! But as soon as we update existing lines, e.g. add a new line on top, the absolute position of the mininotation string changes. I therefore suggest that deltaMini should only ever be used with relative positions.

If we use relative positions, we need to include an ID in the event to know which section of the code that the position is relative to. That way changes outside this section may update the absolute position but the relative position within the section will remain constant and the editor knows how to calculate the position in the editor by keeping track of the section.

tidal-listener used to accept /code <id> <source> messages so that the editor could include such IDs. Combined with including id in OSC events (added in d15a87e and 5f0fa2b), the editor could be able to know which section is being referred to. But the from the editor was used by tidal-listener as the pattern ID and the needed to evaluate to a ControlPattern. This puts efforts on the editor to parse the line and figure out what is the pattern ID and what is the ControlPattern before sending it. This also limits what code can be sent from the editor. For example, we can't use do-notation to send two patterns. I don't think the pulsar-tidalcycles implementation does this correctly (see https://github.com/tidalcycles/pulsar-tidalcycles/blob/207bb39e4c7e1789582cf35550002befee10b83b/lib/tidal-listener-repl.js#L88 + https://github.com/tidalcycles/pulsar-tidalcycles/blob/207bb39e4c7e1789582cf35550002befee10b83b/lib/editors.js#L44 where only getWholeEditorExpressions seems to create the needed id and I can't see how the created ID is correct)

The change away from /code <id> <source> to `/eval in tidal-listener (done in 33b5d75) is good. It means we are no longer limited to sending only code of type ControlPattern and this moves us toward a better separation of concern between tidal-listener and the editor.

As mentioned in the beginning of my text, two changes are needed to tidal-listener to re-enable event highlighting: Add a meta ID and inject deltaMetadata automatically.

We should add a new meta ID (not pattern ID) so the command becomes /eval <meta-id> <source>. The meta-id would be used as mentioned above - to provide a reference for what section the OSC Event refers to. The editor should probably generate this meta-id randomly.

Unfortunately, I don't know a great way for tidal-listener to inject deltaMetadata. yaxu explains that the current best approach is to assume that "" is always mininotation. It would be neat if there was some way to ask Hint about the source code to identify the mininotation locations - e.g. if it was possible to query the type information of the code to be evaluated (including the type of sub-expressions). Any modifications that could be made to Tidal itself to make this easier to implement? E.g. template Haskell (#756) or changes in how mininotation is written / parsed? Could there be some way to make the evaluation in tidal-listener create a data-structure where the mininotation locations are returned, inspired by lisp "homoiconicity"?

People that I think are relevant for the discussion: @yaxu, @mindofmatthew, @polymorphicengine

[Zalastax]

The discussion in haskell-hint/hint#85 seems relevant. If we could use Template Haskell, then we could be informed of what string lierals are Signals or Sequences. According to that issue, the best approach would be to pretty-print the Template Haskell output and run that in Hint.https://stackoverflow.com/questions/31412444/template-haskell-is-there-a-function-or-special-syntax-that-parses-a-string-a also seems relevant.

Another idea would be to look for string lierals ("..."), optimistically inject deltaMetadata and if it fails to compile then we revert back to evaluating the original string. This seems like a simple start.

If I get an OK for it I would like to go ahead with an implementation based on optimistically injecting deltaMetadata and adding a meta ID to keep track of the section. The meta ID would also need to be injected via deltaMetadata (or a sibling function).

[mindofmatthew]

I'll mention that @reckoner165 is currently working on Tidal highlighting for text.management, and I think we'll get it working with current Tidal, but it's not a great implementation. Our plan is to use the "row" element of the deltaContext position as just an incrementing unique ID assigned to each mininotation string, and then using the relative offsets within that string. This covers the case where the mininotation string moves around, but even the relative offsets break down when the string itself is edited. At that point, I think we're going to declare the mininotation "dirty" to the editor and disable highlighting for it.

I'm all in favor of you going forward with an implementation! Looking at it from the point of view of an editor, here's what I think would be ideal: First, the editor sends over a code string to be evaluated. There would be a single "evaluation ID" attached to this (which would also be useful for labeling error responses etc). Second, the listener does its parsing, finds the relevant code ranges (either of mininotation strings, or ideally, individual mininotation symbols) and then sends back the full set of mininotation ranges (relative to the evaluated string), and a unique ID for that range. The editor can pull this information in, store internal annotations for all of these mininotation strings, and keep track of moving them around an edited document. Finally, when highlights happen in real time, Tidal sends highlight messages that refer to the ID of the mininotation (either the whole pattern or the symbol within that pattern).

Strudel does a version of this currently. Right now, when the code is evaluated, the parser does a single pass and extracts every mininotation symbol, assigning each one an ID. Once these IDs are associated with the code positions at evaluation time, the Strudel patterns just deal in terms of these IDs.