autoformatting: reflow multi-line comments and strings

[shwestrick]

Currently, the pretty-printer doesn't handle multi-line elements very well. See example below.

This is a little tricky because in the layout algorithm, each text element is assumed to not contain newlines. To fix this, we should probably just do special-case handling of comments and strings, perhaps as a separate pass over a TokenDoc.

Example of bad formatting

INPUT

val x = (* this is a
         * multiline comment
         * which needs some love
         *)
    "I wonder where \
    \the rest of this string will go?"

OUTPUT

val x =
  (* this is a
         * multiline comment
         * which needs some love
         *) "I wonder where \
    \the rest of this string will go?"

[shwestrick]

We now correctly handle the example given above. The current output is:

val x =
  (* this is a
   * multiline comment
   * which needs some love
   *)
  "I wonder where \
  \the rest of this string will go?"

[shwestrick]

I believe this is now correct as long as every tab in the input is properly aligned.

(By "properly aligned", I mean that tabs appear only at multiples of the tab-width. For example with a tab-width of 4, the line \t1234\t12345678\t123 is aligned, because the tabs appear at positions which are multiples of 4, but the line 123\t12345678 is not.)

What is the effective spacing of a misaligned tab? Maybe we just need to advance to the next aligned position? This seems to be the way VSCode handles it. Same for other editors too?

[shwestrick]

I finished this up by advancing to the next tabstop. Tab widths are controllable via -tab-stop <int> arg at the command-line.